visual_search MCP Tool

The visual_search tool searches the merchant’s product catalog by an image. Supply it as image_url — a publicly reachable HTTPS URL — or as image_base64, the bytes in standard base64, but not both; the whole request must stay under 2 MiB. Such a call is a complete search on its own and reports the objects detected in the image alongside the results.

To narrow those results over several calls against the same image, generate a UUID and send it as session_id on that first call. A call repeating that session_id is always answered from the image the session was opened with — omit the image there; one sent under a live session is ignored. A different image requires a new session_id. This tool answers under any authenticated credential, including the site’s read-only token.

Request (VisualSearchRequest)

Field	Type	Description
image_url	string or null	Publicly reachable HTTPS URL of the image to search by. Supply at most one of image_url or image_base64. A call that carries a session_id already in use is always answered from the image that session was opened with, so leave both unset there. Searching a different image requires a new session_id.
image_base64	string or null	The image bytes, standard base64-encoded, without a data: prefix. The whole request must stay under 2 MiB. Supply at most one of image_url or image_base64. A call that carries a session_id already in use is always answered from the image that session was opened with, so leave both unset there.
crop	McpCropArea or null	Region of the image to search by, instead of the whole image — narrows what the search matches on (e.g. one product in a busy photo). A detected object’s bounding_box from a previous response can be echoed back here.
include_detected_objects	boolean or null	Whether the response carries detected_objects for the image being searched. Defaults to true on the call that opens a session and false on one that resumes it, where they would usually repeat the opening response. Set it to true under a session to have the objects reported again — the session’s image is detected on, so no image has to be re-sent.
campaign	string or null	Campaign ID to use instead of the merchant’s default visual-search campaign. Leave unset to use the default.
columns	array of string, or null	Catalog columns to materialize into every result’s attributes. Leave unset to receive the default preset.
limit	number (uint32) or null	Upper bound on the number of results returned. When omitted the merchant’s campaign limit applies. When set, the effective limit is min(limit, campaign_limit).
additional_filter	McpStaticFilter or null	Free-form boolean filter tree applied in addition to the campaign’s static filters. Same semantics as in text_search.
dynamic_filters	array of McpDynamicFilterSelection, or null	Facet selections echoed back from a previous response’s dynamic_filters. Same semantics as in text_search.
group_variants	boolean or null	Whether variants of the same product are collapsed into a single result. Leave unset to use the merchant’s configured default; set true to force grouping on or false to force it off.
session_id	string or null	Identifier (UUID) tying successive calls about the same image into one search session. Generate the UUID yourself — no response returns one — and send it with the image on the first call. Later calls carrying it are answered from that image; leave the image out of them, since supplying one has no effect. Searching a different image takes a fresh UUID. A session is short-lived (about an hour); once it lapses a new session needs to be initiated.

Response (VisualSearchResponse)

Field	Type	Description
results	array of McpItem	The ranked products matching the image. Always present.
dynamic_filters	array of McpDynamicFilter	The facets available for refining this result set. Always present.
detected_objects	array of McpDetectedObject, or null	Objects detected in the supplied image, each with a bounding box and a confidence. Absent when include_detected_objects is false or detection found nothing.
stream	array of McpStreamItem, or null	Stream items for this search — for example a suggested filter or a recognized barcode. Absent when the search produced none.
redirect_url	string or null	A URL the merchant mapped this search to, when one is configured. Absent otherwise.

Searching one image over several calls

A single call carrying an image is a complete search. Sessions exist for the case where the agent wants to keep working on that same image — narrowing to one object in a busy photo, refining facets, or asking for different columns — without re-sending it.

1. Generate a UUID and send it as session_id together with the image. The response carries the results and, by default, the image’s detected_objects.
2. Repeat that session_id on later calls and omit the image. Each call is answered from the session’s image, so it can vary crop, dynamic_filters, or columns freely. To narrow to one object, echo its bounding_box from step 1 back as crop.
3. Resuming calls leave detected_objects out by default, since they would repeat the opening response. Pass include_detected_objects: true to have them reported again.

A session lasts about an hour. Once it lapses — or when the shopper switches to a different image — open a new one with a fresh UUID and the image.