Visual-Search Products
const url = 'https://api.vviinn.com/visual-search';const options = { method: 'POST', headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'}, body: '{"requestId":"6edf9d4d-60e8-49f3-b120-e89d728b8a89","query":{"image":"iVBORw0KGgoAAAANSUhEUgAAAAMAAAADCAYAAABWKLW/AAAABHNCSVQICAgIfAhkiAAAACxJREFUCJkFwQENACAMBLEjwQMOZvMxg4H5QAgqjpZGfZHSeRmwxGyg9Bgb/YMgFBc0Hx5MAAAAAElFTkSuQmCC","url":"https://example.com/product-image.jpg","highResolutionImage":"iVBORw0KGgoAAAANSUhEUgAAAAMAAAADCAYAAABWKLW/AAAABHNCSVQICAgIfAhkiAAAACxJREFUCJkFwQENACAMBLEjwQMOZvMxg4H5QAgqjpZGfZHSeRmwxGyg9Bgb/YMgFBc0Hx5MAAAAAElFTkSuQmCC","searchArea":{"left":0.1728,"top":0.3641,"width":0.4575,"height":0.2917}},"resultsCount":8,"campaign":28,"staticFilters":{"excluded":[{"column":"brand","values":["Nike","Puma"]}],"included":[{"column":"brand","values":["Nike","Puma"]}]},"dynamicFilters":[{"column":"product_type","values":["cm9vdA","TcO2YmVsPlRpc2NoZQ"]}],"objectDetection":{"include":true}}'};
try { const response = await fetch(url, options); const data = await response.json(); console.log(data);} catch (error) { console.error(error);}curl --request POST \ --url https://api.vviinn.com/visual-search \ --header 'Authorization: Bearer <token>' \ --header 'Content-Type: application/json' \ --data '{ "requestId": "6edf9d4d-60e8-49f3-b120-e89d728b8a89", "query": { "image": "iVBORw0KGgoAAAANSUhEUgAAAAMAAAADCAYAAABWKLW/AAAABHNCSVQICAgIfAhkiAAAACxJREFUCJkFwQENACAMBLEjwQMOZvMxg4H5QAgqjpZGfZHSeRmwxGyg9Bgb/YMgFBc0Hx5MAAAAAElFTkSuQmCC", "url": "https://example.com/product-image.jpg", "highResolutionImage": "iVBORw0KGgoAAAANSUhEUgAAAAMAAAADCAYAAABWKLW/AAAABHNCSVQICAgIfAhkiAAAACxJREFUCJkFwQENACAMBLEjwQMOZvMxg4H5QAgqjpZGfZHSeRmwxGyg9Bgb/YMgFBc0Hx5MAAAAAElFTkSuQmCC", "searchArea": { "left": 0.1728, "top": 0.3641, "width": 0.4575, "height": 0.2917 } }, "resultsCount": 8, "campaign": 28, "staticFilters": { "excluded": [ { "column": "brand", "values": [ "Nike", "Puma" ] } ], "included": [ { "column": "brand", "values": [ "Nike", "Puma" ] } ] }, "dynamicFilters": [ { "column": "product_type", "values": [ "cm9vdA", "TcO2YmVsPlRpc2NoZQ" ] } ], "objectDetection": { "include": true } }'This endpoint allows you to search with an image in visual features and textual attributes. The response contains the search result for each product.
Authorizations
Section titled “Authorizations”Request Body
Section titled “Request Body”Visual Search request body
object
Example
6edf9d4d-60e8-49f3-b120-e89d728b8a89object
This image is used for finding products.
The image will be temporarily stored on the server-side, enabling subsequent requests to omit the image.
In that case, the requestId must be set in the request.
When sending, it should be either the base64-encoded image or a data URL containing an image in base64 encoding.
Example
iVBORw0KGgoAAAANSUhEUgAAAAMAAAADCAYAAABWKLW/AAAABHNCSVQICAgIfAhkiAAAACxJREFUCJkFwQENACAMBLEjwQMOZvMxg4H5QAgqjpZGfZHSeRmwxGyg9Bgb/YMgFBc0Hx5MAAAAAElFTkSuQmCCURL of the image to use for finding products.
Either image or url must be provided. If both are given, image takes precedence.
When using a URL, highResolutionImage is not needed and will be ignored,
as the image is fetched server-side at full resolution.
Example
https://example.com/product-image.jpgIn addition to the image, a high resolution image can be uploaded to ensure accurate results when using a search area.
Before sending a high resolution image, we recommend to send a small image initially to have results early.
When sending, it should be either the base64-encoded image or a data URL containing an image in base64 encoding.
Ignored when url is used.
Example
iVBORw0KGgoAAAANSUhEUgAAAAMAAAADCAYAAABWKLW/AAAABHNCSVQICAgIfAhkiAAAACxJREFUCJkFwQENACAMBLEjwQMOZvMxg4H5QAgqjpZGfZHSeRmwxGyg9Bgb/YMgFBc0Hx5MAAAAAElFTkSuQmCCA rectangle is defined by the position of its top-left corner and its side lengths.
Each value is relative to an image’s dimensions, i.e., they are in the range [0, 1].
The top-left corner is defined to be at (0, 0), the bottom-right corner at (1, 1).
object
The relative distance to the left edge of the image
Example
0.1728The relative distance to the top edge of the image
Example
0.3641The width relative to the image’s width
Example
0.4575The height relative to the image’s height
Example
0.2917Returns less than or equal to the campaign’s defined results
Example
8The static filters are predefined parameters that set the initial scope for the data retrieval process.
These filters are configured to extract a broad, meaningful set of results based on your criteria.
Once applied, they help in reducing the dataset to a manageable and relevant subset before any further, more granular filtering occurs via dynamicFilters.
object
Each item describes a column and values from that column that should be excluded from the result set
object
Example
brandExample
[ "Nike", "Puma"]Each item describes a column and values from that column that should be included in the result set
object
Example
brandExample
[ "Nike", "Puma"]The dynamic filters are parameters that refine the already narrowed down dataset from the staticFilters based on real-time request criteria.
These filters are flexible and can be adjusted dynamically to specific user queries or requirements, allowing for further granular filtering.
They provide a more precise result based on the user’s immediate needs.
object
Example
product_typeEach item describes a category path that should be used instead of the product’s actual category paths
Example
[ "cm9vdA", "TcO2YmVsPlRpc2NoZQ"]Example
[ "Nike", "Puma"]The interval object that is defined by a [start, end).
object
Example
[ "black > #1A1A1A", "beige > #C7B3A8"]Example
[ "black", "beige"]object
If true, objects in the image will be detected and returned as suggested search areas
Responses
Section titled “Responses”OK
object
object
object
This field is superseded by the entity type and therefore deprecated.
The API usually returns products; it can also return entities representing something else.
To enable different handling of these entities, the API uses an entity-type to distinguish between them, namely:
- A
productentity represents a product in the catalog. - A
contententity contains information relevant to a user, e.g., trends, opening hours. - A
listentity comprises a collection of distinguishable items, e.g., a category page.
Link to the product detail page
object
object
object
object
The base price per measure is derived by adjusting the product’s total cost to a standardized unit of measurement, making it easier to compare products of varying volumes.
object
The base price per measure is derived by adjusting the product’s total cost to a standardized unit of measurement, making it easier to compare products of varying volumes.
object
object
object
object
object
object
object
In a category tree, a category path identifies a category of the tree. The API expects and returns each category path only in encoded form.
A category path is a sequence category labels, separated by the > character.
Whitespace around the category labels is ignored.
If the > character is part of a category label, it must be escaped with %3E.
The root node has no label and is omitted in a category path.
To represent the root node, use root as the sole label instead.
Consider the following category tree:
.
├── Furniture
│ ├── Benches
│ └── Cabinets & Storage
└── Home & Garden
├── Plants
├── Wood Stoves
└── Fish Pond (<><)That contains the following category paths:
rootFurnitureFurniture > BenchesFurniture > Cabinets & StorageHome & GardenHome & Garden > PlantsHome & Garden > Wood StovesHome & Garden > Fish Pond (<%3E<)
Category paths are transmitted in encoded form. To this end, we use the Base64-URL encoding without any padding. The category paths above are encoded as follows:
cm9vdARnVybml0dXJlRnVybml0dXJlID4gQmVuY2hlcwRnVybml0dXJlID4gQ2FiaW5ldHMgJiBTdG9yYWdlSG9tZSAmIEdhcmRlbgSG9tZSAmIEdhcmRlbiA-IFBsYW50cwSG9tZSAmIEdhcmRlbiA-IFdvb2QgU3RvdmVzSG9tZSAmIEdhcmRlbiA-IEZpc2ggUG9uZCAoPCUzRTwp
On Linux, the basenc command can be used to encode category paths:
echo -n 'Furniture > Cabinets & Storage' | basenc -w0 --base64url
# returns: RnVybml0dXJlID4gQ2FiaW5ldHMgJiBTdG9yYWdl
echo -n 'RnVybml0dXJlID4gQ2FiaW5ldHMgJiBTdG9yYWdl' | basenc -d -w0 --base64url
# returns: Furniture > Cabinets & Storageobject
object
object
object
The interval object that is defined by a [start, end).
object
The interval object that is defined by a [start, end).
object
object
object
The interval object that is defined by a [start, end).
object
object
object
The interval object that is defined by a [start, end).
object
object
A rectangle is defined by the position of its top-left corner and its side lengths.
Each value is relative to an image’s dimensions, i.e., they are in the range [0, 1].
The top-left corner is defined to be at (0, 0), the bottom-right corner at (1, 1).
object
The relative distance to the left edge of the image
The relative distance to the top edge of the image
The width relative to the image’s width
The height relative to the image’s height
object
The URL to which the user should be redirected.
A stream guides a user to fulfilling an intent by providing precise and relevant insights.
The engine behind a stream can be likened to an assistant that continuously observes the user’s actions and tries to reconcile it with the product catalog. As soon as the imagined assistant sees an opportunity to help, it provides stream items. They can be presented to the user in a chat-like interface and be acted upon by the user. Alternatively, they can be processed automatically, to act on unambiguous, high-confidence suggestions.
An important distinction to interactions is a stream’s requirement for precision and a higher threshold for relevance: For the former, the interactions might contain multiple filters with many options, whereas a stream would suggest only a single filter with ideally only one option. For the latter, an interaction might contain filter options that are unlikely to be relevant, whereas a stream would never suggest such options. That is, the interactions give the user the full flexibility to explore, whereas a stream guides the user by focusing on an intent.
object
object
A stream involves multiple actors, each with a different purpose and capabilities.
They are distinguished by their role in the stream:
- The
assistantis the part of the system that drives the stream, usually by means of AI. - Items produced by the
systeminitiate the stream’s context, usually, by defining the assistant’s goals, means and limitations. - The
useris the person who interacts with the system, e.g., a customer. - The
clientreceives the stream and processes it for the user. This can be as simple as rendering the stream in a chat interface; or as complex as reacting automatically to stream items, e.g., by automatically applying a filter.
The distinction between user, client and assistant is crucial:
It allows to produce effects on the frontend that are not initiated by the user, but requested by the assistant.
Likewise, information from the frontend can flow to the assistant, without involving the user.
object
A context item summarizes the current situation and informs about next steps.
It is composed of text (content) and structured data (annotations), intended for the user and client, respectively.
With this separation, the content can be arranged in a human-readable way, without any need for further processing.
However, the client can use an annotation to replace its referenced part of the content with a more meaningful representation, e.g., product cards.
object
The content is human-readable markdown text, intended for the user.
A context annotation provides structured information about a specific part of a context’s content.
The locator describes the part of the text that the annotation refers to.
The data contains the structured information related to the part referred to by the annotation, e.g., the part’s contained products.
The interactions describe how the user can interact with the annotation or its parts, e.g., by setting a filter when clicking on a card.
object
This object describes the location of what the annotation refers to.
object
The position of the annotation’s first character in the text
The annotation’s length in the text
object
object
This field is superseded by the entity type and therefore deprecated.
The API usually returns products; it can also return entities representing something else.
To enable different handling of these entities, the API uses an entity-type to distinguish between them, namely:
- A
productentity represents a product in the catalog. - A
contententity contains information relevant to a user, e.g., trends, opening hours. - A
listentity comprises a collection of distinguishable items, e.g., a category page.
Link to the product detail page
object
object
object
object
The base price per measure is derived by adjusting the product’s total cost to a standardized unit of measurement, making it easier to compare products of varying volumes.
object
The base price per measure is derived by adjusting the product’s total cost to a standardized unit of measurement, making it easier to compare products of varying volumes.
object
object
object
object
object
object
object
In a category tree, a category path identifies a category of the tree. The API expects and returns each category path only in encoded form.
A category path is a sequence category labels, separated by the > character.
Whitespace around the category labels is ignored.
If the > character is part of a category label, it must be escaped with %3E.
The root node has no label and is omitted in a category path.
To represent the root node, use root as the sole label instead.
Consider the following category tree:
.
├── Furniture
│ ├── Benches
│ └── Cabinets & Storage
└── Home & Garden
├── Plants
├── Wood Stoves
└── Fish Pond (<><)That contains the following category paths:
rootFurnitureFurniture > BenchesFurniture > Cabinets & StorageHome & GardenHome & Garden > PlantsHome & Garden > Wood StovesHome & Garden > Fish Pond (<%3E<)
Category paths are transmitted in encoded form. To this end, we use the Base64-URL encoding without any padding. The category paths above are encoded as follows:
cm9vdARnVybml0dXJlRnVybml0dXJlID4gQmVuY2hlcwRnVybml0dXJlID4gQ2FiaW5ldHMgJiBTdG9yYWdlSG9tZSAmIEdhcmRlbgSG9tZSAmIEdhcmRlbiA-IFBsYW50cwSG9tZSAmIEdhcmRlbiA-IFdvb2QgU3RvdmVzSG9tZSAmIEdhcmRlbiA-IEZpc2ggUG9uZCAoPCUzRTwp
On Linux, the basenc command can be used to encode category paths:
echo -n 'Furniture > Cabinets & Storage' | basenc -w0 --base64url
# returns: RnVybml0dXJlID4gQ2FiaW5ldHMgJiBTdG9yYWdl
echo -n 'RnVybml0dXJlID4gQ2FiaW5ldHMgJiBTdG9yYWdl' | basenc -d -w0 --base64url
# returns: Furniture > Cabinets & Storageobject
object
object
object
The interval object that is defined by a [start, end).
object
The interval object that is defined by a [start, end).
object
object
object
The interval object that is defined by a [start, end).
object
object
object
The interval object that is defined by a [start, end).
object
object
A rectangle is defined by the position of its top-left corner and its side lengths.
Each value is relative to an image’s dimensions, i.e., they are in the range [0, 1].
The top-left corner is defined to be at (0, 0), the bottom-right corner at (1, 1).
object
The relative distance to the left edge of the image
The relative distance to the top edge of the image
The width relative to the image’s width
The height relative to the image’s height
object
The URL to which the user should be redirected.
A blob is used to transfer binary data within a stream, e.g., an image uploaded by the user.
object
A function-call represents an invocation of a function.
In most cases, a function is executed by the assistant, e.g., to predict a category based on the user’s input. However, a function can also be executed by the client, e.g., to automatically set a filter. In that case, the assistant can only request the client to execute a function. To do so, the assistant needs to be aware of the functions supported by the client.
object
If set, the result of this function call is required to have the same id.
The name of the function to be called.
object
A function-response contains the result of a function call.
The client can show the structured data in the response in a more meaningful way, e.g., by using a dedicated UI component.
object
If a function call has an id, the function response must use the same id.
The name of the function that was called.
object
file-data references an uploaded file in a stream, thus avoiding to repeatedly sending the same file.
object
Example
{ "requestId": "6edf9d4d-60e8-49f3-b120-e89d728b8a89", "data": { "products": [ { "kind": "product", "entityType": "product", "groupId": "55fc5f71d6362321cb93b82b387e2f32", "productId": "55fc5f71d6362321cb93b82b387e2f32", "title": "Livingwalls Fototapete Walls by Patel iconic 3", "deeplink": "http://mainUrl/site/5/product/55fc5f71d6362321cb93b82b387e2f32/details", "brand": "Fender", "productType": "guitar", "image": { "original": "http://mainUrl/image.jpg", "thumbnail": "http://mainUrl/thumbnail-image.jpg" }, "images": [ { "original": "http://mainUrl/image.jpg", "thumbnail": "http://mainUrl/thumbnail-image.jpg" } ], "price": { "actual": 19.9, "sale": 19.9 }, "basePrice": { "actual": { "price": 15.92, "quantity": 100, "measure": "ml" }, "sale": { "price": 15.92, "quantity": 100, "measure": "ml" } }, "energyEfficiencyClass": "A+++", "minEnergyEfficiencyClass": "A+++", "maxEnergyEfficiencyClass": "A+++", "energyCertifications": [ { "certificationClass": "A+++", "classRange": { "min": "A+++", "max": "A+++" } } ] } ] }, "interactions": { "dynamicFilters": [ { "type": "categorical", "column": "product_type", "label": "Product Type", "isPrimary": true, "values": [ { "value": [ "RnVybml0dXJl", "SG9tZSAmIEdhcmRlbg" ], "label": "Tische", "active": true, "selected": false } ] } ], "suggestedSearchAreas": [ { "rectangle": { "left": 0.1728, "top": 0.3641, "width": 0.4575, "height": 0.2917 } } ], "redirect": { "url": "https://www.example.com/some-category" } }, "stream": { "items": [ { "role": "assistant", "parts": [ { "text": { "content": "You are looking for a couch. We have two categories that might be of particular interest to you:\n\n* [Lounge](https://www.example.com/lounge)\n* [Divan](https://www.example.com/divan)\n\nThey represent different kinds of couches.\n", "annotations": [ { "locator": { "start": 98, "length": 83 }, "data": { "type": "products", "items": [ { "kind": "product", "entityType": "product", "groupId": "55fc5f71d6362321cb93b82b387e2f32", "productId": "55fc5f71d6362321cb93b82b387e2f32", "title": "Livingwalls Fototapete Walls by Patel iconic 3", "deeplink": "http://mainUrl/site/5/product/55fc5f71d6362321cb93b82b387e2f32/details", "brand": "Fender", "productType": "guitar", "image": { "original": "http://mainUrl/image.jpg", "thumbnail": "http://mainUrl/thumbnail-image.jpg" }, "images": [ { "original": "http://mainUrl/image.jpg", "thumbnail": "http://mainUrl/thumbnail-image.jpg" } ], "price": { "actual": 19.9, "sale": 19.9 }, "basePrice": { "actual": { "price": 15.92, "quantity": 100, "measure": "ml" }, "sale": { "price": 15.92, "quantity": 100, "measure": "ml" } }, "energyEfficiencyClass": "A+++", "minEnergyEfficiencyClass": "A+++", "maxEnergyEfficiencyClass": "A+++", "energyCertifications": [ { "certificationClass": "A+++", "classRange": { "min": "A+++", "max": "A+++" } } ] } ] }, "interactions": { "dynamicFilters": [ { "type": "categorical", "column": "product_type", "label": "Product Type", "isPrimary": true, "values": [ { "value": [ "RnVybml0dXJl", "SG9tZSAmIEdhcmRlbg" ], "label": "Tische", "active": true, "selected": false } ] } ], "suggestedSearchAreas": [ { "rectangle": { "left": 0.1728, "top": 0.3641, "width": 0.4575, "height": 0.2917 } } ], "redirect": { "url": "https://www.example.com/some-category" } } } ] }, "inlineData": { "mimeType": "image/png", "data": "iVBORw0KGgoAAAANSUhEUgAAAAMAAAADCAYAAABWKLW/AAAABHNCSVQICAgIfAhkiAAAACxJREFUCJkFwQENACAMBLEjwQMOZvMxg4H5QAgqjpZGfZHSeRmwxGyg9Bgb/YMgFBc0Hx5MAAAAAElFTkSuQmCC" }, "fileData": { "mimeType": "image/png", "fileUri": "https://example.com/path/to/file.png" } } ] } ] }}Authentication Error
Example
{ "message": "Authentication Error"}Authorization Error
Example
{ "message": "Authorization Error"}