Docs

Unbxd Product Feed

Introduction

Product feed is a snapshot of your site’s product catalog data in Unbxd format. A Product feed essentially contains product-specific information of all the products (in stock, out of stock) like Schema, SKU (Stock Keeping Unit), title, price, category, color, description, availability, and other product attributes.

Product feed is fundamental for us to power search on your ecommerce store. All our product discovery algorithms work around your catalog data, which must be uploaded in a particular format in order for us to relate to your products and their fields (product attributes).

Schema

<schema> is the key component of a Product Feed. It contains definitions of all fields used in your catalog. A field is defined by the following field properties:

  1. fieldname.
  2. datatype.
  3. multi-value.
  4. isVariant.
  5. id.

1. fieldname

A <fieldname> denotes the name of a field. For example, "product title", "brand", "color", "style", "imageUrl", "sku", "producUrl", etc.

A fieldname:
  • cannot be null or empty.
  • is case-sensitive.
  • can contain alphabets, numbers, hyphens, or underscores.
  • can begin with an alphabet or an underscore.
  • cannot end with an underscore.
  • cannot have spaces between words.

2.dataType

A <dataType> determines the type of data a field can take and also the searchability of a field. Unbxd supports 9 data types, which are explained in the table below:

dataTypeFormatDescription
textStringUsed for fields that briefly describes a product such as, title, occasion, color, brand, etc. This data type must be used for fields that are often used in search queries.
longTextStringUsed for fields that describes a product in detail such as, description, seller_name, etc. This data type must be used for fields that are not used in search queries.
decimal (or double)DecimalUsed for fields that take decimal values such as price, discount, etc.
numberNumericUsed for fields that take numeric values such as, quantity, orderCount, viewCount, etc.
linkURL stringUsed for fields that take URL strings as values such as, productUrl, imageUrl, etc.
dateISO 8601 date format (YYYY-MM-DDTHH:MM:SSZ)Used for fields that take date values such as, createdAt, updatedAt, etc.
booltrue/falseUsed for fields that take boolean values such as, availability, hasBanner etc.
skuString valueUsed for fields that denote unique information of the product such as, sku, uid (or pid), etc. Fields with this data type can be used in search queries.
pathString value Used for fields that are hierarchical in nature. Eg, categoryPath. Different values within a hierarchy are separated using “>”. Eg, “Men>Shoes>Casual Shoes”
Mapping the field to its correct dataType is very important as the Unbxd algorithm will relate to the fields based on this information.

3. multiValue

<multiValue> property determines whether a field can take multiple values. For example, fields like color, size, etc. should have the "multiValue" property set to true (i.e. multiValue=true) else multiValue=false.

  • add: An add tag is used for adding a product with all its associated fields to the existing product feed.
  • items: An item denotes the field of a product.

4. id

“id” property specifies the numerical identifier of a field (if present). For example, "id": "16295". This is of importance as few platforms provide field IDs instead of field names, or field IDs along with field names.

5. isVariant

Products can be available in different sizes, colors, styles, materials, etc. For example, a dress can be available in different sizes, colors and/or styles.

Variant 1 : Color - Blue, Size: Small, Style - Solid print

Variant 2 : Color - Red, Size: Small, Style - Solid print

Variant 3 : Color - Blue, Size: Large, Style - Polka-dot

.....

<isVariant> property is used when the feed has variants for products. It determines whether the field belongs to a variant in a product. It can take two values, “true” or “false”

Example Schema:

{
    "feed": {
        "catalog": {
            "schema": [{
                "fieldName": "vColor",
                "id": "76678",
                "dataType": "text",
                "multiValue": "true",
                "autoSuggest": "false",
                "isVariant": "true"
            }, {
                "fieldName": "vSize",
                "dataType": "text",
                "multiValue": "true",
                "autoSuggest": "false",
                "isVariant": "true"
            }, {
                "fieldName": "vImages",
                "dataType": "link",
                "multiValue": "true",
                "autoSuggest": "false",
                "isVariant": "true"
            }, {
                "fieldName": "vPrice",
                "dataType": "decimal",
                "multiValue": "true",
                "autoSuggest": "false",
                "isVariant": "true"
            }]
        }
    }
}

Schema Upload

Path /{siteKey}/upload/schema

HTTP Verb POST

Description: This api will upload a given schema file to search servers.

Every schema upload appends to the existing schema. In case the new schema upload has fields that are overlapping with the existing schema - only the overlapping fields are updated, and other fields are appended to the existing schema.

Parameters:

  • secretKey used to authorize the request
  • fileName json file to upload

Example:

curl -X POST \  
  http://feed.unbxd.io/api/{siteKey}/upload/schema \
  -H 'Authorization:{secretKey}' \
  -F file=@{fileName}.json

Upon success response

{
"statusCode":200,
"message":"File has been processed successfully.",
"unbxdFileName":"{fileName}.json",
"unknownSchemaFields":[],
"fieldErrors":[],
"rowNum":null,
"colNum":null,
"integrationStep":false
}

Feature Fields

Feature fields are pre-defined fields essential for Unbxd Ecommerce Search to work on your site.

fieldnamemultiValuedataType Description
uniqueId (mandatory field)falsetextUnique identifier of a product. It cannot have special character except ‘-’ and ‘_’
variantIdfalsetextUnique identifier of a variant (if present). It cannot have special character except ‘-’ and ‘_’. It needs to be unique across entire product catalog.
variantstruetextField holding all the variants for a product.
titlefalsetextTitle or name of a product.
pricefalsedecimal(or double)Price of a product.
descriptionfalselongTextDescription of a product.
categorytruetextName of the category a product belongs to.
subCategorytruetextName of the sub-category a product belongs to.
categoryPath (mandatory field)truepathThe category hierarchy a product belongs to. Different values present need to be separated using “>”. For example, “Men>Shoes>Casual Shoes”
categoryPathIdtruetextThe category comprises of names and Ids, a product belongs to. Different values present need to be separated using “>”. IDs and Names need to be separated using “|”. For example, “cat100|Home>cat101|Luggage & Travel Accessories>cat102|Travel Accessories”
catlevel1NamefalsetextIf you consider the category structure as a tree, this value should contain those category values, which belongs to the 1st level of the tree.
catlevel2NamefalsetextIf you consider the category structure as a tree, this value should contain those category values, which belongs to the 2nd level of the tree.
catlevel3NamefalsetextIf you consider the category structure as a tree, this value should contain those category values, which belongs to the 3rd level of the tree.
catlevel4NamefalsetextIf you consider the category structure as tree, this value should contain those category values, which belongs to the 4th level of the tree.
imageUrlfalselinkURL of a product image.
productUrlfalselinkURL of the Product Detail Page (PDP) of a product.
currencyfalsetextCurrency of a product.
brandfalsetextBrand of a product.
colortruetextColor of a product.
availabilityfalseboolThe availability of a product in “true”/“false” format.
skufalseskuUnique identifier for an item.
genderfalsetextGender for a product.
sizetruetextSize of a product.
ratingfalsedecimalRating for a product.
discountfalsedecimalDiscount on a product.
sellingPricefalsedecimalSelling price of a product.

The Console lets you map these feature fields to your custom fields in order for our search to relate to your catalog data.

Notes:

  • Feature fields can be used without defining it in the schema. If you have a similar field, you can either map it to the corresponding feature field from the Console or define it in the schema.
  • Presently, the Console doesn't let you map the "categoryPath" feature field, so if you have a similar field, you must rename it to "categoryPath".

Adding variant information in feed:
In addition to the existing feed format, variants can be added as a JSON array to a reserve field called “variants” (which is a JSON object) under each product.

Each 'variant' should have the mandatory field - variantId. variantId is unique across the entire product catalog for a site. There can be one or more fields associated with each of the variant, where each of these fields should have the schema defined.

Currently, we do not support some products with variants, and some without variants. As a workaround, we add a dummy field for all products that are without variants in the catalog.

Adding id information in feed:
The Unbxd Product Feed supports IDs for values of fields in the feed. Such fields need to have the following fields:

  • “id” - Numerical ID of the value.
  • “value” - Text value.

For example, if a customer wants to use brand-id instead of brand-name, same can be achieved using this operation.

Example feed format:

{
    "feed": {
        "catalog": {
            "add": {
                "items": [{
                        "uniqueId": "parent-id-1",
                        "title": "batman",
                        "variants": [{
                                "variantId": "child-1",
                                "vColor": [{
                                    "id": "15245",
                                    "value": "Black"
                                }],
                                "vSize": "m",
                                "vPrice": 19,
                                "vImages": "http://example.com/images/1-1.jpg"
                            },
                            {
                                "variantId": "child-2",
                                "vColor": [{
                                    "id": "86787",
                                    "value": "Blue"
                                }],
                                "vSize": "xl",
                                "vPrice": 10,
                                "vImages": "http://example.com/images/1-2.jpg"
                            }
                        ]
                    },
                    {
                        "uniqueId": "parent-id-2",
                        "title": "superman",
                        "variants": [{
                            "variantId": "child-1",
                            "vColor": [{
                                    "id": "47670",
                                    "value": "Red"
                                }],
                            "vSize": "s",
                            "vPrice": 190
                        }, {
                            "variantId": "child-2",
                            "vColor": [{
                                    "id": "86787",
                                    "value": "Blue"
                                }],
                            "vSize": "xxl",
                            "vPrice": 100,
                            "vImages": "http://example.com/images/2-2.jpg"
                        }],
                        "someField": "extra field data"
                    }
                ]
            }
        }
    }
}

File Formats

A product feed can be uploaded in the following file formats:

Product Feed Upload

You can upload your Product Feed depending on your integration approach:

Note: If you are choosing to upload your catalog from the Console, it will always be a Full Feed Upload.

Request Parameters

  • operation : This parameter take the following values:
    • add : Inserts a new document or overwrites if a document with the same uniqueId already exists.
    • delete : Deletes the document matching the uniqueId, only uniqueId is required to perform this operation.
    • update : Update is currently not supported through the single record upload API. Updating the records is supported only through the delta upload API.
  • secretKey : A unique key provided on successful site creation. This key can also be retrieved from the console.

Response

Response would return the following HTTP status codes:

  • 201: On successful feed upload
  • 401: On authentication failures
  • 400: On Bad Request
  • 500: On Internal server errors

Sample code in C#

var client = new RestClient("http://feed.unbxd.io/api/{siteKey}/upload/feed?operation={operationType}");  
var request = new RestRequest(Method.POST);  
request.AddHeader("postman-token", "9f6c9a98-86f0-d52e-3fe3-4854efc5caf0");  
request.AddHeader("cache-control", "no-cache");  
request.AddHeader("authorization", "d9b5d63136630d90de7fb53a756586a3");  
request.AddParameter("application/json; charset=utf-8", "{\n  \"uniqueId\": \"parent-id-1\",\n  \"title\": \"batman\",\n  \"variants\": [\n    {\n      \"variantId\": \"child-1\",\n      \"vColor\": \"black\",\n      \"vSize\": \"m\",\n      \"vPrice\": 19,\n      \"vImages\": \"http://example.com/images/1-1.jpg\"\n    },\n    {\n      \"variantId\": \"child-2\",\n      \"vColor\": \"blue\",\n      \"vSize\": \"xl\",\n      \"vPrice\": 10,\n      \"vImages\": \"http://example.com/images/1-2.jpg\"\n    }\n  ]\n}", ParameterType.RequestBody);  
IRestResponse response = client.Execute(request);  

Upload Types

There are three types of Product Feed upload:

  • Full Feed Upload
  • Incremental Feed Upload
  • Single Record Upload

Product Feed can be updated by performing these operations - add, update and delete.

Example feed format:

{
    "feed": {
        "catalog": {
            "add": {
                "items": [{
                        "uniqueId": "parent-id-1",
                        …}


                    },
                    {
                        "uniqueId": "parent-id-2",
                        "title": "superman",
                        "variants": [{
                            "variantId": "child-1",
                            "vColor": [{
                                    "id": "47670",
                                    "value": "Red"
                                }],
                            "vSize": "s",
                            "vPrice": 190
                        }, {
                            "variantId": "child-2",
                            "vColor": [{
                                    "id": "86787",
                                    "value": "Blue"
                                }],
                            "vSize": "xxl",
                            "vPrice": 100,
                            "vImages": "http://example.com/images/2-2.jpg"
                        }],
                        "someField": "extra field data"
                    }
                ]
            }
        }
    }
}

Full Catalog Upload

A Full Feed Upload will replace any previously uploaded Product Feed file with the current one. This type of upload preferred for initial uploads or when you have untracked changes in the feed.

Note: If you have a large catalog, upload the entire Product Feed file in one go. Performing a full feed upload batch-wise (or in smaller chunks) will only upload the most recent data replacing the previously uploaded data.

Path /{siteKey}/upload/catalog/full

HTTP Verb POST

Description: This API will perform a full upload of the catalog when provided with the JSON file containing all the records.

  • secretKey will be used to authorize the request
  • fileName will be the json file to be uploaded
curl -X POST \  
  http://feed.unbxd.io/api/{siteKey}/upload/catalog/full \
  -H 'Authorization:{secretKey}' \
  -F file=@{fileName}.json

Output

{
  "fileName": "{fileName}",
  "uploadId": "{id}",
  "timeStamp": 6581239201
}

Note: For full upload uploadId is to be used to get status

Incremental Feed Upload

You can use Incremental Feed Upload to update/add multiple records, performing partial catalog upload. Doing Incremental will update only the products and product fields sent in json file, by replacing the existing product information with new product information sent in the current feed file. Other products which were the part of previous incremental feed upload or full feed upload are not the part of new feed file remain untouched and retain previous information.

Path : /{siteKey}/upload/catalog/delta
HTTP Verb : POST

Description:
This API will perform an upload of the partial catalog when provided with the feed file that contains a batch of records.

Request Parameters:

  • secretKey will be used to authorize the request
  • fileName will be the json file to be uploaded

Request

curl -X POST \  
 http://feed.unbxd.io/api/{siteKey}/upload/catalog/delta \
  -H 'Authorization:{secretKey}' \
  -F file=@{fileName}.json

Response

{
  "fileName": "{fileName}",
  "uploadId": "{id}",
  "timeStamp": 6581239201
}

Mandatory Fields in Feed upload file.

Variants disabledUniqueId
Variants enabledUniqueId, VariantId

Incremental feed upload in case Variants are enabled

  1. If Variants are enabled:
    a. uniqueID and variantID are mandatory fields.

    b. The incremental feed upload doesn’t support the “update” block, and only “add” and “delete” blocks are supported. So, if an update request has to be made, entire product block should be sent in the “add” block, including all the variants of the product. This will replace the existing product completely.

Single Record Upload

Incase you want to update/add just one product , single record update lets you send the product along with request.

Path : /{siteKey}/upload/feed

HTTP Method : POST

Request

Below request will add/update the product with "uniqueId": "parent-id-1". In case we want to update only partial information we can send the partial information. Here uniqueId is mandatory field. But when variants is enabled, one need to send the entire products block, including all the variants of the parent products and parent product info. ( in case of product update as well).

Example:

curl -X POST 'http://feed.unbxd.io/api/{siteKey}/upload/feed?operation={operationType}'  
  -H 'Authorization:{secretKey}'
  -d '{
  "uniqueId": "parent-id-1",
  "title": "batman",
  "variants": [
    {
      "variantId": "child-1",
      "vColor": "black",
      "vSize": "m",
      "vPrice": 19,
      "vImages": "http://example.com/images/1-1.jpg"
    },
    {
      "variantId": "child-2",
      "vColor": "blue",
      "vSize": "xl",
      "vPrice": 10,
      "vImages": "http://example.com/images/1-2.jpg"
    }
  ]
}'

Mandatory Fields in Feed upload file.

Variants disabledUniqueId
Variants enabledUniqueId, VariantId

Single record feed upload in case Variants are enabled

  1. If Variants are enabled, uniqueID and variantID are mandatory fields.
  2. Also when variants is enabled, one need to send the entire products block, including all the variants of the parent products and parent product info. This is because, in case of variants, whatever is sent in feed file replaces the existing product completely and not just the partial feeds which are sent with the feed file.

Feed Status

Full Feed Upload Status History

Path /api/{siteKey}/catalog/status

HTTP Verb GET

Description:

This will give you history of full feed upload statuses for a given siteKey.

Parameters:

  • secretKey used to authorize the request
  • count no of last records to fetch default:5
  • status can be INDEXING,INDEXED,FAILED

Example:

curl -X GET \  
  http://feed.unbxd.io/api/{siteKey}/catalog/status?count={count} \
  -H 'authorization: {secretKey}'

Response :

[
  {
    "fileName": "products.json",
    "uploadId": "{id}",
    "status": "INDEXED",
    "timeStamp": 1497016711231
    "message":"success"
  }
]

Note:

In case of failure, the message field in response would contain the reason for the failure.

Full Feed Upload Status For UploadId

Path /api/{siteKey}/catalog/{uploadId}/status

HTTP Verb GET

Description:
Returns full feed upload status for a given upload id.

Parameters:

  • secretKey used to authorize the request
  • uploadId id wrt the full feed upload (returned in response of full feed upload)
  • status can be INDEXING,INDEXED,FAILED

Example:

curl -X GET \  
  http://feed.unbxd.io/api/{siteKey}/catalog/{uploadId}/status \
  -H 'authorization: {secretKey}'

Response:

{
    "fileName": "products.json",
    "uploadId": "{id}",
    "status": "INDEXED",
    "timeStamp": 1497016711231
    "message":"success"
  }

Single Record Upload Status History

Path /{siteKey}/history/catalog/event
HTTP Verb GET

Description:
This will give you history of single record upload statuses for a given siteKey.

Parameters:

  • secretKey used to authorize the request
  • count no of last records to fetch default:5
  • status can be INDEXED,FAILED

Example:

curl -X GET \  
  http://feed.unbxd.io/{siteKey}/history/catalog/event?count={count} \
  -H 'authorization: {secretKey}'

Response:

[
  {
    "fileName": "NA",
    "uploadId": "{id}",
    "status": "INDEXED",
    "timeStamp": 1497016711231
    "message":"success
  }
]

Single Record Upload Status For UploadId

Path /{siteKey}/history/catalog/event/{uploadId}

HTTP Verb GET

Description
Returns single record upload status for a given upload id.

Parameters

  • secretKey used to authorize the request
  • uploadId id wrt the full feed upload (returned in response of full feed upload)
  • status can be INDEXED,FAILED

Example:

curl -X GET \  
  http://feed.unbxd.io/{siteKey}/history/catalog/event/{uploadId} \
  -H 'authorization: {secretKey}'

Response:

{
    "fileName": "NA",
    "uploadId": "{id}",
    "status": "INDEXED",
    "timeStamp": 1497016711231
    "message":"success"
  }

Feed Record Count

Returns the number of records present for a given site.

curl -X GET  http://feed.unbxd.io/api/{siteKey}/catalog/size  

Output

2345