Query understanding is a very important part of the journey from the time a user makes a query to the point search results are shown. A typical search engine involves 3 phases:
NER enables us to perform contextual query expansion which can be then fed to our search engine for culling out a more precise and relevant set of results as compared to the traditional query expansion. The most common ways of query expansion include using generic synonyms, antonyms, spell checks which apply to all the entities/attributes. One of the basic disadvantages of this approach is the expansion of ambiguous entities like a cap which can be a sleeve type for queries like cap sleeve dress and type for other queries like a cap for full sleeve dress.
Query expansion without using NER and entity specific synonyms can totally change the meaning of above queries for example:
cap sleeve dress will become
(cap | cap_synonyms_fashion) (sleeve | sleeve_synonyms) (dress | dress_synonyms).
However, if we were to use NER and entity-specific synonyms it would be
( cap sleeve | cap_sleeve_sleevetype_synonyms) (dress | dress_product_type_synonyms)
where cap_sleeve_sleevetype_synonyms is a synonym for cap sleeve specifically in the context of sleeve type and so on.
So without contextual query expansion, we would end up showing results for caps along with the dress. However, with NER we would show results for dresses that have cap sleeves which was the user intent.
The above-mentioned approach involves two important processes:
Needless to say, all the entities would have some score associated with them that comes out of clickstream. This score signifies the importance of each attribute for a query and finally for the whole data set i.e customer catalog.
Training data can be handcrafted or auto-generated, at our scale we prefer to go with auto-generated. Following are sources we choose to go with:
Input sequence is the query terms and output includes are the corresponding entity tags and the query intent. We learn a model such that for new input query terms the model outputs the predicted entity tags and the query intent.
- query expansion (in native query language),
- getting results with query criteria,
- scoring results.
NER enables us to perform contextual query expansion which can be then fed to our search engine for culling out a more precise and relevant set of results as compared to the traditional query expansion. The most common ways of query expansion include using generic synonyms, antonyms, spell checks which apply to all the entities/attributes. One of the basic disadvantages of this approach is the expansion of ambiguous entities like a cap which can be a sleeve type for queries like cap sleeve dress and type for other queries like a cap for full sleeve dress.
Query expansion without using NER and entity specific synonyms can totally change the meaning of above queries for example:
cap sleeve dress will become
(cap | cap_synonyms_fashion) (sleeve | sleeve_synonyms) (dress | dress_synonyms).
However, if we were to use NER and entity-specific synonyms it would be
( cap sleeve | cap_sleeve_sleevetype_synonyms) (dress | dress_product_type_synonyms)
where cap_sleeve_sleevetype_synonyms is a synonym for cap sleeve specifically in the context of sleeve type and so on.
So without contextual query expansion, we would end up showing results for caps along with the dress. However, with NER we would show results for dresses that have cap sleeves which was the user intent.
The above-mentioned approach involves two important processes:
- Entity Recognition
- Query Expansion using entity synonyms
- Simple Entity recognition models
- Joint models for Entity recognition and user intent (product type(s) or category)
Needless to say, all the entities would have some score associated with them that comes out of clickstream. This score signifies the importance of each attribute for a query and finally for the whole data set i.e customer catalog.
Training data can be handcrafted or auto-generated, at our scale we prefer to go with auto-generated. Following are sources we choose to go with:
-
- Catalog data (product data)
- Clickstream data (user actions (click/cart/buy) for a specific query and a product)
Input sequence is the query terms and output includes are the corresponding entity tags and the query intent. We learn a model such that for new input query terms the model outputs the predicted entity tags and the query intent.
- Conditional Random Fields (CRF): CRFs are a class of statistical modeling methods and take context (neighboring tags) into account when predicting a tag. The features used to generate this are mainly the current word, next/previous words, labels of next/previous words, prefix/suffix of the words, word shapes (Digits/Alphabets, etc), n-gram variations of all these. While this ensures we consider the context to predict more relevant tags it also needs enough training data set to produce good recall.
- Recurrent Neural Networks and Convolutional nets based models: We tried out a variety of neural network-based models for sequence tagging search queries. Transition learning can be simply understood as a state machine kind of approach where the input sequence passes through multiple states and decision is made at each state to generate the label for that state. It takes considerable large training time and the performance optimization might require good enough infrastructure but we were able to achieve state of the art performance (> 99.9 % F1 scores) for these models. These models include the following:
- bidirectional LSTM networks (BI-Long Short Term Memory [1])
- Bi-LSTM with char CNN
- Bi-LSTM networks with a CRF layer (BILSTM-CRF)
- Bi-LSTM – CNN – CRF with the intent prediction
- Domain-specific Models: Domain/vertical specific models are the ones where we have identified a set of common attributes/entities for that vertical. We try to fit all the queries for that vertical on it. It makes it easier for us to enable default relevance for a customer. However there are cases where a customer could belong to a subset of a vertical or a combination of verticals, these are the cases we might not see a good performance from generic vertical-specific models.
- Customer Specific Models: These are the datasets that do not fit into a certain vertical or have attributes that are uncommon. A few challenges which occur while building such models are catalog quality, getting training data for uncommon attributes.
- Catalog Enrichment: Some of the customer catalogs are not very clean/structured. This is where we do catalog enrichment to include additional attributes that describe the product in a more structured manner. Catalog enrichment in itself is a very interesting and challenging problem. Once catalog enrichment is done, our NER models start performing well.
- Attribute selection for recognition
