Search, and You Will Find: Predicting Users’ Intent
Posted: 28 Jan 2021
Date Written: November 4, 2020
In the age of big data the amount of content is increasing all while users expect to find information more quickly. As more and more of our users were raised with Google their expectations on search have also been raised. While in the past we asked users to neatly spell out their query into individual fields, products now move towards the “single search box”.
This is why we want to understand better what exactly users are searching for, and we want to engage users in a conversational search experience to guide them quickly to relevant information. Eventually we want to present the user with a powerful, feature-rich search that is still easy and intuitive to use.
A thorough analysis of our query logs showed different usage patterns that fall broadly into two categories. On the one side stand exploratory queries in which a user is looking for information about a particular topic or for documents containing certain keywords. On the other hand, however, nearly half of the queries are targeting a concrete entity, for us mostly specific scientific articles but also authors, organizations or journals. Users implicitly expect very different results to these query types.
This makes it important to determine the user's intent, primarily by analyzing the query, but also using additional features. We will discuss the particular challenges that we are facing in doing NLP on queries and how we are addressing problems like ambiguous terms, defective syntactic structure and limited context.
Using a range of methods – from simple patterns and dictionary lookups to word embeddings, conditional random fields and transformer models – we will present solutions to recognize "exotic" entities like publication years, author names and article titles, and we will give a taste of our latest work on question answering and dynamic, search history based suggestions.
Suggested Citation: Suggested Citation