Contextual Rephrasing in Google Assistant


When individuals converse with each other, context and references play a vital position in driving their dialog extra effectively. As an illustration, if one asks the query “Who wrote Romeo and Juliet?” and, after receiving a solution, asks “The place was he born?”, it’s clear that ‘he’ is referring to William Shakespeare with out the necessity to explicitly point out him. Or if somebody mentions “python” in a sentence, one can use the context from the dialog to find out whether or not they’re referring to a sort of snake or a pc language. If a digital assistant can not robustly deal with context and references, customers could be required to adapt to the limitation of the expertise by repeating beforehand shared contextual info of their follow-up queries to make sure that the assistant understands their requests and might present related solutions.

On this submit, we current a expertise presently deployed on Google Assistant that enables customers to talk in a pure method when referencing context that was outlined in earlier queries and solutions. The expertise, primarily based on the newest machine studying (ML) advances, rephrases a person’s follow-up question to explicitly point out the lacking contextual info, thus enabling it to be answered as a stand-alone question. Whereas Assistant considers many sorts of context for decoding the person enter, on this submit we’re specializing in short-term dialog historical past.

Context Dealing with by Rephrasing

One of many approaches taken by Assistant to grasp contextual queries is to detect if an enter utterance is referring to earlier context after which rephrase it internally to explicitly embrace the lacking info. Following on from the earlier instance wherein the person requested who wrote Romeo and Juliet, one might ask follow-up questions like “When?”. Assistant acknowledges that this query is referring to each the topic (Romeo and Juliet) and reply from the earlier question (William Shakespeare) and might rephrase “When?” to “When did William Shakespeare write Romeo and Juliet?”

Whereas there are different methods to deal with context, for example, by making use of guidelines on to symbolic representations of the which means of queries, like intents and arguments, the benefit of the rephrasing method is that it operates horizontally on the string degree throughout any question answering, parsing, or motion success module.

Dialog on a sensible show system, the place Assistant understands a number of contextual follow-up queries, permitting the person to have a extra pure dialog. The phrases showing on the backside of the show are solutions for follow-up questions that the person can choose. Nevertheless, the person can nonetheless ask totally different questions.

A Large Number of Contextual Queries

The pure language processing discipline, historically, has not put a lot emphasis on a basic method to context, specializing in the understanding of stand-alone queries which can be totally specified. Precisely incorporating context is a difficult downside, particularly when contemplating the big number of contextual question sorts. The desk beneath incorporates instance conversations that illustrate question variability and among the many contextual challenges that Assistant’s rephrasing technique can resolve (e.g., differentiating between referential and non-referential circumstances or figuring out what context a question is referencing). We exhibit how Assistant is now capable of rephrase follow-up queries, including contextual info earlier than offering a solution.

System Structure

At a excessive degree, the rephrasing system generates rephrasing candidates by utilizing several types of candidate turbines. Every rephrasing candidate is then scored primarily based on quite a few alerts, and the one with the best rating is chosen.

Excessive degree structure of Google Assistant contextual rephraser.

Candidate Era

To generate rephrasing candidates we use a hybrid method that applies totally different methods, which we classify into three classes:

  1. Turbines primarily based on the evaluation of the linguistic construction of the queries use grammatical and morphological guidelines to carry out particular operations — for example, the substitute of pronouns or different sorts of referential phrases with antecedents from the context.
  2. Turbines primarily based on question statistics mix key phrases from the present question and its context to create candidates that match widespread queries from historic knowledge or frequent question patterns.
  3. Turbines primarily based on Transformer applied sciences, comparable to MUM, be taught to generate sequences of phrases in accordance with quite a few coaching samples. LaserTagger and FELIX are applied sciences appropriate for duties with excessive overlap between the enter and output texts, are very quick at inference time, and aren’t weak to hallucination (i.e., producing textual content that isn’t associated to the enter texts). As soon as introduced with a question and its context, they’ll generate a sequence of textual content edits to rework the enter queries right into a rephrasing candidate by indicating which parts of the context ought to be preserved and which phrases ought to be modified.

Candidate Scoring

We extract quite a few alerts for every rephrasing candidate and use an ML mannequin to pick probably the most promising candidate. Among the alerts rely solely on the present question and its context. For instance, is the subject of the present question much like the subject of the earlier question? Or, is the present question stand-alone question or does it look incomplete? Different alerts rely upon the candidate itself: How a lot of the knowledge of the context does the candidate protect? Is the candidate well-formed from a linguistic perspective? And so on.

Not too long ago, new alerts generated by BERT and MUM fashions have considerably improved the efficiency of the ranker, fixing about one-third of the recall headroom whereas minimizing false positives on question sequences that aren’t contextual (and subsequently don’t require a rephrasing).

Instance dialog on a cellphone the place Assistant understands a sequence of contextual queries.


The answer described right here makes an attempt to resolve contextual queries by rephrasing them as a way to make them totally answerable in a stand-alone method, i.e., with out having to narrate to different info throughout the success section. The good thing about this method is that it’s agnostic to the mechanisms that might fulfill the question, thus making it usable as a horizontal layer to be deployed earlier than any additional processing.

Given the number of contexts naturally utilized in human languages, we adopted a hybrid method that mixes linguistic guidelines, massive quantities of historic knowledge by means of logs, and ML fashions primarily based on state-of-the-art Transformer approaches. By producing quite a few rephrasing candidates for every question and its context, after which scoring and rating them utilizing a wide range of alerts, Assistant can rephrase and thus appropriately interpret most contextual queries. As Assistant can deal with most sorts of linguistic references, we’re empowering customers to have extra pure conversations. To make such multi-turn conversations even much less cumbersome, Assistant customers can activate Continued Dialog mode to allow asking follow-up queries with out the necessity to repeat “Hey Google” between every question. We’re additionally utilizing this expertise in different digital assistant settings, for example, decoding context from one thing proven on a display screen or enjoying on a speaker.


This submit displays the mixed work of Aliaksei Severyn, André Farias, Cheng-Chun Lee, Florian Thöle, Gabriel Carvajal, Gyorgy Gyepesi, Julien Cretin, Liana Marinescu, Martin Bölle, Patrick Siegler, Sebastian Krause, Victor Ähdel, Victoria Fossum, Vincent Zhao. We additionally thank Amar Subramanya, Dave Orr, Yury Pinsky for useful discussions and assist.


Please enter your comment!
Please enter your name here