Skip to main content

Sentiment Analysis

In this section, we show how to use the Choose Best endpoint to do sentiment classification for reviews of a bakery.

The problem we want to solve#

For this demo, let's assume that we want to classify a set of reviews for a bakery into positive and negative classes. We might for instance have a review like this:

'Simply best cookies in town. The hype everyone has is real. A bit too sweet so perfect to share with someone.'

that we want to automatically classify as a positive review.

Naturally, the same techniques that we'll use for this problem can be used for any other task where we want to classify a given text according to a fixed set of classes.

Using Choose Best for our task#

Choose Best takes in a query as well as a list of options, and returns scores that indicate how likely each of the options is to follow (or preceed, depending on the request) the given query. In our case, we could pass a given review as the query followed by a prompt such as The sentiment of this text is and the different classes, e.g. positive and negative, as options to Choose Best. The scores we obtain from the endpoint then direclty indicate how likely it is that a given review falls into the given classes.

Putting the above together, we can then call the API with the following arguments:

  • Query: Simply best cookies in town. The hype everyone has is real. A bit too sweet so perfect to share with someone. The sentiment of this text is ,
  • Option 1: positive
  • Option 2: negative
  • Mode: APPEND_OPTION

This then gives us the following return value:

{
"scores": [
-0.004,
-5.379
]
}

which indicates that, as we would expect, our model thinks that the review is overwhelmingly positive.

Note#

There are a few design decisions with regards to the query and the options that can greatly impact the quality of the classification we get from the models when using Choose Best:

  • In general, differences in the formulation of the options that seem small to us can have a large impact on the output of the model. (See prompt engineering)
  • Try to make the combination of query and a given option sound like something that you would write yourself, or something that you might read online. In our sentiment analysis example this means that we add The sentiment of this text is so that it's natural for the options, positive and negative, to follow this prompt.