In this section, we show how to use the Classify endpoint to do sentiment classification for customer satisfaction survey responses an e-commerce website may receive.
For this demo, let's assume that we want to classify a set of reviews for a newly released feature into positive and negative classes. We might for instance have a review like this:
The item exceeded my expectations
that we want to classify as a positive review.
Naturally, the same techniques that we'll use for this problem can be used for any other task where we want to classify a given text according to a fixed set of classes.
Classify takes in example inputs with their labels, as well as the input texts we aim to classify. It then creates a classifier using the power of a generative model and its ability to learn from a small number of examples (few-shot learning). The classifier is made by creating a classification input prompt from the examples and from two optional pieces of texts we can pass to improve the classifier: a task description and an output indicator.
Labeled examples are used to demonstrate the classification task to the model so it grasps the task. Examples provide two important pieces of information: 1- The inputs and expected outputs for the task we're interested in. 2- The number of output classes. Every class should appear in at least one example in the labeled examples.
In this case we will be passing in the following examples:
These are the input texts that we would like to classify
The task description is used to describe the type of classification we would like - in this case we’d send something like
The following is a sentiment classifier regarding customer orders for an e-commerce company.
This is a string of text appended to the end of every input text. In this case we’d like to append
this is: after every example we pass in.
Adding everything above together, we can call the API with the following arguments:
[Example("The order is 5 days late","negative"), Example("The order came 5 days early","positive"), Example("I would recommend this to others","positive"), Example("The package was damaged","negative"), Example("The order was incorrect","negative"), Example("The item exceeded my expectations","positive"), Example("I want to return my item","negative"), Example("I ordered more for my friends","positive"), Example("The item's material feels low quality","negative")]
["This item was broken when it arrived","This item broke after 3 weeks"]
The following is a sentiment classifier regarding customer orders for an e-commerce company
The corresponding code snippet for the API call is as follows.
It gives us the following values:
which indicates that, as we would expect, our model thinks that both texts are negative.
The playground has a user interface to help you set up the classification prompts, which can then be exported as code.
In the Cohere Playground, click on ‘Classify’. Select the model size of your choice. Smaller models are faster, while larger models have a better grasp of language and are more able to capture and replicate the patterns in the input prompt.
Add your labeled examples in the ‘Examples’ section. The first column is for the examples while the subsequent columns are for the associated labels. Our example here consists of 2 labels but there is no limit as to how many labels you can specify, depending on the task you have.
You can also add your labeled examples using a CSV file by selecting 'upload your labelled examples'.
Add at least 5 examples for each label. The more examples you have, the higher the chance of getting more accurate outcomes.
If you are not sure, there are also some preset examples to help you get started.
Add the inputs you want to classify in the ‘Inputs’ section. Once done, click on ‘Classify’ to start the classification step.
Once the classification step is completed, you will see the output labels next to the inputs you added.
In the ‘Results’ section, you will also see the confidence levels associated with each output. The confidence level represents the model's degree of certainty that the query falls under a given label. The label with the highest confidence is chosen.
Now the code is ready to be exported. Click on ‘Export code’ and you can choose to export from a few different options. You can use this to start integrating the current API configuration to your application.
You can opt to finetune a model if you have a dataset of at least 250 labeled examples (500 or more for best results) with at least 5 examples per label.
A finetuned model can potentially lead to a better classification performance than a baseline model. See here to get an overview of what finetuning is about.
To finetune a model for your classification task, click on “create a finetuned model”.
The subsequent steps are the same as how you would finetune a representation model. For this, follow the steps described here.