The Classify Endpoint

LLMs have been pre-trained with a vast amount of training data, allowing them to capture how words are being used and how their meaning changes depending on the context. A very common application of this is text classification. Cohere’s Classify endpoint makes it easy to take a list of texts and predict their categories, or classes. If you have read a little bit about text classification, you may have come across sentiment analysis, which is the task of classifying the sentiment of a text into a number of classes, say, positive, negative, or neutral. This is useful for applications like analyzing social media content or categorizing product feedback.

For example, a human can easily tell you that “Hello, World! What a beautiful day” conveys a positive sentiment, but let’s see if our models can do that too. And while we’re at it, let’s try classifying other phrases that you might find on social media.

Colab Notebook

This chapter comes with a corresponding notebook, and we encourage you to follow it along as you read the chapter.

For the setup, please refer to the Setting Up chapter at the beginning of this module.

Prepare Input

A typical machine learning model requires many training examples to perform text classification, but with the Classify endpoint, you can get started with as few as five examples per class. With the Classify endpoint, the input you need to prepare is as follows:

Examples

• These are the training examples we give the model to show the output we want it to generate.
• Each example contains the text itself and the corresponding label, or class.
• The minimum number of examples required is two per class.
• You can have as many classes as possible. If you are classifying text into two classes, that means you need a minimum of four examples, and if you have three, that means you need six examples, and so on.

Inputs

• These are the list of text pieces you’d like to classify.

Our sentiment analysis classifier has three classes with five examples each: “Positive” for a positive sentiment, “Negative” for a negative sentiment, and “Neutral” for a neutral sentiment. The code looks as follows.

The examples:

from cohere import ClassifyExample

examples = [
  ClassifyExample(text="I'm so proud of you", label="positive"), 
  ClassifyExample(text="What a great time to be alive", label="positive"), 
  ClassifyExample(text="That's awesome work", label="positive"), 
  ClassifyExample(text="The service was amazing", label="positive"), 
  ClassifyExample(text="I love my family", label="positive"), 
  ClassifyExample(text="They don't care about me", label="negative"), 
  ClassifyExample(text="I hate this place", label="negative"), 
  ClassifyExample(text="The most ridiculous thing I've ever heard", label="negative"), 
  ClassifyExample(text="I am really frustrated", label="negative"), 
  ClassifyExample(text="This is so unfair", label="negative"),
  ClassifyExample(text="This made me think", label="neutral"), 
  ClassifyExample(text="The good old days", label="neutral"), 
  ClassifyExample(text="What's the difference", label="neutral"), 
  ClassifyExample(text="You can't ignore this", label="neutral"), 
  ClassifyExample(text="That's how I see it", label="neutral")            
]

The inputs (we have twelve in this example):

inputs=["Hello, world! What a beautiful day",
        "It was a great time with great people",
        "Great place to work",
        "That was a wonderful evening",
        "Maybe this is why",
        "Let's start again",
        "That's how I see it",
        "These are all facts",
        "This is the worst thing",
        "I cannot stand this any longer",
        "This is really annoying",
        "I am just plain fed up"
        ]

Get output

With the Classify endpoint, setting up the model is quite straightforward. The main thing to do is to define the model type. For the Classify endpoint, we need to use an embedding model, which we'll useembed-english-v3.0(we'll learn more about embedding models in the next chapter).

Putting everything together with the Classify endpoint looks like the following:

co = cohere.Client(api_key)

def classify_text(inputs, examples):

  response = co.classify(
    model='embed-english-v3.0',
    inputs=inputs,
    examples=examples)
  
  classifications = response.classifications
  
  return classifications

predictions = classify_text(inputs,examples)

Together with the predicted class, the endpoint also returns the confidence value of the prediction (between 0 and 1). These confidence values are split among the classes, in this case three, in which the values add up to a total of 1. The classifier then selects the class with the highest confidence value as the “predicted class.” A high confidence value for the predicted class therefore indicates that the model is very confident of its prediction, and vice versa.

Here’s a sample output returned:

Input: Hello, world! What a beautiful day
Prediction: positive
Confidence: 0.83
----------
Input: It was a great time with great people
Prediction: positive
Confidence: 0.99
----------
Input: Great place to work
Prediction: positive
Confidence: 0.91
----------
Input: That was a wonderful evening
Prediction: positive
Confidence: 0.96
----------
Input: Maybe this is why
Prediction: neutral
Confidence: 0.70
----------
Input: Let's start again
Prediction: neutral
Confidence: 0.83
----------
Input: That's how I see it
Prediction: neutral
Confidence: 1.00
----------
Input: These are all facts
Prediction: neutral
Confidence: 0.78
----------
Input: This is the worst thing
Prediction: negative
Confidence: 0.93
----------
Input: I cannot stand this any longer
Prediction: negative
Confidence: 0.93
----------
Input: This is really annoying
Prediction: negative
Confidence: 0.99
----------
Input: I am just plain fed up
Prediction: negative
Confidence: 1.00
----------

The model returned a Positive sentiment for “Hello, world! What a beautiful day,” which is what we would expect! And the predictions for all the rest look spot on too.

That was one example, but you can classify any kind of text into any number of possible classes according to your needs.

As your task gets more complex, you will likely need to bring in additional training data and finetune a model. This will ensure that the model can capture the nuances specific to your task and realize performance gains. You can read more about finetuning representation models in one of the following chapters.

If you’d like to learn more about text classification, here are some additional resources:

• Intuition and use case examples
• An example application: toxicity detection
• Evaluating a classifier’s performance
• Classify endpoint API reference

Conclusion

In this chapter, you've learned to classify text based on mood using Cohere's Classify endpoint. Follow along the next chapters to learn other endpoints and how to use them for many other tasks!

Original Source

This material comes from the post Hello, World! Meet Language AI: Part 2