We will use a simple prompt that includes two examples and a task description:
Our prompt is geared for paraphrasing to simplify an input sentence. It contains two examples that demonstrate the task to the model. The sentence we want it to summarize is:
Killer whales have a diverse diet, although individual populations often specialize in particular types of prey.
We get several completions from the model via the API
The model suggests the following candidate summaries for the sentence:
"Killer whales have a diverse diet, although individual populations often specialize in particular types of prey."
|0||Killer whales have a diverse diet"||-3.208850|
|1||Its diet is diverse"||-3.487236|
|2||Their diet is diverse"||-3.761171|
|3||Different populations have different diets"||-6.415764|
|4||Their diet consists of a variety of marine life"||-11.764865|
In a lot of cases, better generations can be reached by creating multiple generations then ranking and filtering them. In this case we're ranking the generations by their average likelihoods.
It's worth spending some time learning the various hyperparameters of the generation endpoint. For example, temperature tunes the degree of randomness in the generations. Other parameters include top-k and top-p as well as
presence_penalty which can reduce the amount of repetition in the output of the model. See the API reference of the generate endpoint for more details on all the parameters.