Learning Resource

Learn some of the key concepts involved in language generation and understanding.

• Tokens are words or parts of words that our models take as input or produce as output.
• Embeddings are lists of numbers that represent a word, token, or an entire piece of text. Embeddings capture information about the meaning and context of the words or sentences they represent.
• Temperature is a value that controls the outputs of a generation model by tuning the degree of randomness involved in picking output tokens.
• Likelihood is a measure of how “expected” each token is in a piece of text.
• Model Evaluation: Learn how likelihood scores make it possible to evaluate generative models