Model Limitations

The following factors may impact our language models’ performance.

• Language: Due to the lack of available training data and evaluation datasets for the majority of the world’s languages, the model is unlikely to perform well on languages other than the dominant dialects of English (Joshi, 2020; Dodge, 2021). We are actively working to increase the number of languages supported by the Cohere API.

  Example: Our language models may fail to meaningfully represent non-English phrases.

• Socio-economic: The majority of publicly available data used to train the model is from wealthier individuals in more developed countries, and is largely Western-centric (Pew, 2021). As a result, performance will likely degrade on text about concepts, people, and places from other regions, especially that of the Global South.

  Example: The models may prefer phrases and ideas associated with Western ideals, or with wealthier and more technologically developed cultures.

• Historical: At any point, the model will only represent the concepts, events, places, and people from data on which it was trained. Information from after the dataset was gathered will not be represented. For example, if an event occurred today, the model would not be able to return a meaningful representation of the event name. Additionally, the model may amplify outdated societal biases about groups of people.

  Example: The models may fail to generate the correct answer to a factual question if the information regarding that fact has recently changed since the models were trained.

• Ungrounded: Model outputs are derived statistically rather than from any direct modeling of the meaning of words and phrases, and therefore they should not be interpreted as a grounded means for understanding text (Bender, 2020).

• Biases: Language models capture the hegemonic viewpoint, reflecting and magnifying biases that exist on the internet (Bender,2021). As a result, marginalized groups can be harmed by entrenching existing stereotypes, or producing demeaning portrayals (Crawford, 2017). Despite our active efforts to mitigate these biases, we acknowledge that this is an ongoing research area (Gonen, 2019).

  Example: The models may associate gender, racial, and other identities with professions and concepts which are semantically unrelated to those identities. These associations are likely to reflect biases present in the historical data the model was trained on. For research on bias in embeddings, see, for example, (Kurita et al., 2019); for a survey of the space of language model generation bias, see (Sheng et al., 2021).