Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜.” In
Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–23. Virtual Event Canada: ACM.
“we understand the term language model (LM) to refer to systems which are trained on string prediction tasks: that is, predicting the likelihood of a token (character, word or string) given either its preceding context or (in bidirectional and masked LMs) its surrounding context.” (Bender et al., 2021, p. 611)
resource use
bias (gender, class, language, geography)
false narrative (coherence is not sentience)
“LMs are not performing natural language understanding (NLU), and only have success in tasks that can be approached by manipulating linguistic form” (Bender et al., 2021, p. 610)
