This post explains how large language models work, including how they represent words using vectors, how they predict the next word, and how they are trained. It also discusses the surprising performance of GPT-3 on tasks requiring high-level reasoning and its potential to understand meanings of words.
Table of contents
Word vectorsWord meaning depends on contextTransforming word vectors into word predictionsCan I have your attention pleaseA real-world exampleThe feed-forward stepFeed-forward networks reason with vector mathThe attention and feed-forward layers have different jobsHow language models are trainedThe surprising performance of GPT-31 Comment
Sort: