LLMs contain a LOT of parameters. But what’s a parameter?
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
Parameters are the numerical values that control how large language models behave. They come in three types: embeddings (mathematical representations of words in high-dimensional space), weights (connection strengths between model parts), and biases (threshold adjustments). During training, billions of these parameters get
Table of contents
What is a parameter?How are they assigned their values?Sounds straightforward …Oof. What are all these parameters for, exactly?Okay! So, what are embeddings?Sort: