Positional Embeddings
Token embeddings alone throw away word order. How position gets encoded into a sentence, and why the sinusoidal scheme is built the way it is.
–––
Topics / Embeddings
Embeddings
2 posts.
Token Embeddings
How tokens turn into meaningful vectors: the core idea behind embeddings, how the training data is prepared, and a working implementation.
–––