What an embedding really is

Lesson 4 of Build with LLMs introduced embeddings as coordinates for meaning and left it there, because that is all you need to build the first thing. This course is the second pass. Same idea, much closer up: what the numbers are, why the distance measure is the one it is, and where the whole approach quietly stops working.

1A fixed-length list of floats, and what it is not

An embedding model takes any text and returns the same shape of output every time: a list of floating point numbers, a few hundred to a few thousand long. Two sentences, forty pages, one word — same length out. That fixed length is the point. It makes text something you can put in an array and compare with arithmetic.

The model is trained so that texts meaning similar things come out with similar numbers. Nothing more mystical than that is going on. During training the model is repeatedly shown pairs that should be close and pairs that should be far, and it adjusts until the geometry agrees.

Now the part people get wrong. Here is what those numbers are not:

That last row causes real outages. If you re-embed half your corpus with a new model and leave the other half alone, search does not degrade gracefully — it returns confident nonsense from whichever half happens to score higher.

An embedding is a position, and a position only means something relative to the other positions in the same space. There is no meaning inside one vector.

2Cosine, and why not Euclidean

Picture each embedding as an arrow from the origin out to its point. Cosine similarity is the cosine of the angle between two arrows. Pointing the same way scores 1, at right angles scores 0, opposite scores -1. In practice you will see roughly 0 to 0.3 for unrelated text and 0.8 upwards for near-duplicates, and the useful range is narrower than you expect — get used to reading your own scores rather than trusting a threshold you read somewhere.

The reason it is cosine and not straight-line distance is direction versus magnitude. Arrow length tends to track things you do not care about, like how long the text is or how emphatic it is. Angle tracks topic. A one-line answer and a three-page article on the same subject point the same way but sit at different distances from the origin, and cosine ignores exactly that difference.

There is a catch worth knowing, because it saves you an argument. Most embedding APIs return unit-length vectors — already scaled so the arrow has length 1. On unit vectors, cosine similarity is just the dot product, one matrix multiply, no division. And Euclidean distance becomes a rearrangement of the same quantity, so it ranks results in exactly the same order. So:

The last row is a real bug source: build an index with cosine distance, query it with L2, and you get results that are wrong in a way no error message will tell you about.

# pip install sentence-transformers numpy
import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

texts = [
    "How do I reset my password?",                                   # the query
    "Use Forgot password on the sign-in page to recover your login.",  # the answer
    "Invoices are charged to the card on file each month.",            # unrelated
    "How do I avoid resetting my password?",                           # the opposite
]
v = model.encode(texts, normalize_embeddings=True)   # unit length
q = v[0]

print("cosine    ", (v @ q).round(3))                  # dot product == cosine here
print("euclidean ", np.linalg.norm(v - q, axis=1).round(3))
# Same ordering both ways: on unit vectors, d^2 = 2 - 2*cos.
# And look at the last entry - the opposite question scores near the top.

Run it. The score for the negated question is the lesson of section 3.

3What it is good at, and where it falls over

Embeddings are strong on the thing keyword search cannot do: matching meaning across different words. They are weak on the thing keyword search is trivially good at: exact strings.

None of these are fixable by picking a better model. They are properties of the approach. The fix is architectural: keep a keyword index alongside for identifiers, put dates and numbers in metadata columns you filter on, and never expect a vector to enforce a rule. Part 3 of this course builds that out.

One more asymmetry, and it explains a frontmatter field you may have skipped. A question and the passage that answers it do not look alike. "What is the refund window?" shares little with "Requests submitted within 30 days of purchase are honoured in full." Retrieval models are trained on exactly that mismatch, and many of them expose it as two modes — an input_type of query or document, or a short prefix string you prepend. Same model, two jobs.

Use them. Embedding your documents in query mode, or your queries in document mode, costs you real recall for no reason, and it fails silently: everything still returns results, they are just quietly worse. Lesson 2 comes back to this, because forgetting one half of the pair is one of the most common bugs in a working RAG system.