VITON13School

Generative AI for BeginnersLesson 9 of 21

Semantic search with embeddings

Keyword search finds the words you typed; semantic search finds what you meant. Embeddings make that possible by turning text into lists of numbers that sit close together when the meanings are close. You will use them to let Study Buddy search your own notes.

Course by , Founder of VITON13 SCHOOL
  • Part 2: Build
  • 40 min

You will be able to

  • Explain the difference between keyword search and semantic search
  • Describe what an embedding is and why similar texts get similar vectors
  • Compute cosine similarity between two vectors and read the score
  • Build an in-memory semantic search over a list of notes
  • Decide when a plain list in memory is enough and when you need a vector database

Keyword search versus search by meaning

Classic search engines match words. If your notes say that plants make glucose from light and you search for how plants make food, a keyword search may return nothing, because the word food never appears. It also gets fooled the other way, returning every note that happens to contain a common word.

Semantic search compares meaning instead of spelling. It would connect food with glucose and make with produce, because in the language a model has learned those ideas appear in similar contexts. The trick that makes this work is called an embedding.

The two approaches are not rivals. Keyword search is still better for exact things such as an error code, a product number or a person's name. Many real systems run both and merge the results, which is known as hybrid search.

What an embedding is

An embedding model reads a piece of text and returns a vector: a fixed-length list of floating-point numbers. Depending on the model, the list may hold a few hundred or a few thousand values. You cannot read meaning from any single number, but the position of the whole vector in that space carries it.

The key property is that texts with similar meaning land near each other. A sentence about photosynthesis and a question about how plants feed themselves will produce vectors pointing in roughly the same direction, while a sentence about Python lists points somewhere else.

Embedding models are separate from chat models. You call a different endpoint, embeddings.create, and you usually pick a model built only for this job. Not every provider offers one; Anthropic, for example, does not serve embeddings, while OpenAI and Ollama do through the same OpenAI-compatible interface. That is why the example reads a fourth environment variable, LLM_EMBED_MODEL, and you may need to point LLM_BASE_URL at a provider that has embeddings for this lesson.

One rule matters more than any other: compare only vectors made by the same model. Vectors from two different embedding models live in unrelated spaces, so their similarity score means nothing. If you switch models, re-embed every stored note.

Cosine similarity in plain terms

To rank notes against a query you need a way to say how close two vectors are. The usual choice is cosine similarity, which measures the angle between them and ignores their length. Two vectors pointing the same way score close to 1, unrelated ones score near 0, and opposite ones approach -1.

The formula is short. Multiply the two vectors element by element and add the results to get the dot product, then divide by the length of each vector. Many embedding models already return vectors with a length of 1, and in that case the dot product alone gives the same ranking.

Do not treat the score as a percentage of relevance. The typical range differs between models, so a score that means a strong match for one model may be average for another. Look at the order of results first, and only set a cut-off after you have seen real scores on your own data.

  • Embed the query with the same model you used for the notes
  • Score the query vector against every stored vector
  • Sort from highest to lowest and keep the top few
  • Show the matching text, and later its source, to the user

From a list in memory to a vector database

For a few thousand short notes, a Python list and a loop are perfectly fine. Embed the notes once, keep the vectors alongside the text, and score every query against all of them. Save the vectors to a JSON file so you do not pay to embed the same notes on every run.

Long documents need chunking before they are embedded. One vector for a whole chapter blurs many ideas together, so split the text into passages of a paragraph or two, with a little overlap so a sentence cut in half still appears whole in one chunk. Lesson 16 covers chunking in depth.

When the collection grows to hundreds of thousands of chunks or many users search it at once, a loop becomes too slow. Vector databases and search engines with vector support store the embeddings, build an index for approximate nearest-neighbour search, and let you filter by metadata such as date or course. Options include dedicated vector stores, PostgreSQL with the pgvector extension, and general search engines that added vector fields.

Whatever you store, keep the original text and a source reference next to each vector. A vector alone cannot be turned back into words, and your users will want to see where an answer came from.

Putting search to work in Study Buddy

Search on its own is already useful: a learner types a vague question and gets back the two notes that best match it. That alone saves the scrolling and guessing that a long notes file usually demands.

The bigger win comes when you pass the top results to the chat model. Put the matching notes into the prompt and ask the tutor to answer only from them, saying clearly when the notes do not cover the question. This pattern is called retrieval-augmented generation, and it is the backbone of lesson 16.

Search quality depends on what you feed in. Short, focused notes with one idea each embed better than long paragraphs that mix topics. Remove duplicates, fix obvious typos and keep a date or source with each note so you can filter or explain results later.

Embedding calls cost money or compute, though far less than chat calls. Send notes in batches by passing a list to input rather than one request per note, and cache the results so the same text is never embedded twice.

Example

A tiny semantic search over study notes

import mathimport osfrom openai import OpenAI client = OpenAI(base_url=os.environ["LLM_BASE_URL"], api_key=os.environ["LLM_API_KEY"])EMBED_MODEL = os.environ["LLM_EMBED_MODEL"]  # an embedding model, not a chat model notes = [    "Photosynthesis turns light, water and carbon dioxide into glucose and oxygen.",    "The French Revolution began in 1789.",    "A Python list keeps items in order and can be changed after it is created.",    "Mitochondria release energy from glucose during cellular respiration.",] def embed(texts):    result = client.embeddings.create(model=EMBED_MODEL, input=texts)    return [item.embedding for item in result.data] def cosine(a, b):    dot = sum(x * y for x, y in zip(a, b))    return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b))) index = list(zip(notes, embed(notes)))  # embed the notes once, reuse for every query def search(query, k=2):    q = embed([query])[0]    scored = sorted(((cosine(q, vec), text) for text, vec in index), reverse=True)    return scored[:k] for score, text in search("How do plants make their food?"):    print(f"{score:.3f}  {text}")

Set LLM_EMBED_MODEL to an embedding model your endpoint offers, for example nomic-embed-text in Ollama after ollama pull nomic-embed-text. The plant question should rank the photosynthesis note first even though it shares almost no words with it.

Practice

Let Study Buddy search your notes

  1. Create notes.txt in your Study Buddy folder with at least fifteen one-line notes from a subject you are studying, one note per line.
  2. Change the example so it reads the notes from that file and saves the text and vectors to index.json the first time it runs.
  3. On later runs, load index.json instead of calling the embedding endpoint for the notes again.
  4. Write five questions in your own words that avoid the exact wording of the notes, and run each one through search.
  5. Run the same five questions as a plain keyword search, counting notes that contain any word from the question, and compare the top results.

Done when: The second run starts without embedding the notes again, and for at least four of your five questions the semantic search puts a correct note first while keyword search misses or ranks it lower.

Common mistakes

  • Embedding the notes with one model and the query with another, which makes every score meaningless.
  • Re-embedding the whole collection on every query instead of storing the vectors once.
  • Treating a similarity score as a fixed percentage and setting a cut-off before looking at real results.
  • Storing vectors without the original text and source, so results cannot be shown or cited.

Check yourself

Pick an answer to see whether it is right and why.

A learner searches for how plants make food, and the best note talks about glucose and light. Why can semantic search still find it?

Right.Embeddings place texts with related meaning close together, so the similarity score is high even with few shared words.

Not quite.The right answer is “The query and the note get embeddings that point in a similar direction”. Embeddings place texts with related meaning close together, so the similarity score is high even with few shared words.

What does cosine similarity measure?

Right.Cosine similarity depends only on direction, which is what carries meaning in an embedding space.

Not quite.The right answer is “The angle between two vectors, regardless of their length”. Cosine similarity depends only on direction, which is what carries meaning in an embedding space.

You switch to a new embedding model. What must you do with the vectors you already stored?

Right.Each model has its own vector space, so old and new vectors cannot be compared and the collection has to be embedded again.

Not quite.The right answer is “Re-embed all the stored texts with the new model”. Each model has its own vector space, so old and new vectors cannot be compared and the collection has to be embedded again.

From the founder

Use your real notes for the practice, not the sample sentences, because search only becomes interesting on messy text you actually wrote. Keep the five test questions; you will reuse them when we build retrieval in lesson 16.
Vitalii Tarasov, Founder of VITON13 SCHOOL

Course by Vitalii Tarasov, VITON13 SCHOOL. Draws in part on Microsoft's Generative AI for Beginners, MIT licence, Copyright (c) Microsoft Corporation. microsoft/generative-ai-for-beginners