Here is the short version.
Most writing about embedding models right now is either marketing copy or research summaries that stop right before the part that matters to people who actually have to ship something. I got tired of reading both. Semantic Workshop exists so I can run the experiments myself, document what actually happens on ordinary hardware, and share the notes without pretending a single leaderboard score is a final answer.
I am Julian Whitmore. I live in Seattle, used to build ranking and retrieval systems at a large cloud company, and now spend most of my working hours testing EmbeddingGemma 2 and the surrounding retrieval stack on machines that fit under a desk. This first post is the origin story and the operating principles. Everything that follows will try to live up to them.
The Problem I Kept Running Into
A few months ago I needed a private semantic search system for a small collection of family documents and children’s books. Nothing exotic. Just something that would stay on my own hardware, return useful results, and not require me to upload private data to a hosted API every time I wanted to find an old note.
I did the usual thing. I read the model cards. I scrolled through Hugging Face discussions. I looked at a few blog posts that claimed “EmbeddingGemma 2 changes everything.” The claims were confident. The reproduction details were missing.
When I finally sat down to test it, the numbers I got on my own hardware did not match the polished demos. Throughput was lower. Memory use was higher than the “minimal requirements” suggested. Hybrid retrieval with BM25 helped more than the pure dense results suggested it would. The gap between the marketing story and the weekday reality was large enough that I started keeping a notebook.
That notebook is what became this site.
What Semantic Workshop Is (and Is Not)
Semantic Workshop is a set of field notes focused on EmbeddingGemma 2 and the practical problems around dense retrieval, hybrid search, RAG pipelines, and small-scale deployment. The editorial territory is deliberately narrow:
Independent, reproducible field notes on EmbeddingGemma 2 and practical retrieval systems.
I am not trying to cover every new model that drops on a Tuesday. I am not ranking models I have never run. I am not translating press releases into “independent” reviews. The goal is simpler and more useful: run the test, show the setup, report what happened, and be honest about the limits of the sample.
Every post should make it possible for a reader to answer four questions:
What was actually measured?
On which hardware, with which software versions, and under which conditions?
Against which baseline?
Under what circumstances would the result change?
If a post cannot answer those, it does not belong here.
The Persona Behind the Notes
I am a 38-year-old former search and recommendations engineer. I have a master’s in computer science and spent years working on vector retrieval, ranking pipelines, and offline evaluation. These days I work independently, consult a little, and maintain a small open-source hybrid-retrieval project that exists mainly so other people can reproduce the same experiments I am running.
Outside the technical work I am married with a four-year-old. Weekends are parks, libraries, and children’s museums. Writing happens in Seattle coffee shops between school drop-off and the next rain delay. The home lab is deliberately modest: a Mac Mini, a low-power NUC, and the occasional used GPU when the experiment actually needs it. Privacy, electricity cost, and the time required to keep a system running are treated as real engineering constraints, not afterthoughts.
That background shapes the voice. I assume you are intelligent and busy. I will start with the practical conclusion, show the steps, explain the principle, and then list the limitations. Personal details appear in the introduction or the closing notes when they help set context. The middle of every technical post stays focused on the experiment.
Editorial Rules I Will Follow
Do
Lead with the short answer.
Show commands, hardware specs, dataset details, parameters, and failure cases.
Explain metrics in plain English before using the acronyms.
Treat small samples as small samples.
Publish failed or inconclusive experiments when they teach something useful.
Keep the family details non-identifying. No names, no photos, no specific schools or routines.
Don’t
Publish rankings without having run the models myself.
Turn vendor marketing into an independent review.
Fabricate or selectively present results.
Use language like “revolutionary,” “destroys the competition,” or “changes everything.”
Pretend a weekend project is an enterprise platform.
The signature phrase I keep returning to is simple:
The benchmark is the conversation starter, not the verdict.
A single Recall@10 number on a small support dataset is interesting. It is not a universal ranking. It is the beginning of a conversation about what the number means under different hardware, different data distributions, and different failure modes.
Why EmbeddingGemma 2 as the Center of Gravity
There are many embedding models. I chose EmbeddingGemma 2 as the consistent center for two practical reasons.
First, it is small enough to run on the kinds of machines individual developers and small teams actually own. Second, the community discussion around it still leaves large gaps between the published numbers and the weekday experience of loading the model, embedding a real document collection, and measuring latency after the first request.
By keeping the same model at the center of most experiments, I can accumulate a coherent set of measurements instead of starting from zero every time a new model appears. The plan is not to claim it is the best model for every use case. The plan is to understand, in detail, what it does well, where it struggles, and how it behaves when you combine it with BM25, different vector databases, and different quantization settings.
The Four Categories That Will Organize the Work
Everything will live under one of four headings.
The Retrieval Bench
Controlled evaluations of EmbeddingGemma 2 against BM25, other embedding baselines, and simple hybrid configurations. Metrics explained through concrete examples rather than abstract definitions.
Reproducible Builds
Step-by-step guides with repositories, Dockerfiles, notebooks, and environment notes. The goal is that a reader can follow the same steps and get the same result on their own machine.
Small-System Reality
Hardware, memory, latency, throughput, electricity, cold starts, batching, and the actual monthly cost of keeping a small private retrieval service running.
Search in the Wild
Applications that stay close to real constraints: private document archives, children’s book collections that never leave the home, small-business FAQ systems, and the difference between clean demo data and the messy documents people actually own.
The first twenty posts are already mapped to these categories. This post is the opening note for The Retrieval Bench.
What the First Real Experiments Will Look Like
The next few posts will follow a consistent pattern.
I will start with a small, publicly describable support-style dataset so the evaluation is transparent. I will run EmbeddingGemma 2 against a BM25 baseline and a simple hybrid configuration. I will report Recall@10, MRR, and NDCG, and I will explain each metric with the same concrete example so the numbers are not floating abstractions.
Hardware details will be explicit: which machine, how much RAM was available, whether the model was quantized, what the batch size was, and what the latency looked like after the first request. I will also note the failures. Cold starts that surprised me. Memory spikes that forced a smaller batch size. Cases where the dense-only results looked strong until I looked at the actual retrieved documents.
That last part matters. Leaderboard scores can hide retrieval failures that only become visible when you read the results yourself. I will include those examples.
A Note on Scope and Honesty
I am one person with a modest home lab and limited time. The experiments will be careful, but they will not be comprehensive. Sample sizes will often be small. Hardware will rarely match a datacenter. When the result is narrower than the title might suggest, I will say so.
That limitation is also the point. Most developers who need retrieval do not have unlimited GPU budgets or dedicated evaluation teams. They have a Mac Mini, a deadline, and a collection of documents that must stay private. The measurements that matter are the ones taken under those constraints.
If a post ever drifts into untested claims or starts treating a single experiment as a universal ranking, call it out. The standard I am holding myself to is the same one I want the posts to meet: reproducible, limited, and honest about the limits.
Closing the Opening Note
I started this site because the gap between model claims and weekday reality kept costing me time. The solution was not another summary of someone else’s numbers. It was to run the tests myself, write down the setup, and share the notes.
The next post will be the first concrete comparison: EmbeddingGemma 2 versus BM25 on a small support dataset, with the full reproduction steps and the metrics explained through the same example. After that we will move into the Dockerized pipeline, the hybrid retriever, and the hardware measurements that determine whether any of this is practical on the machines people actually own.
Until then, remember the working rule:
The benchmark is the conversation starter, not the verdict.
See you in the next experiment.
No feedback yet — submit the first.