Every row I sent to the LLM cost me money. So I started asking the cheapest possible question: have I already seen this one?
The idea for Label Buddy tool came from my colleague Mohana. She works with our customers every day, and she was the one who saw that bad labels were quietly corrupting what our agents learned. She built the first version to prove it out. I picked it up from there and rebuilt it into something that could run a real customer file end to end. You upload a CSV of marketing copy variants, define what each label means, and an AI pipeline classifies every row as Valid, Review, Invalid, or Skipped, with a plain-language reason for each call.
The scale is the whole reason it exists. At Aampe, one agent is assigned per end user, and those agents learn from the labels on every message. For example, a single customer can generate tens of thousands of copy variants, and a real file runs to around 140,000 rows. At that size, “just send every row to the model” is a non-starter on both cost and time.
So the first answer I reached for was a SHA-256 hash. Normalize the text, hash it, and if the exact same row shows up again, reuse the result. Fast. Zero ambiguity.
One detail that turned out to matter: the hash isn’t of the text alone. It’s computed over the component type, the label, and the normalized text together. The same sentence can be Valid under one label and Invalid under another, so identical words aren’t automatically the same row. The hash has to carry the label, or you collapse rows that should stay separate.
But that got me thinking. What about rows that aren’t identical, but close enough to be the same thing? That question cracked open something I hadn’t thought much about before.
What is “similar enough”?
When you start asking this seriously, you quickly run into a spectrum of tools.
Jaccard similarity is the most stripped-back version. You tokenize two strings into sets of words, then compute:
J(A, B) = |A ∩ B| / |A ∪ B|How much do they share, divided by everything they collectively contain. A score of 1 means identical. A score of 0 means nothing in common. No models, no API calls, no infrastructure. Just sets.
For example:
“Save money every month” → {save, money, every, month}
“Save more money each month” → {save, more, money, each, month}
Intersection: {save, money, month} = 3
Union: {save, money, every, month, more, each} = 6
Jaccard score: 3/6 = 0.5 — meaningfully similar, not identical
Cosine similarity on embeddings is what ended up in the actual pipeline. First you run the text through an embedding model, which turns the string into a long list of numbers, a vector that represents its meaning. Two sentences that mean similar things produce vectors pointing in roughly the same direction.
Cosine similarity measures the angle between those two vectors. A score of 1 means they point the same way and are semantically identical. A score of 0 means they're unrelated.
For example:
“Affordable plans for every budget” and “pricing that won’t break the bank” share zero words
Jaccard score: 0 — looks completely different
Cosine score: likely 0.85+ — because the meaning is nearly the same
That’s the gap Jaccard can’t cross. Once words diverge but meaning converges, you need embeddings. The tradeoff is that every comparison requires an API call, a model, and latency none of which Jaccard needs.
Why both concepts matter
The honest reason cosine is in the pipeline and Jaccard is not: the embeddings stage was generated as part of the build. But learning about Jaccard while building this is what made me understand why the pipeline is layered the way it is.
The full pipeline runs eight stages, in strict order:
Parse → Normalize → Ignore → Dedup + Cache → Rules → Embeddings → LLM → Finalize
Here is what each one is doing, and where the money goes:
Parse. Stream the CSV into the database in chunks of a thousand rows.
Normalize. Lowercase and trim, then compute the SHA-256 hash of type, label, and text. This is the fingerprint everything downstream relies on.
Ignore. Any label toggled off in setup resolves to Skipped and leaves the pipeline here. It never reaches the model and never costs anything.
Dedup and cache. Identical rows collapse to one. A semantic cache from earlier runs fills in matches in bulk. This is the quietest cost win in the whole thing: if I validated this exact row last week, I’m not paying to validate it again.
Rules. Deterministic checks with no AI cost. Empty text, placeholder copy like “Lorem ipsum”, all-caps, length outliers.
Embeddings. Cosine similarity against a library of curated exemplars resolves the clear matches before any generation. This runs on the OpenAI path using text-embedding-3-small.
LLM. Only what survives all of that reaches GPT-4o-mini, with Claude Haiku as the fallback, in batches of 25, with each label’s strictness and overlap rules applied.
Finalize. Fan the results back out to every duplicate, write the output, update the counts.
Each layer filters what reaches the next. Dedup and cache catch exact and previously-seen copies. Embeddings catch semantic near-matches. The LLM handles only what is genuinely nuanced.
One thing worth being precise about: the embeddings stage is specific to the OpenAI path. When you run on Claude, there’s no cosine pre-pass, so the rows that embeddings would have resolved go straight to the model instead. The savings I was chasing in the title come from dedup, cache, and rules on every provider, and from embeddings on top when you use OpenAI.
Jaccard would have fit right between dedup and embeddings, as a cheap pre-pass for near-duplicate text before spending on embedding calls. It didn’t make it into the build. But understanding it clarified why the layers exist at all.
The real lesson
There is a temptation when building with LLMs to throw everything at the model and let it figure it out. But every row you send costs time and money, and a system that does too much in one step is harder to debug when it gets something wrong.
Understanding similarity, even just conceptually, even from a formula that’s over a hundred years old, changes how you think about where the smart part of your system should begin.
You don’t need transformers to catch a duplicate. You don’t need cosine to skip an identical string. And you don’t need an LLM to know that two rows are close enough to share an answer.
The LLM does what only it can do. The math handles the rest.
Is this the final solution? Probably not.
This pipeline works for what I needed it to do right now. But I'm still figuring it out, and honestly, that's the part I enjoy the most. There are better architectures, smarter retrieval strategies, and tools I haven't stumbled into yet. This post isn't a blueprint. It's a snapshot of thinking-in-progress. The rabbit hole is still open 🕳️
If you have built something with LLMs and found yourself reaching for older, simpler tools to make it work, what was it?