# Fixing catalogue data for semantic search

By [Anubhav Choubey](https://anubhavchoubey.com/) · Data & Speed · 2026-11-03 · 3 min read

> Semantic search failed on missing and wrong product data. Gemini verified and enriched it, Pydantic kept it parseable and Typesense made it fast.

```stats
Thousands of SKUs · verified + enriched by Gemini · multilingual + Hinglish tags · Typesense
```

## The problem

Semantic search was not working well, and the search engine wasn't the reason. The **product data** was. A lot of it was missing, and a lot of what existed was wrong. Search can only be as good as what it searches, so better ranking alone wasn't going to fix it.

So this became more than a tagging job. The first step was **verifying the data**: checking every product and correcting what was wrong, not just adding to it.

## What Gemini filled in

For each product, Gemini helped with:

- **Descriptions** where they were missing or poor.
- **Categories** that were consistent across the catalogue.
- **Tags** for what the product is and what people might call it.
- **Tags in multiple languages, including Hinglish**, because people search the way they speak.

With that data in place, both **semantic search** and **semantic product suggestions** got better. Search started to understand intent, not just matching words: someone searching for *red lays* would be shown *Lay's Tomato*, because the data now described the flavour and how people refer to it.

## Guardrails: choose, don't invent

Letting a model write free-form tags is how a catalogue fills up with near-duplicates and made-up categories. So the output was a mix of two things:

- **Fields picked from a strict, fixed list.** For things like categories, the model could only choose from allowed values.
- **Free text** only where it was needed, such as descriptions.

```callout tip
Where a field has a right answer, give the model the list to choose from. A guardrail that limits the choices beats any amount of checking afterwards.
```

## Parsing: the early problem

In the beginning, the hard part was simply reading the responses. The model's output didn't always come back in the shape the code expected, and those records broke the pipeline. Switching to a **Pydantic response format** solved it: every response had to match a defined schema, so it was either valid structured data or a clear error.

## One big run, then a pipeline

The whole catalogue was processed once to fix what already existed. After that, an **automatic pipeline** took care of new products, so the data stayed clean without anyone running it by hand.

```text
product ──▶ verify ──▶ Gemini (fixed lists + free text) ──▶ Pydantic schema ──▶ Typesense
```

## Typesense for speed

The clean data was indexed in **Typesense**, which made search much faster and handled the semantic search itself. Fast search on bad data is still bad search, so the data work came first and the index came second.

## What I'd do differently

- **Better guardrails.** Tighter limits on what the model can return, in more fields.
- **A better validator model.** A separate, stronger check on the model's output before it reaches the catalogue, instead of relying mostly on format checks.

## Takeaway

When search isn't working, look at the data before the search engine. Verifying and enriching the catalogue, with a model that had to choose from fixed lists and answer in a strict schema, is what made semantic search actually understand what people were looking for.