Tax-code classification: call the expensive model only on doubt

Auditing invoices means checking, line by line, whether the declared tax code matches the product described. A mid-sized accounting firm processes tens of thousands of those lines a month.

The naive solution is to call a language model for every line. It works, and the bill explodes: cost grows linearly with volume, and volume is exactly why the problem exists. Worse — the model's answer is hard to audit afterwards, and auditing is the product.

The pattern: cheap common path, expensive exception

What we built inverts the order. Every line goes first through a deterministic classifier running inside the database, comparing the tokens of the product description against the base of official tax-code descriptions. That classifier returns two things: a suggestion and a confidence measure.

When confidence is above the threshold, the answer is accepted and no model is called. When it is below, and only then, the line goes to a language model.

invoice line
     ↓
token classifier (Postgres)
     ↓
confidence ≥ threshold ?
     ├── yes → answer accepted        ← common path, zero cost
     └── no  → language model         ← exception, cost per call
                     ↓
            answer feeds the base back

In practice, the overwhelming majority of a firm's line items repeat month after month. After the first few rounds the common path absorbs nearly everything and the model is only invoked on what is genuinely new or badly described.

The gain that isn't the obvious one

The cost saving is real, but it isn't the main benefit. The main one is auditability.

When the classification comes from the deterministic engine, you can show the accountant exactly why that code was suggested: which tokens matched, against which entry, with what score. That is defensible in a tax inspection. "The model classified it that way" is not.

Having a deterministic common path isn't about saving money. It's about saving explanation.

A base that feeds itself

When the model is invoked and its result is confirmed by human review, that description-to-code pair goes back into the token base. The deterministic classifier gets better with use, and the share of model calls falls over time.

Worth recording what we chose not to use: there is no vector database and no embeddings at this stage. Product descriptions on invoices are short, full of abbreviations and internal codes — terrain where literal token search beats semantic search comfortably, and without the extra infrastructure.

How to pick the confidence threshold

It is the only genuinely delicate decision in the design, and it isn't technical: it's a business call. Too low a threshold accepts bad classifications and undermines trust in the whole report. Too high sends excessive volume to the model and hands the cost problem back.

What worked was starting conservative — high threshold, lots going to the model — and lowering it gradually, measuring the classifier's hit rate against human reviews. Guessing the threshold up front is what makes this kind of system launch already distrusted.

Where else this pattern fits

The shape is general and applies to virtually any classifier in production:

1. 1Solve the common path with a deterministic rule: cheap and explainable.
2. 2Measure the confidence of that answer — without a confidence measure there is no way to decide when to escalate.
3. 3Call the expensive model only when confidence is low.
4. 4Feed confirmed results back into the base, so the common path grows.

Ticket triage, email routing, expense categorization, record deduplication: all share the same long-tail shape, where few cases are hard and most repeat. Treating every case as if it were hard is the most common waste in applied AI projects.