How we build it

How We Test That an App Really Remembers a Food Allergy

By the RecipePT teamAugust 12, 20267 min read

Saying an app remembers your allergy is easy. Proving it is a different job, because people don't announce an allergy in one tidy sentence. They say "my daughter can't have nuts," or "my throat closes up," or they type a doctor's instruction with the accents missing. So before we shipped RecipePT in ten new languages, we wrote 2,153 ways to say "I can't eat that" and spent two days trying to break our own memory with them. Ten of our eleven test suites failed the first run.

How do you test whether an app actually remembers a food allergy?

The hard part isn't collecting phrases that clearly mean allergy. Any system catches "I'm allergic to peanuts." The useful test cases are the ones sitting one word away from something we're supposed to ignore. So each suite pairs them deliberately. "Make it without shrimp tonight" has to be forgotten by tomorrow. "Always cook without shrimp" has to survive forever. Same food, same grammar, opposite answers.

We also graded the two kinds of failure differently, because they aren't equally bad. Storing a rule we should have let go is annoying: you get dinners without mushrooms for a while until you tell us otherwise. Dropping a real allergy is the failure that matters. So the gate is asymmetric. Any missed allergy blocks the language from shipping. Over-storing gets logged and reviewed, and it never blocks anything.

What did the test actually find?

Reading the miss list is the part that changed how we think about this feature. A sample of what our own app shrugged at:

• "Von Nüssen schwillt mir der Hals zu." Nuts make my throat swell shut. That's anaphylaxis described without the word allergy anywhere in the sentence.
• "Meine Tochter hat eine schwere Nussallergie." Someone else's body, but it's still the kitchen's rule. The cook isn't the only person eating.
• "Jag får utslag av jordgubbar." Strawberries give me a rash. A described reaction, not a diagnosis.
• "医者に止められているので甲殻類は使わないでください." My doctor has told me not to, so please don't use crustaceans. A medical instruction, relayed politely.
• "Tengo el ácido úrico alto, nada de marisco." High uric acid, no shellfish. Extremely common phrasing in Spain, and we ignored it.
• "soy selaco." Celiac, misspelled the way people actually type it in a hurry.

Six of the German misses matched patterns the classifier had already been told to save. The instruction said to store "I'm not allowed to eat X," and the test typed "I'm not allowed to eat X," and it was ignored anyway. That told us the problem wasn't a missing rule. Something was pushing every uncertain call toward forgetting.

Why did one language pass when ten failed?

Dutch had one other thing none of the others did. It spelled out three phrasings that look nearly identical and mean completely different things: mag geen and kan niet tegen are medical and get saved, lust geen is a preference and gets dropped, heb geen meer is about the pantry and gets dropped. A system that treats all three as the same word either loses allergies or starts memorizing your shopping list.

That gave us the fix. We rewrote the nine failing languages on the Dutch shape: qualify the doubt rule, then name the categories out loud. Reactions described physically. Doctors and medications. Household allergies. Halal, kosher, vegan, and other standing household rules. Disgust strong enough to be permanent. Explicit instructions to add something to a list. Every phrasing from the miss list went in as a worked example, with the ignore cases mixed in beside them so the list couldn't drift back toward forgetting.

Did the rewrite work?

Two stragglers needed a second pass. French still dropped "je ne supporte pas l'odeur du poisson," because a rejection aimed at the smell of a food is still a rejection of the food. Norwegian still dropped "nøtteallergi her," a bare noun with no sentence around it, which is exactly how people type in a chat box.

Why re-run a test that already passed?

That single result reshaped how we treat the whole feature. A green run is evidence, not a certificate. So the suites are checked into the repo and re-run whenever the prompts change, rather than being a one-time launch chore.

We also found a bug in the test rig itself, which is a humbling thing to admit in public. When the classifier timed out, it returned an empty result, and an empty result looks exactly like a confident decision to forget. Three "misses" across our runs were actually network timeouts wearing a costume. The gate now retries an empty result on any case that's supposed to be remembered, and logs every retry. The final Portuguese and Norwegian passes used zero retries, so those scores are real.

What does this mean if you cook around an allergy?

Working by meaning is the point. A word filter that removes "peanut" sails right past satay, and it also throws out peanut-free things for containing a similar string. We covered that failure in our piece on dinner apps for picky eaters. Judging by meaning catches the satay. It's also, honestly, a judgment, which is why we test it this hard and why we tell you to read the recipe rather than promising you'll never need to.

The reason we care is the same reason the app exists. Re-explaining your constraints every single night is its own kind of exhausting, and it stacks on top of the decision fatigue that made dinner hard in the first place. Say it once. We'll carry it.

Back to the RecipePT blog