Testing non-deterministic software

Testing non-deterministic software means testing a program that can give a different answer to the same input each time it runs. Software built on an AI model, such as a chatbot or a voice agent, behaves this way. Ordinary tests do not work on it, because they check the output against one exact expected answer. So the tests for AI software judge whether each answer is acceptable, instead of whether it is identical to one answer.

Why does an AI model answer differently? A large language model writes its reply one word at a time, and at each step it picks from several likely words, partly at random. Ask it "When is my EMI due?" twice, and the wording will differ. Both answers may be right. The model can also give a wrong answer in a confident tone, which is called AI hallucination.

Here is how such testing works in general. A tester writes a scenario: who the user is, what the user wants and what a good result looks like. The good result is written as a rubric, a list of things the answer must and must not do. For example, it must give the right date, it must not promise a waiver, and it must stay polite. The test runs the scenario many times, often 20 or 50, because one good run proves little. A judge scores each answer against the rubric. The judge can be a set of fixed rules, a second AI model told to grade the answer, or a person checking a sample. The test passes if enough runs reach the required score, for example 95 of 100. For a conversation, the test plays the user's side too, and the whole conversation is scored, not one reply.

Take, for example, an insurer's voice agent that calls customers about renewals. Voice agent testing runs 300 imaginary callers against it, some angry and some speaking Hindi, and scores whether each call ended correctly.

Consider an online store that has an AI model write product descriptions. A test checks every description for the rules: under 80 words, the right size and colour, and no claims about health.

Say a bank adds a chatbot to its app. Before each release, the same 500 questions are asked again, and a judge model flags answers that got worse since the last version.

How is this different from ordinary automated testing? An ordinary test expects one exact result: the total must be ₹1,180, every time. A test for non-deterministic software accepts many different answers and rejects the bad ones, and it reports a pass rate rather than a simple pass or fail. It is also close to LLM evaluation, which measures a model on a fixed set of questions before it is chosen or released.