← Back to projects

Project 05 / Case study

Game engine · local AI · verification

Rules that do not depend on the AI being right.

Complex game rules, a deterministic engine and a language model running on the same computer that decides for the players. The interesting part is not that the machine plays well: it is that the engine does not break if the model fails, disconnects or writes nonsense. This is what designing around a part you cannot trust looks like.

3,959lines of Python
24card effects
100rounds per test profile
0external dependencies
RoleRules, engine and verification design
EnvironmentWindows · Python 3.12 · local Ollama
TechnologiesPython, tkinter, urllib, unittest
01

The challenge

A card game with chained effects, phases, bets and a castle that gets stormed. The rules fit in the head of the person who designed them and nowhere else. The only way to know whether they are fun, and above all whether they are consistent, is to play them many times, and nobody plays a thousand games to check one sentence of the rulebook.

02

The solution

A deterministic engine that is the absolute owner of the state, and a language model that is asked for a decision in JSON and answered with a closed vocabulary. Every card effect is written as an engine function, not as text that the AI has to interpret. If the model fails twice in a row, it is switched off for the rest of the game and a local heuristic finishes the job.

03

What I learned

That delegating to a model is delegating to something that can return anything, and that the design question is not how to make it get it right, but what happens when it gets it wrong. The one hundred and seventy-four engine functions are fine; the part that made me grow is the forty-line normaliser that rejects what the game cannot represent.

Architecture

The engine decides. The AI suggests.

A single file of three thousand four hundred and sixty-seven lines holding the interface and the rules, and a narrow contract between the game state and the model. Zero external dependencies at run time: only the Python standard library.

The state lives in the engine: cards, players, tokens, castle goals. The model is sent a JSON portrait of the state (player, hand, tokens, what has happened) and returns a decision. The engine checks whether that decision is legal and, if it is, applies it. Never the other way round: the model does not touch the state.

Every card effect is a hand-written function over the state. The model does not know how a card effect works: it only knows that an option with that name exists. Everything that can get it wrong sits outside the rules, and the rules do not depend on anybody.

The engine is deterministic when the AI is not involved. The dice come from a seeded generator, so a game is repeatable: if something breaks on round sixty-three, you replay round sixty-three and look at it. Without that, debugging a game by simulation is impossible.

Five instruction texts: one with the game rules and four different strategic profiles (team player, team dealer, conservative player, conservative dealer), because the same model has to play like a different style depending on who it represents.

game state (engine, seeded) → JSON portrait · 4 role prompts → POST /api/generate · format="json" · temperature 0.25 · no streaming → strict json.loads · if it fails, trim { … } → normalise against the vocabulary of the game → the engine validates legality · applies · next player → 2 failures in a row ⇒ AI off, local heuristic for the rest of the game
  • Engine

      Python 3.12Card, Scheme, Player classes24 card effectsSeed and reproducible dice
  • Interface

      tkinterttkBetting and simulation dialogsOn-screen event log
  • AI

      Ollama locallyForced JSON formatTemperature 0.25Configurable model and hostMinimum wait of 3 s
  • Verification

      unittestunittest.mockSimulated HTTP responses492 lines of tests
  • Dependencies

      Standard library onlyjson, random, urllib, tkinterNo network services

The output contract

Rejecting what the game cannot represent.

JSON is requested with the format forced, but that does not stop the model from adding prose around it, inventing a reward that does not exist or returning an action the rulebook does not know.

The reading happens in two stages. First a strict json.loads. If it fails, a second attempt trimming from the first opening brace to the last closing one, which is how you survive the model explaining itself before it answers.

Then comes the important part: a normaliser that checks every field against the real vocabulary of the engine. The reward has to be on the list of rewards; the betting action, one of four possible values. What is not on that list does not exist and turns into the neutral option. An invented name does not throw an exception: it degrades to none and the game carries on.

And a ceiling on everything that comes in from outside: the name of the beneficiary of a shot is cut at eighty characters. A free-text field that ends up in a query or in a window is an open door, and here the text is written by a model that nobody controls.

def normalize_ai_decision(self, decision): if not isinstance(decision, dict): decision = {} reward = str(decision.get("reward", "none")) valid = {"none", *(n for n, _c, _d in REWARDS)} if reward not in valid: reward = "none" # invento ⇒ opción neutra action = str(decision.get("poker_action", "check")).lower() if action not in {"check", "bet", "fold", "all_in"}: action = "check" return { "reward": reward, "take_shot": bool(decision.get("take_shot", False)), "shot_beneficiary": str( decision.get("shot_beneficiary", "self"))[:80], ... }

Failure tolerance

The worst case is not the AI not answering: it is answering late.

The obvious failure is caught. What kills the simulation is the model that answers after twenty seconds on every turn of a hundred-round game and never lets it finish.

There is a counter of consecutive failures that trips a permanent short circuit. On the second failure in a row the AI provider is disabled for the rest of that simulation and every remaining decision is taken by a local heuristic, with a warning in the log so that nobody later reads it as if the model had been playing.

The counter resets with a single valid response. An unstable network does not switch the AI off for good: only during that run, which is the only horizon over which you can decide.

Every call carries a configurable timeout with a floor of three seconds (the code writes max(3, timeout)) so that a badly filled field cannot produce an indefinite wait. There is never an unlimited wait, which is the only kind that does not recover.

The heuristic is not an empty patch: it reproduces a real conservative strategy, so a game finished by heuristic is still useful for validating rules. That was the objective from the beginning: measure the game, not measure the model.

except (OSError, urllib.error.URLError, json.JSONDecodeError, ValueError) as exc: self.ollama_consecutive_failures += 1 self.log_event(f"Ollama no respondio; uso heuristica. {exc}") if self.ollama_consecutive_failures >= 2: self.ollama_runtime_disabled = True self.log_event( "Ollama se desactiva para el resto de esta " "simulacion; se usara la heuristica sin mas esperas." ) return None

Verification

Testing logic that depends on a model that is not always there.

The tests run without Ollama installed. The HTTP call is replaced by a fake response and the contract is validated against three situations that are chosen on purpose.

A double of urllib.request.urlopen returns whatever I am interested in: valid JSON, unstructured text, a transport error. That checks the happy path and the two ways of breaking, and the engine is tested with no network and no model downloaded.

The test profiles play six named players plus a dealer, with a maximum of one hundred rounds and a summary at the end. If a rule makes it so that nobody can win, the profile does not finish and the maximum stops it: an infinite loop in a game engine is exactly that case.

And what is missing

With TEST_PROFILE_MAX_ROUNDS = 100 the cost of a regression is high and the current tests do not cover the twenty-four effects one by one against fixed states. It is real and bounded debt. The reason every effect is already a separate function is precisely that: they can be tested without rewriting anything of the engine.

  • 3,467lines of engine in a single file
  • 492lines of automated tests
  • 174functions and 4 classes
  • 5instruction texts: rules and four roles
  • 2failures in a row to switch the AI off for a game
  • 3restore-point scripts with written confirmation

What I would do differently

Separate the domain from the interface. Right now the engine and the tkinter window live in the same file of three thousand four hundred and sixty-seven lines, and that forces you to bring the application up in order to test one rule. The next thing is a table of tests per card effect (input state, expected state) that runs without opening a window. And storing the restore points in the versioned history instead of in three PowerShell scripts that require typing a word to restore: the manual confirmation avoids a disaster, but the history should not live outside Git.