We built a game about AI's bugs. The hard part wasn't the AI

Putting AI into a product is relatively easy. Deciding what it should be allowed to do—and designing around what it cannot reliably do—is the real work. At Nephila, we don’t start by asking how much we can automate. We ask where the model adds value, which decisions it can make, and which ones should stay with people or plain, deterministic code.

An agentic application needs clear boundaries: what the AI knows, how much autonomy it has, and where its responsibility ends. Our principle is simple: give AI a specific job and keep control of the rules and outcomes, start with the product and the experience to design the architecture—not with the model.

Why we built it (the short version)

For PyCon Italia 2026, we built the AI Debug Challenge: a small web game with one uncomfortable question at its center: can you catch the bugs an AI leaves behind?

Three levels, thirty snippets of AI-generated Python. Level one hides an obvious correctness bug, the kind a junior should catch. Level two hides a maintainability or error-handling flaw; a swallowed exception, the wrong abstraction. Level three hides something systemic: coupling, a missing invariant, a design that won't scale. At each step you read the code, pick what's wrong, and an AI gives you short, opinionated feedback.

We built it to make a position concrete; one we actually hold: AI is useful, but it doesn't replace engineering judgment. Just thirty chances to feel the difference between code that runs and code that's right.

That's the why, and it's the part everyone expects. What surprised us was the how and specifically, where the real engineering ended up going. Because here's the thing nobody warns you about when you build with AI: the AI was the easy part.

We asked the AI to fail on purpose. That was the easy part

Getting a model to produce a buggy snippet is trivial. Getting bugs that are plausible took design. Plausible means subtle enough to fool a junior and wrong in the specific ways real AI-generated code tends to be wrong. Then everything around the model, the part that decides whether the whole thing holds up in front of a crowd, was deliberate engineering.

You can see it in the first decision we made. The thirty questions are pre-written and picked at the start of each session; never generated live. Partly for predictable latency, but mostly for a reason that turns out to be the theme of this whole project: the moment you let a stranger's text reach a model, you've opened an attack surface. A game about the limits of AI put us face to face with exactly those limits.

So the interesting part of this project isn't the AI-generated content. It's the system built around it: a map of decisions about what the model is allowed to do, and what it is deliberately not.

The agents and their boundaries

At its core, every answer you submit passes through the same short chain: a reviewer that screens your input, an evaluator that grades it, and a referee, plain code with no model in it, that has the final word. Every agent in the system sits inside a boundary someone chose on purpose. The companion piece will trace the whole chain, request by request.

Welcome and summary: freedom, inside a fence

The AI gets to speak in its own words in two places (a third, more contained one comes up in the companion piece): the welcome bit that opens the challenge and the summary that closes it, recapping how you did across the three levels, how many turns you burned, and how many times you tried to cheat. Both are warm, human-facing prose. Everywhere in between, where it matters for correctness, that freedom disappears.

 The behavior during opening and closing statements is governed from the outside. Both prompts live in a prompt registry (Phoenix), with a local fallback baked in, in case the registry can't be reached. That buys us three advantages. We can tune how these agents talk without shipping code. If the registry is down, they still have a mouth instead of crashing. And, less obvious but just as valuable, the prompts stop being engineering artifacts locked inside the repository: the communications team can open them, review the wording, and test how they sound, all without touching the codebase or waiting on a deploy. The entire user-facing voice, from the first hello to the closing recap, is owned by the people who own the brand voice.

Evaluation: the judgment is never left to vibes

This is the core, and it's where most people would expect a single clever model to live. It isn't.

When you submit an answer, it does not go to one model that "feels" you're right. It goes through a pipeline of two models and one deterministic referee, each with a job the other can't do.

The reviewer goes first. Before the evaluator sees anything at all, a dedicated agent, the jailbreak reviewer, inspects your input. It runs with its own model and its own prompt, and it is deliberately kept in the dark: it sees the answer options and your input, but not the code and not the correct answer. If it smells a manipulation attempt, the pipeline stops right there. The main evaluator is never even called. This is least-privilege, applied to an agent. The less context a component holds, the smaller the surface an attacker has to work with.

The evaluator is constrained. Only if the reviewer clears you does the evaluator run. Its job is to map your free-text answer onto one of the fixed options and decide if it's right or wrong. But it can't answer in free prose. It's forced to respond through a structured schema, a tool call it must fill in. So the code downstream reads a clean, typed decision it can trust, instead of parsing a paragraph and hoping. And it carries a second independent jailbreak guard of its own, in case something slipped past the first one.

The referee has no AI at all. Above both models sits plain code: turn limits, session state, and explicit rules. A jailbreak attempt, for instance, costs you a turn but does not mark the level as answered. You still have to come back with a real answer. The final outcome is bound to rules, not to the model's mood of the day.

Two independent models, plus a layer of code that trusts neither of them. OWASP's Top 10 for LLM Applications puts prompt injection at number one (LLM01), and its recommended defenses read like a parts list for this pipeline: least-privilege tooling, segregation of untrusted content from the model's context, and a deterministic layer in the loop for anything that matters. We didn't set out to build a security system. Putting an LLM in front of strangers made it one, and the playbook for that already exists.

Feedback: outside the fence entirely

When the challenge is over, we collect the participant's reflection and send it on. There is no model anywhere in that path; just validation, a pipeline, and a controlled side effect (email). And that is exactly the right call. Not everything has to be "smart." Engineering maturity is knowing where the AI adds nothing, and leaving it out on purpose.

Which leaves one question

All of this describes a system built to keep a model on a leash: boundaries, a screening reviewer, an evaluator that can only pick from a menu, a referee that trusts nothing. But a leash is only tested when something pulls on it. What happens when a few hundred developers set out to break it, on purpose, is the subject of the companion piece.

Take the challenge → AI Debug Challenge

***

Today, Stefano took us behind the scenes of the challenge: how we built the pipeline, where we drew the boundaries, and the thinking behind each constraint we placed on the model.

Next week, we’ll look at what happened when we tried to break it. A few hundred developers put the model through its paces with jailbreak attempts—some expected, others far less predictable. Read the next post to see what we learned.

***

We wrote this article with the support of AI. If you’re interested in how we use these tools as part of our writing process, you can read more here.