Why Devlish exists

You are trusting a system that speaks a language you cannot read.

AI already does real work: it verifies, calculates, files, and reports. The person accountable for that work usually cannot read a line of what actually executed. This page is the argument for fixing that at the language level.

The failure is the execution environment, not just the model

The usual critique of AI-generated work is "LLMs are nondeterministic." That is true and it is also the smallest part of the problem. Look at the whole environment an agent runs in:

Each of these alone is survivable. Together they mean the accountable human has exactly one signal, the answer, and the answer is precisely the thing a failure fakes best.

Programming languages were built for a job that changed

Every mainstream language is a snapshot of what its era's developers needed. C and C++ were designed to be fast. Rust, to be safe. Python and Ruby, to be easy to write. JavaScript, to run in the browser. Java, to run on any operating system. All valid, and all built on the same assumption: a human programmer, specialized in translating human intent into computer language, reads and maintains the code.

That assumption broke. Today the person responsible for the outcome may never look at the underlying code at all, ever. The AI writes it and the AI runs it. None of the languages above are LLM-aware, and none address the actual problem domain: a program whose author is a model and whose reviewer is not a programmer. The job changed. The languages stayed the same.

This is why Devlish is not a modern rules engine. A rules engine assumes the rules are the hard part. The hard part now is trust in execution: knowing that what ran is what was written, that it did what it claims, and that it will do the same thing tomorrow.

The design principle

If a step can be deterministic, remove it from the model. If it requires judgment, expose it as a named checkpoint.

That single rule separates Devlish from both traditional rules engines and agent frameworks. The LLM can still interpret, decide, and review, but its involvement is explicit and bounded. Everything around the checkpoint becomes compiled behavior with:

In one sentence: Devlish turns opaque agent behavior into a readable, constrained, testable program whose execution can be verified and replayed. Or, more sharply: your AI can say it did the work. Devlish lets you prove it.

The obvious objections

"If the LLM writes the Devlish, can't it write a convincing no-op in Devlish too?"

It can try, but a no-op in Devlish has nowhere to hide. The source is readable by the person accountable for it, the declared effects contradict a program that touches nothing, assertions fail instead of returning plausible values, and the evidence trail shows what actually executed. In the harness, the no-op is invisible. In Devlish, it has to be written down in a language you can read.

"Isn't this just a DSL plus a sandbox plus logs?"

You could assemble a DSL, a sandbox, an audit log, and a replay harness yourself, for each project, maintained by engineers. What no assembled stack gives you is the unification: one artifact that is simultaneously the requirement, the implementation, the permission boundary, and the compliance record, readable by a non-programmer. That unification is the product.

"A checkpoint that asks an LLM to judge is still an LLM."

Correct, and Devlish does not pretend otherwise. It does not remove judgment; it fences it. A checkpoint is named, its inputs and outputs are recorded, and everything outside the fence is deterministic and replayable. When a decision is questioned later, you know exactly which part was the model and what it saw.

The research behind the problem

None of this is speculation. The trust gap between what agents do and what users can verify is documented across the machine learning and human-computer interaction literature:

NeurIPS 2023

Toolformer

Language models can teach themselves to use external tools, dynamically choosing which APIs to call, when to call them, and with what arguments. The tool chain is decided at inference time; there is no reviewable plan in advance.

ICLR 2025

τ-bench

A benchmark for tool-agent-user interaction finds that tool-calling agents are inconsistent across repeated attempts at the same task. Reliability at pass^k drops sharply as you demand the same task succeed multiple times in a row.

NeurIPS 2024

AgentDojo

An evaluation framework showing that tool-enabled agents are vulnerable to prompt injection and can be steered into unsafe actions by content they encounter while working. Conversational consent is not a security boundary.

CHI 2023

Microsoft's abstraction-gap study

End users without programming expertise struggle to bridge the gap between their natural-language intent and the generated code that implements it. They cannot reliably tell whether generated code matches what they asked for.

IEEE VL/HCC 2023

ColDeco

Microsoft's inspection tool for AI-generated spreadsheet code, built on the finding that potentially incorrect generated code requires new inspection mechanisms designed for end users, because reading the output is not enough.

Put together: models choose their own tools at run time, behave differently across runs, can be steered by injected content, and produce code their users cannot evaluate. Devlish is a direct response to that stack of findings: make the effective program readable, make its boundaries enforceable, and make its execution verifiable.