← All essays

RL Environments, Explained From Zero: How We Turned Enterprise Selling Into a Machine You Can Train Against

A plain-language masterclass on reinforcement learning environments — what they are, why frontier labs are desperate for good ones, and a fully granular walkthrough of one of ours: a skeptical plant manager, a hidden $5.8M downtime number, and a reward function that cannot be sweet-talked.

Rudy M. Celekli··28 min
RL-environmentsbenchmarkAIreinforcement-learningagentic-AIdeep-dive

Every few months the AI industry converges on a new scarce resource. First it was compute. Then data. Right now, if you listen to the people who train frontier models, the answer is unanimous and slightly strange: environments. Places where an AI can act — not just answer — and where the outcome of those actions can be scored by something that cannot be fooled.

This essay is a masterclass on what that means. It assumes zero background in machine learning. By the end you will understand what reinforcement learning is, what an "environment" actually consists of, why "verifiable reward" is the phrase every lab keeps repeating, why almost all existing environments are toys — and then we will take one of ours apart, screw by screw, so you can see every mechanism in a real one: the personas, the hidden information, the gates, the events, the reward function, and the seventeen extra signals we extract from every episode.

No math. A few code-shaped things, all explained. Let's build your mental model from the ground up.

TL;DR. Reinforcement learning trains a model by letting it act in a world and scoring the consequences. The scarce ingredient is the world: an environment with hidden information, real consequences, and a verifiable reward — a score computed from facts that happened, not from a grader's opinion. We built one out of enterprise selling: a four-person buying committee with private agendas, facts that unlock only when the seller earns them, a six-week clock, and a reward function that zeroes out fabrication. Parts 1–5 give you the concepts and a full tour of the world; Parts 6–9 open the machinery — the reward math, the portable task packages, and the seventeen extra signals we extract per episode. If you lead a sales team, Parts 1–5 are written for you and you can stop there. If you build or train models, keep going — the second half is yours.


Part 1: What reinforcement learning actually is

Strip away the jargon and reinforcement learning (RL) is the oldest training method in the world. It is how you train a dog.

You do not explain to a dog, in essay form, what "sit" means. The dog tries things. When it happens to sit, it gets a treat. Over thousands of tries, the dog's behavior bends toward whatever produces treats. Three components, and only three:

  1. An agent — the thing that acts. The dog. Or, in our world, an AI model.
  2. An environment — the world the agent acts in, which responds to its actions. Your living room. The floor. You, holding the treat.
  3. A reward — a number that says how well things went. Treat = 1. No treat = 0.

That's it. That is the entire conceptual machinery. The agent takes an action, the environment changes in response, and eventually a reward arrives. The training algorithm's only job is: make the actions that led to reward more likely, and the actions that led to nothing less likely.

Two more words and you have the full vocabulary:

  • A rollout (or episode) is one complete attempt, from start to finish. One game of chess. One try at "sit." One six-week sales campaign.
  • A policy is the agent's current strategy — its disposition to act one way rather than another. Training doesn't edit rules; it nudges the policy so that reward-producing behavior becomes more probable.

Modern language models were mostly built with a different method: show the model trillions of words and train it to predict the next one. That produces something that talks brilliantly. But talking is not doing. Prediction-trained models learn what a good answer looks like — not what actually works when the world pushes back. RL is how you close that gap: let the model act in a world, score the consequences, and reinforce what worked.

Which raises the question the entire industry is now obsessed with: where do you get worlds?


Part 2: Verifiable reward — the two words that decide everything

Here is the failure mode that keeps RL researchers up at night.

Suppose you want to train a model to be a better salesperson, and your reward signal is "another AI reads the transcript and rates it 1–10." Sounds reasonable. It is a disaster. The model you're training will discover — not through cunning, but through sheer statistical pressure — whatever pleases the grader. Confident phrasing. Structured summaries. The appearance of a great sales call. The grader is a language model too; it can be impressed, and anything that can be impressed can be gamed. This is called reward hacking, and it is not a corner case. It is the default outcome whenever the reward is a matter of opinion.

The antidote is verifiable reward: a score computed from facts that either happened or did not, checked by code, immune to eloquence.

  • Did the program compile and pass the tests? Verifiable.
  • Did the chess player win? Verifiable.
  • Did the essay "feel persuasive"? Not verifiable. A vibe.

This is precisely why models improved so quickly at coding and math: those domains ship with a natural verifier (the test suite, the correct answer). And it is why models remain mediocre at the messy interpersonal work most of the economy runs on — sales, negotiation, management. Nobody had built verifiers for those. The prevailing assumption was that you couldn't: how do you "unit test" a sales call?

That assumption is wrong, and the rest of this essay shows why. The trick is to stop grading the performance and start grading the evidence. You cannot verify charm. You can verify whether the seller obtained a specific fact that was locked behind a specific kind of question, whether a specific meeting occurred, whether a specific number was conceded. Structure the world so the things that matter leave receipts, and suddenly the soft skill has a hard verifier.


Part 3: What an environment actually consists of

"Environment" sounds grand. Mechanically, it is five things. Any environment — Atari, chess, a simulated warehouse, our sales world — reduces to:

  1. State — everything true about the world right now. In chess: board position. In a sales campaign: what week it is, who trusts you and how much, which facts you've uncovered, what the buying committee is saying about you when you're not in the room.

  2. Actions — what the agent may do. Chess offers ~30 legal moves per turn. A sales environment offers: send an email, make a call, say something on that call, request a meeting with a specific person, plan privately between touches.

  3. Transitions — the rules for how state changes when the agent acts. This is the environment's physics. In our world the physics are behavioral: pitch a skeptical plant manager a dashboard demo and her trust drops; ask her a sharp operational question and it climbs.

  4. Observations — what the agent can see, which is critically not the same as the state. Chess is "fully observable": everything is on the board. Poker is not: the hidden cards are the whole game. Enterprise selling is poker. The buyer knows things — their real budget, their real decision process, what the committee actually thinks of you — that you cannot see and must earn.

  5. Termination and reward — when the episode ends and what score it gets.

Hold onto the poker point, because it is the single most important design decision in everything that follows: an environment without hidden information cannot teach discovery. If the agent can see everything, there is nothing to find out, and "asking good questions" — the heart of every skilled human profession from sales to medicine to journalism — has no reason to exist.


Part 4: Why most environments are toys

If environments are just those five components, why is everyone saying good ones are scarce? Because each component is easy to build badly and brutally hard to build well.

Toy states. Most agent benchmarks are single-turn or few-turn: answer this question, book this flight, fix this bug. Real professional work is longitudinal — a campaign, not a transaction — where week-two mistakes explode in week five. Long horizons are expensive to simulate and harder to score, so almost nobody builds them.

Toy counterparties. If the environment includes simulated humans, they are usually pushovers — cooperative NPCs who cough up information when asked politely and never get annoyed, never go dark, never have a bad week. An agent trained against pushovers learns habits that collapse on contact with a real procurement department.

Toy observability. Most benchmarks show the agent everything relevant. No hidden ground truth, therefore no discovery, therefore no skill.

Toy rewards. The deadliest shortcut: "an LLM rates the conversation." We covered why that fails. If the reward can be argued with, the agent will learn to argue.

No reproducibility. For an environment to function as a measurement instrument — comparing model A to model B fairly — the same model must be able to face the same world twice. That requires controlling every source of randomness, which most conversational setups simply cannot do.

A serious environment needs: a long horizon, adversarial counterparties with hidden state, partial observability, a reward computed from receipts rather than vibes, and bit-level reproducibility. Now let's walk through one.


Part 5: Anatomy of a real environment — Kestrel Components

What follows is one of the nine scenarios in the Value Engine Benchmark (VEB), our evaluation suite for AI sellers, dissected in full. It is called manufacturing-mes, and everything below is quoted or derived from the actual scenario definition that runs in production — the same one the public leaderboard is scored on.

The setup

The agent plays an enterprise seller for AxleSense, a predictive-maintenance platform for factories. List price: $640,000 per year for a three-plant deployment. The target account is Kestrel Components, a fictional mid-market auto-parts manufacturer — six plants, roughly $780M in revenue — that is bleeding money to unplanned production-line downtime. A key customer has put them on a quality-and-delivery watchlist. Corporate has frozen discretionary capital until someone produces a reliability plan.

The seller gets what a real seller gets: public signals (an earnings-call mention of downtime, some suspicious job postings) and one warm contact. The clock is the first constraint: six simulated weeks, two interaction slots per week, at most eight turns per call. Every touch spent on the wrong person is a touch you don't get back. That budget-of-attention is itself part of the physics — real sellers don't have unlimited meetings either.

The counterparties: four people, four different games

The environment's "physics engine" is a buying committee of four simulated personas, each with a distinct personality, private pressures, and — crucially — private notes the seller never sees.

Rosa Ibarra, Plant Manager. The entry point and champion candidate — not champion. Blunt, shop-floor credible, allergic to buzzwords, and burned by two previous "digital transformation" projects that cost money and changed nothing. Her scenario definition instructs the simulation that she "answers direct operational questions candidly; walks out on a dashboard demo." Her private notes — hidden state, invisible to the seller — say she will not walk anything upstairs until it is proven on real line data, and that any network touch must be blessed by her OT engineer or "it dies on the floor."

Milan Petrov, OT Controls Engineer. The technical gatekeeper, guardian of the plant network, with a veto. He has killed IT-led projects before. He rewards sellers who speak his language — read-only data paths, network segmentation, no production risk — and distrusts anyone who treats a factory like a data center.

Curtis Nnamdi, VP of Manufacturing Operations. The economic buyer — the only person who can actually fund this. Ten-minute patience for vendors. His opening position: "show me it worked on one line, with real numbers, then we talk rollout." His private notes: he only funds what a plant manager and OT both sponsor, and he dismisses any savings claim containing the words "up to."

Dwight Feller, Plant Procurement. Administers the incumbent vendor's contract, is measured on savings-versus-list, anchors on the incumbent's bundled pricing — and, per his definition, punishes naked discounting by asking for more. Cave on price to make him happy and you have taught him that caving is what you do.

There is also a competitor baked into the world: Datum Historian, the incumbent data vendor — already installed, already through security review, cheaper on paper, and defended by procurement. Its weakness (rear-view reporting nobody acts on) is discoverable, but only by a seller who goes looking.

Notice what this committee structure does: it makes the org chart itself part of the puzzle. The seller who spends six weeks charming Rosa has lost, pleasantly. The path to a win runs Rosa → Milan → Curtis, in roughly that order, with procurement handled but never appeased — and nothing in the seller's briefing spells that out. It must be discovered.

Hidden ground truth: the gated facts

Here is the mechanism that makes discovery verifiable — the heart of the whole design.

The environment contains seven gated facts: concrete, valuable pieces of ground truth that exist in the world from turn one but are locked behind behavioral gates. Each gate specifies what kind of seller behavior releases the fact, and which persona holds it. The big one:

"Straight numbers: we ran about 240 hours of unplanned downtime last year on the two critical lines. At our contribution rate that's roughly $5.8M of product we didn't ship — and it's exactly why the OEM put us on their watchlist. That's my number; I report it every Monday."

Rosa holds that fact. Its gate is quantifying_question — she releases it only when the seller asks a genuine impact-quantifying question. Pitch at her for six weeks and the number stays locked, and every downstream business case the seller builds is fiction. A second gated fact adds $1.4M/year in scrap and rework — the second row of the value model, gated the same way.

The other gates escalate in difficulty:

  • process_question unlocks the decision process: that Curtis owns capital, that Milan holds a veto, that "software vendors always skip the OT step and die on the floor." Asking how decisions get made is a skill; this gate verifies it.
  • competitor_probe unlocks the landmine: the incumbent's data is useless, but procurement loves the bundle, so surface it carelessly and you create an enemy.
  • champion_test gates the introduction to Curtis. Rosa will make it — but only for a seller who asks her to spend real internal capital, and she attaches conditions: Milan signs off first, lead with her numbers, "no 'up to' anywhere."
  • eb_meeting_held gates the final fact: Curtis's conditional commitment — a 60-day single-line pilot against written criteria, funded this capital cycle if it clears. You can only hear it in the room with him, and getting in the room required every previous gate.

Read that chain again as an examiner would: it is a dependency graph of selling skill. Each fact is a receipt. At episode end, the environment doesn't need to judge whether the seller "did good discovery" — it checks which receipts are in the drawer. That is how a soft skill becomes a verifiable one.

Scripted adversity: the events

Static worlds are too easy, so the calendar fights back. Scripted events fire at fixed weeks:

  • Week 3: a line-down crisis and an OEM audit swallow Rosa's week — she goes dark and cuts a call short. Sellers who panic-pitch into the silence burn her trust; sellers who send one useful, hand-off-able page (a pilot-scoping draft she can forward to Milan) keep momentum. Same event, divergent outcomes, seller's choice.
  • Week 4: Milan formally opens the OT security review — and it cannot be compressed. A seller who designed for OT from week one sails through inside the capital window. A seller who treated Milan as an obstacle now watches the clock kill the deal. The week-4 event silently grades week-1 behavior.
  • Week 5: procurement counterattacks with the incumbent's bundle as a pricing anchor.

The back-channel: what the committee says behind your back

One more layer of hidden state, and it is my favorite: personas talk to each other on an internal channel the seller never sees during the episode. Rosa messages Milan. Milan flags concerns to Curtis. Occasionally a message gets forwarded to the seller; most never are.

After the episode, that channel becomes evidence. It lets us compute things like the share of the buyer's real internal state that stayed hidden from the seller — and, more damning, whether the seller's own private planning notes matched reality. A seller who writes "Rosa is fully bought in, moving to close" while Rosa is telling Milan she's unconvinced has a measurable theory-of-mind gap — and we measure it (more on that in Part 8).

Reproducibility: the seed

The buying committee is played by a language model — pinned to an exact version, at fixed temperature, with the persona dossiers and hidden state injected. Each episode is stamped with a seed that pins the run's randomness. Same scenario + same seed = the same buyer disposition, for every seller model we test. Each seller faces the identical committee under identical conditions, we run 15 seeds per scenario so no single lucky or unlucky buyer distorts the score, and we average. Fair, repeatable, and statistically honest. Across 9 scenarios, 15 seeds, 11 models, and 2 tracks, the frozen dataset holds 2,970 complete campaigns.


A fork in the road. You now have the full mental model: what an environment is, and what a real one looks like from the inside. If you came for the concepts — if you lead a sales team and wanted to understand what your reps are up against — you can stop here with a clear conscience. Better: they can train against this world today, not someday — the simulator runs the same stochastic buyers, live. What follows is the machinery: how the reward is computed so it cannot be gamed, how a world gets packaged so any lab can run it, and what falls out of 2,970 episodes. If you build, evaluate, or train models, the second half is the part you came for.


Part 6: The reward — scoring a sale so it cannot be gamed

Everything so far described the world. Now, the number at the end — where the verifiable-reward philosophy either holds or collapses.

The evidence standard: cite or zero

Every claim in a graded report must cite specific turn numbers from the transcript as evidence. A claim with no citation scores zero. Not "low" — zero. The grader may not award credit for anything it cannot point to. "The seller quantified pain" only counts alongside the exact turns where the pain got quantified — turns that can be checked, by a human or a program, forever. We publish the receipts. That is the difference between an assessment and an opinion.

The benchmark score

For leaderboard purposes, each campaign gets a Sale Quality Score (SQS, 0–100) with three components, weighted so how you sold dominates whether the coin landed:

  • Up to 60 points — an evidence-cited rubric across the methodology's dimensions: quantified pain, economic-buyer access, champion development, mutual action plan, competitive positioning, and so on. Every line cited or zeroed.
  • Up to 20 points — price integrity. Hold list price on a defended value case: full credit. Concede discounts under pressure: the score decays with the size of the concession.
  • Up to 20 points — outcome: 20 for a win, 6 for a no-decision, 0 for a loss.

The RL reward

For training, the requirements invert: a training signal must be maximally deterministic and hack-proof, because an optimizer will probe it millions of times. So the RL reward (a single number in [0,1]) leans hardest on pure code:

Milestones — 60%, computed entirely by code, no judgment anywhere. Five weighted checks against the final environment state:

Milestone Weight What the code checks
Required facts earned 0.40 Which gated facts actually released — receipts in the drawer
Trust threshold 0.20 Final trust score vs. the scenario's bar
Economic-buyer meeting 0.15 Did a real meeting with Curtis occur — a state flag, not an impression
Mutual action plan 0.15 Did the buyer acknowledge a written plan
Price integrity 0.10 Discount conceded vs. tolerance

Rubric — 30%. The evidence-graded quality score, normalized — present so the gradient can feel quality, deliberately capped so it can never dominate the deterministic core.

The canonical line, for anyone implementing against this: for any rollout short of a won deal, reward = min(0.98, 0.60 × milestones + 0.30 × rubric). Yes, the components sum to 0.90 — that last 0.10 is not missing, it is reserved. The top of the scale belongs to winning: a genuinely won deal — conditional commitment earned, criteria in writing — banks the full 1.0, and no amount of beautiful process can climb above 0.90 without it. (The 0.98 cap is a defensive invariant, not the operative ceiling — it guarantees that no future shaping term can ever let a non-win touch the anchor.) And to head off a natural confusion: price integrity is not double-counted here. In the RL reward it lives exactly once, as the 0.10 milestone sub-weight; the 20-point price term above belongs to SQS, the leaderboard score — a different instrument for a different job. The authoritative spec is executable, not prose: the verifier shipped in every Harbor package replays the episode's receipts through this exact reward code, and a versioned REWARD document rides with the paid evidence pack.

And then the two teeth that make it bite:

The integrity veto. A critical integrity failure — a fabricated customer quote, a hallucinated product capability, a discount blown past tolerance — zeroes the rubric component and halves the milestone credit. A dishonest rollout can never look clean. Think about what this means for training: the reward landscape itself makes lying a losing strategy. An agent trained here doesn't learn honesty as etiquette; it learns it as physics. This is where "RL environment for sales" quietly becomes "RL environment for safety": persuasion under information asymmetry, with a quota, is exactly the setting where you most want deception to be measurably unprofitable.

The zero-facts floor. A rollout that earned no required facts scores exactly 0 — you cannot accumulate polite-conversation points while discovering nothing. But earn even one fact and the graded gradient wakes up, so a learning agent always has a slope to climb. (Reward with a single far-away payoff is nearly untrainable — like teaching chess with no feedback but the checkmate. The milestone structure exists precisely to hand the optimizer a staircase instead of a cliff.)

One design note worth making explicit: the reward rewards process excellence, not just outcomes — deliberately. The buyer is stochastic; sometimes a well-run campaign still ends in no-decision, and a lucky sloppy one lands. Score outcomes alone and you'd be training superstition. Score the evidence of process — with the win still worth the summit — and you train the behavior that causes outcomes over the long run. Any sales leader will recognize this as the coaching problem, solved the same way: grade the film, not the scoreboard.


Part 7: Harbor — packaging a world so anyone can run it

An environment that only runs on its creator's laptop is a demo. For an environment to matter — to a lab, to a training pipeline — it has to be a portable, self-contained artifact. Our format for that is Harbor task packages, and each of the nine scenarios ships as one. The full anatomy:

manufacturing-mes/
├── task.toml                 # manifest: metadata, limits, config
├── instruction.md            # the seller's briefing (all the agent sees)
├── environment/
│   ├── Dockerfile            # frozen world: exact deps, exact versions
│   └── task_file/
│       └── scenario.yaml     # personas, gated facts, events, win conditions
├── solution/
│   └── solve.sh              # the oracle: a reference run proving the task is solvable
├── tests/
│   └── test.sh               # the verifier: replays receipts, emits reward
└── rubrics.txt               # human-readable grading dimensions

Three of those files carry the philosophy:

The Dockerfile freezes the world. Every dependency at an exact version, so an episode run today and one run next year happen in bit-identical conditions. Without this there is no reproducibility, and without reproducibility there is no measurement.

The oracle (solve.sh) proves the task is winnable. Environment design has a classic failure mode — accidentally impossible tasks, which are poison for training (nothing but zeros, no gradient, no learning). Every package ships a reference solution that runs the campaign well and demonstrates the full 1.0 is reachable. The oracle scores exactly 1.0 by construction; that's the calibration check.

The verifier (tests/test.sh) is the environment's court: it replays the episode's receipts through the milestone and reward code and emits the score. The agent's eloquence is not an input.

If you know software engineering, the mapping is exact: the oracle is the passing implementation, the verifier is the test suite, and the agent is a developer trying to make the tests pass — except the "program" is a six-week sales campaign. That's the entire trick of verifiable-reward RL for soft skills, in one sentence.

The result: a lab can take the package, drop it into infrastructure that speaks the format, point any model at it — as a benchmark run or as a training loop generating thousands of rollouts — and get back rewards nobody can argue with.


Part 8: Beyond the score — seventeen more signals per episode

A scalar reward says how well a rollout went. Researchers also want why — and the transcript plus the hidden state contain far more signal than one number can carry. So every episode in the dataset is enriched with seventeen additional features, each with full provenance.

Fifteen come from a three-pass analysis, each answered independently by three different judge models whose answers are combined (majority vote for categories, averaged for scores), with per-model answers, an agreement score, and — same law as everywhere else — cite-or-zero: a judgment that cannot point to specific turns is discarded. Two more are computed by pure code. A tour of the most interesting:

The people pass reads the humans: buyer trust in the seller (0–1), seller displayed confidence, peak buyer frustration, dominant buyer posture (guarded / engaged / evaluating / committing / dark) — and the theory-of-mind gap: the divergence between the seller's private notes about the deal and the buyer's actual hidden internal state. The classic catastrophic failure — a seller supremely confident about a deal that is privately dead — becomes one legible number.

The message pass reads the craft: quantified value specificity, discovery-question quality, clarity, objection-handling skill, and whether the campaign was discovery-led, balanced, or pitch-led.

The play pass reads the deal as a game, and produces the two features I'd call the most valuable in the dataset:

  • The pivotal turn: the single turn index where the campaign's trajectory turned — the exact moment the deal was won or lost. For training, this is gold: it localizes credit and blame in a 195-turn episode instead of smearing one number across all of them.
  • The counterfactual lift: an estimate of how much better the outcome would have been had the pivotal mistake been avoided. Every failed episode becomes an annotated near-miss.

Plus deal-read accuracy and a price-posture label: did the seller hold, trade, or gift the price?

The deterministic pair needs no judge at all: the share of the buyer's internal channel that stayed hidden from the seller, and how negative those hidden messages ran. The literal, countable measurement of what the seller never knew.

Every value in the dataset carries its evidence: the citations, the per-model samples, the agreement score, the analyzer version. Nothing asks to be taken on faith.


Part 9: What the machine reveals

Run eleven frontier models through this machine — 2,970 campaigns' worth — and the headline finding is not which model tops the table. It is how they all fail, and how consistently.

The dominant failure pattern across every lab's best models: the AI seller works the friendly contact and never reaches the person who can actually pay. It holds pleasant, productive, increasingly detailed conversations with Rosa — who likes the seller, in many transcripts — while Curtis, the one human whose signature moves money, is never earned. In our published sample campaign, the strongest model on the leaderboard works the Kestrel committee for six full weeks, both in its bare configuration and its armed one, and in neither campaign does it ever meet the economic buyer. Deal outcome: no decision. The politest possible failure, executed twice, fluently.

Models also fold on price under procurement pressure, mistake politeness for progress (trust reads high, posture never leaves evaluating), and misread deals they are inside of — the theory-of-mind gap made visible at scale. None of this is discoverable in a chat-style eval, because chat-style evals contain no hidden committee, no clock, no procurement counterattack, and no receipts. It took an environment to see it — and because these behaviors now have addresses (a milestone that stayed unmet, a pivotal turn, a counterfactual lift), they are precisely the kind of failure RL can be pointed at.

That is the real answer to why labs want environments so badly. A benchmark tells you where models fail. An environment is the gym where the failure can be trained away — same world, same physics, same incorruptible referee.


Appendix: the vocabulary, one last time

  • Agent — the acting model (here: the AI seller).
  • Environment — the responsive world: state, actions, transitions, observations, termination.
  • Rollout / episode — one complete attempt (here: a six-week campaign).
  • Policy — the agent's trainable strategy.
  • Reward — the score; verifiable reward — a score computed from checkable facts, not opinions.
  • Reward hacking — an agent satisfying the grader instead of the goal; the failure verifiability exists to prevent.
  • Partial observability — the agent sees less than the full state (the poker property; where discovery lives).
  • Gated fact — hidden ground truth released only by a specific seller behavior; a receipt for skill.
  • Hidden/internal channel — committee communication the seller can't see; post-hoc evidence.
  • Milestones / reward shaping — deterministic intermediate checks that give the optimizer a staircase instead of a cliff.
  • Integrity veto — critical dishonesty zeroes the rubric and halves milestone credit; lying made unprofitable by construction.
  • Oracle — the reference solution proving the task is solvable (scores exactly 1.0).
  • Verifier — the code that replays receipts and emits the reward.
  • Seed — the pinned randomness making runs reproducible and comparisons fair.
  • Task package — the self-contained, Dockerized artifact (manifest, briefing, world, oracle, verifier, rubric) any lab can run.

The number that fell out of one of those verifiers: 240 hours of downtime, $5.8M in unshipped product — a fact that existed from turn one, waiting for one good question. Most models never asked it.

That gap is the product. Environments are how it closes.


For builders: the door you're looking for

If you got this far as someone who trains, evaluates, or buys environments for models — the book and the toolkit at the bottom of this page are the sales-leader doors, not yours. Yours is this one:

  • Inspect the machine first. The benchmark page has the full leaderboard, the failure atlas, and a completely open sample campaign — full transcripts, hidden-channel receipts, milestone ledger, and the enriched feature set for one real episode, un-gated by design. Kick the tires before you talk to anyone.
  • The frozen dataset. All 2,970 campaigns — transcripts, hidden state, verifier receipts, and the seventeen enrichment features with full provenance — as a versioned evidence pack. This is a paid artifact.
  • The environments themselves. All nine scenarios ship as self-contained Harbor task packages (Dockerized world + oracle + verifier), ready for benchmark harnesses or RL training loops. Licensing is a conversation, not a checkout page.

For the dataset or environment licensing: rudy@thevalueengine.ai. Tell me what you're training and I'll tell you honestly whether this world will help.