Turning Database Grading into RL Rewards: Post-Training Agents with ThinkingBox

Introduction

The recent ThinkingBox‑Bench study revealed a gap between headline accuracy (pass@1) and true dependability when LLM agents manipulate production databases. By turning the executable terminal‑state checks into reinforcement‑learning rewards and carefully curating a training pool based on offline policy‑relative difficulty, researchers fine‑tuned the Qwen3.8‑27B model. The result: a three‑fold increase in tasks that succeed on every of 20 runs, with a notable reduction in cost per dependable task.

---

From Grading to Rewarding

ThinkingBox‑Bench grades agents by comparing the final database state against the expected outcome. Each of its 507 tasks provides a real backend, a toolbox of MCP utilities, and a simulated user. The key insight is that the binary pass/fail verdict—normally a post‑hoc grader—can serve directly as a reward signal. By feeding this signal into a GRPO (Generalized Reward‑Policy Optimization) loop, the model learns to prefer trajectories that leave the database in the correct state.

Offline Policy‑Relative Difficulty

Not every task is useful for training. Some are already solved by the base model; others are hopelessly hard, offering no gradient signal. To identify the sweet spot, the base Qwen3.8‑27B policy was run once on each task, estimating a success probability. Tasks with intermediate difficulty (neither near‑0 nor near‑1) formed a fixed training pool that remained unchanged throughout the 50‑step RL run. This offline selection ensures the model focuses on learning from informative failures and successes.

---

What Changed After Fine‑Tuning?

| Metric | Base Model | RL‑FT (50 steps) | Δ | |---|---|---|---| | Observed 20/20 (tasks succeeded on all 20 attempts) | 38 (7.5 %) | 107 (21.1 %) | +69 tasks | | pass@1 (average success) | 51.70 % | 60.78 % | +9.08 pts | | pass@20 (at‑least‑once) | 89.35 % | 87.97 % | –1.38 pts |

The fine‑tuned checkpoint dramatically improved repeatability (Observed 20/20) while slightly reducing the breadth of at‑least‑once coverage (pass@20). In a concrete hotel‑booking scenario, the base model omitted a required manager approval in 15 of 20 runs; the RL‑FT model adhered to policy in all 20.

Cost Perspective

Using the same inference budget (507 tasks × 20 runs), the campaign cost fell from $925.80 to $855.58 (‑7.6 %). More importantly, the cost per dependable task dropped from $24.36 to $8.00, a 67 % reduction, putting the fine‑tuned Qwen3.8‑27B in the same cost‑effectiveness range as larger commercial models (GPT‑5.4, Claude Opus 5.5).

---

Practical Takeaways

1. Reward Design Matters – Directly using executable database checks as rewards aligns training objectives with real‑world reliability.
2. Curate Training Tasks – Offline policy‑relative difficulty isolates tasks that are informative for the current model, avoiding wasteful extremes.
3. Expect Trade‑offs – Sharpening repeatability can shave a few points off at‑least‑once coverage; evaluate which metric aligns with your production SLAs.
4. Cost Benefits – Improved dependability does not require proportionally higher inference spend; careful RL fine‑tuning can lower the cost per reliable outcome.

---

Closing Thoughts

Turning a benchmark grader into a reward function bridges the gap between evaluation and training. The ThinkingBox experiment shows that even a modest‑size 27 B model can achieve a reliability level previously reserved for much larger systems, provided the training data are selected with an eye on policy‑relative difficulty. For teams deploying LLM agents that write to production databases, adopting this loop offers a concrete path to higher consistency without exploding infrastructure costs.