An agent needs tasks to learn from. Designing them by hand is expensive, and optimizing only for an easily checked final answer can miss much of what people value about the work.
Start with a description of useful work
Skill2Env turns public Agent Skills into executable terminal environments. Its pipeline decomposes a skill into workflows, diversifies tasks, creates an initial environment and reference solution, and verifies the result. Programmatic tests measure outcomes; skill-derived rubrics describe preferred behavior.
The project reports 7,971 tasks from roughly 3.4K skills across 13 domains. In its 300-update Qwen3.8-27B experiment, outcome-only RL moves Terminal-Bench 2.1 from 49.4 to 54.1 and S2EBench pass@1 from 33.4 to 37.7. Adding rubric reward produces different trade-offs: stronger judged skill alignment does not yield the best outcome benchmark scores in that comparison.
Outcome and process are different reward channels
Consider an illustrative report-writing task. A test can check whether the requested file exists and contains the required sections. A rubric can ask whether claims are supported and whether the reasoning addresses the intended audience. Passing the file checks does not logically imply meeting the rubric.
Conversely, a carefully narrated process can still produce the wrong artifact. The two signals answer different questions, so increasing one reward does not automatically increase the other. This is a general reward-design issue, especially when some aspects of quality are easier to verify than others.
Keep the experimental versions separate
The alphaXiv paper page also describes a 150-update FlashREINFORCE result. That is a different reported configuration from the 300-update comparison above. Their step counts and gains should not be combined into a single claim.
What to inspect in a new task corpus
A useful audit asks whether a task is solvable from its starting state, whether the reference solution earns the intended reward, and whether an empty action sequence accidentally passes. It should also inspect how faithfully tests reflect the actual requested work. These checks explain why environment generation needs evaluation of its own.
For the other side of this problem, read OSWorld-Pro, which asks how to observe progress during an agent’s execution.