Algorithm ideas are easier to test when the path from an agent’s action to a parameter update is visible. At scale, that path also needs parallel execution, weight synchronization, and somewhere to put large trajectories.
A small surface over a distributed loop
Molt combines PyTorch- and Hugging Face-native model loading with OpenAI- and Anthropic-compatible agent interfaces. Fully asynchronous execution overlaps rollouts with optimization. Distributed experience storage keeps long, multimodal payloads from accumulating in one controller.
In the system design, workers place trajectory payloads in Ray object stores and pass references through orchestration. Trainer ranks retrieve their assigned experiences. Token IDs, behavior probabilities, masks, and multimodal inputs travel as training data, rather than only as a human-readable transcript.
Why references help
Imagine a batch of desktop agents, each producing many screenshots. Passing every image through a single coordinating process creates a concentrated memory requirement, even if inference itself is spread across machines.
A reference-based handoff separates the small scheduling message from the large payload. This can move memory pressure away from the coordinator, but it does not make payload transfer free. Placement, retrieval, and the learner’s consumption rate still determine whether the pipeline stays busy.
Read scale claims at the right level
The evaluation runs the complete asynchronous path—rollout, weight refit, and an update—on a one-trillion-parameter policy. Sustained learning is separately demonstrated with Qwen3-30B-A3B over 1,464 updates. Its mean reported evaluation accuracy rises from 46.38% at update 50 to 66.36% at update 1,450. These are different experiments, not evidence of a long 1T learning run.
What makes infrastructure useful for research
Framework size alone cannot tell us how easy a new idea will be to implement. A more useful question is where a change belongs: model code, advantage estimation, rollout capture, or experience storage. Clear boundaries make that question easier to answer and failures easier to isolate.
Polar focuses on the harness-facing rollout path; FlashREINFORCE examines an algorithm that consumes asynchronous experience.