Task overview
Summary JSON ↗| Environment | Coverage | Native success | Input tokens | Output tokens | Usage coverage |
|---|
Current recorded result per task · Seed 0 · Native success criteria. Each task shows its latest result and the instruction used for that recording. Rates include valid completed results only; pending tasks are excluded. No pooled score across environments. Input includes cached tokens; output includes reasoning tokens.
No tasks match this search.
The evaluation protocol
Modified instructions are labeled, with changes highlighted. Each recording retains the instruction it used; native success predicates are unchanged.
One independent episode, seed 0. GPT-6 Astra with high reasoning effort. Current RoboCasa, LIBERO and RoboDojo reruns use Kinex v0.10.3; each result records its exact runtime.
RoboCasa and LIBERO: 6,000 steps. RoboTwin and RoboDojo: 7,500 frames. BEHAVIOR: 50,000 frames. SIMPLE G1: 10,000 or 15,000 control frames at 50 Hz; see each task protocol. Stepped simulation, with an eight-hour wall-clock limit.