RoboMME

What the protocol tests

Its task suites separate counting, object permanence, reference and imitation demands. These distinctions explain failures that a single headline average would hide.

Reading results

The paper studies memory-augmented variants built on π0.5. Those modified policies are not the unmodified public base checkpoint, and their results must preserve memory design and training configuration.

Methodology and reuse limits

The default released environment focuses on imitation learning; alternative action interfaces can change the evaluated problem.

Evaluation protocol

Exact version / scope

RoboMME; arXiv:2603.04639v3

Primary specReviewed 2026-09-28

Sources 1

RoboMME/robomme_benchmark official repository

RoboMME · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Environment

simulation

Primary specReviewed 2026-09-28

Sources 1

RoboMME/robomme_benchmark official repository

RoboMME · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Metric definition

Task success rate by memory category

Primary specReviewed 2026-09-28

Sources 1

RoboMME original paper v3

Yinpei Dai and coauthors · Publication date not disclosed · Source accessed 2026-09-28

Task scope

Not publicly disclosed

Embodiment

ManiSkill manipulation environments

Primary specReviewed 2026-09-28

Sources 1

RoboMME/robomme_benchmark official repository

RoboMME · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Access & reuse

Access model

Public research documentation and evaluation resources

Primary specReviewed 2026-09-28

Sources 1

RoboMME/robomme_benchmark official repository

RoboMME · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

License

Not publicly disclosed

Additional disclosed fields

Category

Robot evaluation benchmark

Primary specReviewed 2026-09-28

Sources 1

RoboMME/robomme_benchmark official repository

RoboMME · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Reported benchmark results

Relationships & deployments

Related reading is an editorial crosslink. Sourced connections describe relationships reported in the cited material. A link to a versioned profile does not establish compatibility with that version unless the connection note explicitly identifies it.

Record history & verified changes

A research review records when we checked a source. It does not mark a product launch or a new deployment.

Initial reviewed record. No subsequent field change has been recorded.