PhysicalAI.bestOur standard
Benchmark / simulation

VLABench

VLABench focuses on language-conditioned manipulation with longer task structure and semantic generalization challenges.

Source contextPrimary specReviewed
Sources 1
OpenMOSS/VLABench official repository

OpenMOSS · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Reviewed recordLast reviewed 1 sources ↗6 sourced fields
Official source Suggest a correction

What the protocol tests

Its tracks separate in-distribution testing from category, commonsense, instruction, task and texture changes. The selected track and training setup matter more than an undifferentiated average.

Source contextPrimary specReviewed
Sources 1
OpenMOSS/VLABench official repository

OpenMOSS · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Reading results

The maintainers recommend fixed evaluation episodes for comparable testing and warn that random sampling can produce invalid initializations. A result should cite the episode set and control representation.

Source contextPrimary specReviewed
Sources 1
OpenMOSS/VLABench official repository

OpenMOSS · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Methodology and reuse limits

Cross-track numbers answer different generalization questions. Data and asset reuse terms were not fully established in this review.

Source contextPrimary specReviewed
Sources 1
OpenMOSS/VLABench official repository

OpenMOSS · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Evaluation protocol

Exact version / scope
Official evaluation tracks
Primary specReviewed
Sources 1
OpenMOSS/VLABench official repository

OpenMOSS · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Environment
simulation
Primary specReviewed
Sources 1
OpenMOSS/VLABench official repository

OpenMOSS · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Metric definition
Task success rate by evaluation track
Primary specReviewed
Sources 1
OpenMOSS/VLABench official repository

OpenMOSS · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Task scope
Not publicly disclosed
Embodiment
Task-configured manipulation environments
Primary specReviewed
Sources 1
OpenMOSS/VLABench official repository

OpenMOSS · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Access & reuse

Access model
Public research documentation and evaluation resources
Primary specReviewed
Sources 1
OpenMOSS/VLABench official repository

OpenMOSS · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

License
Not publicly disclosed

Additional disclosed fields

Category
Robot evaluation benchmark
Primary specReviewed
Sources 1
OpenMOSS/VLABench official repository

OpenMOSS · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Reported benchmark results

Context before scores. Results from different benchmarks are not directly comparable. A simulation result does not establish real-world reliability, safety or commercial availability.
No result meets our complete revision and methodology requirements for this record yet. Inspect the original benchmark documentation before comparing published scores.

What this evidence does not establish

  • Cross-track numbers answer different generalization questions. Data and asset reuse terms were not fully established in this review.

Relationships & deployments

modelPhysical Intelligence π0.5 ↗modelOpenVLA ↗benchmarkLIBERO ↗

Related reading is an editorial crosslink. Sourced connections describe relationships reported in the cited material. A link to a versioned profile does not establish compatibility with that version unless the connection note explicitly identifies it.

Record history & verified changes

A research review records when we checked a source. It does not mark a product launch or a new deployment.

Initial reviewed record. No subsequent field change has been recorded.

Inspect the evidence

Sources & evidence

md-vlabench
OpenMOSS/VLABench official repository

OpenMOSS · Repository · Publication date not disclosed · Source accessed 2026-09-28

Factual summary and attribution only; no upstream prose, images, weights, or dataset redistributed.

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Referenced by 1 record: VLABench