Research guide / System architecture

Physical AI Stack Explained

The physical AI stack is a set of connected responsibilities: data, simulated environments, models, robot interfaces, runtime hardware and operations. Map the interfaces and evidence at each layer before assuming that a collection of compatible-looking tools forms a deployable system.

Source contextPrimary specReviewed
Sources 11
google-deepmind/open_x_embodiment official repository

google-deepmind · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

ros2/ros2 official repository

ros2 · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Jetson Thor official product specifications

NVIDIA · Publication date not disclosed · Source accessed 2026-09-28

droid-dataset/droid official repository

droid-dataset · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

google-deepmind/mujoco official repository

google-deepmind · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

isaac-sim/IsaacSim official repository

isaac-sim · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

isaac-sim/IsaacLab official repository

isaac-sim · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Physical-Intelligence/openpi official repository

Physical-Intelligence · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

huggingface/lerobot official repository

huggingface · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Lifelong-Robot-Learning/LIBERO official repository

Lifelong-Robot-Learning · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

GXO multi-year agreement with Agility Robotics

GXO · Published 2024-06-27 · Source accessed 2026-09-28

Customer describes commercial live-warehouse operation after a pilot. Does not identify current Digit 5 hardware.

Reviewed recordLast reviewed 11 sources ↗3 sourced fields
Identity boundary. Original PhysicalAI.best editorial guide. Source links support factual examples; evaluation questions and decision rules are editorial guidance, not upstream endorsements.

Data and provenance

Begin with the observations, actions and outcomes used for training or evaluation. Track collection hardware, operator involvement, scene coverage and dataset terms. DROID documents a shared collection platform, while Open X-Embodiment aggregates constituent datasets. A common storage format does not make every sensor, action convention or license identical. The data contract is an integration artifact as well as a provenance record.

Source contextPrimary specReviewed
Sources 2
droid-dataset/droid official repository

droid-dataset · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

google-deepmind/open_x_embodiment official repository

google-deepmind · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Simulation and task environments

Separate the simulator from the benchmark or task definition that runs on it. MuJoCo is a physics engine; Isaac Sim and Isaac Lab serve distinct simulation and learning workflows. Choose the environment around the required sensors, contacts, assets and experiment. Pin versions and record configuration changes, because a simulator upgrade or altered success rule can change what a result means.

Source contextEditorial synthesisReviewed
Sources 3
google-deepmind/mujoco official repository

google-deepmind · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

isaac-sim/IsaacSim official repository

isaac-sim · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

isaac-sim/IsaacLab official repository

isaac-sim · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Models and adaptation tooling

Identify the checkpoint and the tools used to train, adapt or serve it. OpenPI and LeRobot expose software workflows; their presence in a stack does not specify a single universal policy or a complete supported robot. Record observation preprocessing, action normalization, local task data and execution configuration. Keep the software license distinct from the terms for model weights and training datasets.

Source contextEditorial synthesisReviewed
Sources 2
Physical-Intelligence/openpi official repository

Physical-Intelligence · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

huggingface/lerobot official repository

huggingface · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Robot interfaces and control

Connect the model’s outputs to the actual robot interfaces. ROS 2 provides an ecosystem for components and messages, but a distribution label is not proof that a particular camera, arm or controller is supported. Assign responsibility for drivers, calibration, command limits, state feedback and recovery. Document the tested interface versions rather than relying on a broad compatible-with-ROS statement.

Source contextPrimary specReviewed
Sources 1
ros2/ros2 official repository

ros2 · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Runtime and compute

Select compute for the actual workload and physical installation. The Jetson AGX Thor Developer Kit is a specific development product, not a guarantee about every robot using the same processor family. Evaluate the chosen model, precision, sensor load, latency, memory and thermal conditions together. If inference depends on remote services, include connectivity, failure handling and data access in the architecture review.

Source contextPrimary specReviewed
Sources 1
Jetson Thor official product specifications

NVIDIA · Publication date not disclosed · Source accessed 2026-09-28

Evaluation and operating feedback

Close the diagram with task evaluation, logs, maintenance and change control. Record which layer produced a failure and which revision was running. A new checkpoint, camera, gripper or software package should trigger the tests that depend on it. Benchmark evidence and customer operation belong in separate records connected to the same system, so an improvement in one layer does not silently become a claim about every deployed workflow.

Source contextEditorial synthesisReviewed
Sources 2
Lifelong-Robot-Learning/LIBERO official repository

Lifelong-Robot-Learning · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

GXO multi-year agreement with Agility Robotics

GXO · Published 2024-06-27 · Source accessed 2026-09-28

Customer describes commercial live-warehouse operation after a pilot. Does not identify current Digit 5 hardware.

Additional disclosed fields

Data-layer example
Open X-Embodiment provides a shared robot episode format and constituent catalog
Primary specReviewed
Sources 1
google-deepmind/open_x_embodiment official repository

google-deepmind · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Middleware example
ROS 2 provides robot software libraries, tools and interfaces
Primary specReviewed
Sources 1
ros2/ros2 official repository

ros2 · Publication date not disclosed · Source accessed 2026-09-28

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

Compute example
Jetson AGX Thor Developer Kit is an embedded development platform
Primary specReviewed
Sources 1
Jetson Thor official product specifications

NVIDIA · Publication date not disclosed · Source accessed 2026-09-28

What this evidence does not establish

  • The stack is an editorial responsibility map, not a mandatory vendor architecture.
  • Tool availability does not imply mutual compatibility or production readiness.

Relationships & deployments

Related reading is an editorial crosslink. Sourced connections describe relationships reported in the cited material. A link to a versioned profile does not establish compatibility with that version unless the connection note explicitly identifies it.

Record history & verified changes

A research review records when we checked a source. It does not mark a product launch or a new deployment.

Initial reviewed record. No subsequent field change has been recorded.

Inspect the evidence

Sources & evidence

md-openx
google-deepmind/open_x_embodiment official repository

google-deepmind · Repository · Publication date not disclosed · Source accessed 2026-09-28

License: Apache-2.0 repository software; CC BY 4.0 other repository content; constituent dataset licenses separate. Factual summary and attribution only; no upstream prose, images, weights, or dataset redistributed.

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

md-ros
ros2/ros2 official repository

ros2 · Repository · Publication date not disclosed · Source accessed 2026-09-28

Factual summary and attribution only; no upstream prose, images, weights, or dataset redistributed.

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

md-droid
droid-dataset/droid official repository

droid-dataset · Repository · Publication date not disclosed · Source accessed 2026-09-28

Factual summary and attribution only; no upstream prose, images, weights, or dataset redistributed.

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

md-mujoco
google-deepmind/mujoco official repository

google-deepmind · Repository · Publication date not disclosed · Source accessed 2026-09-28

License: Apache-2.0 code; CC BY 4.0 documentation. Factual summary and attribution only; no upstream prose, images, weights, or dataset redistributed.

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

md-isaacsim
isaac-sim/IsaacSim official repository

isaac-sim · Repository · Publication date not disclosed · Source accessed 2026-09-28

License: Apache-2.0 repository code; NVIDIA and third-party component terms apply separately. Factual summary and attribution only; no upstream prose, images, weights, or dataset redistributed.

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

md-isaaclab
isaac-sim/IsaacLab official repository

isaac-sim · Repository · Publication date not disclosed · Source accessed 2026-09-28

License: BSD-3-Clause framework; isaaclab_mimic Apache-2.0; dependency terms separate. Factual summary and attribution only; no upstream prose, images, weights, or dataset redistributed.

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

md-openpi
Physical-Intelligence/openpi official repository

Physical-Intelligence · Repository · Publication date not disclosed · Source accessed 2026-09-28

License: Apache-2.0 code; Gemma and other component terms apply separately. Factual summary and attribution only; no upstream prose, images, weights, or dataset redistributed.

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

md-lerobot
huggingface/lerobot official repository

huggingface · Repository · Publication date not disclosed · Source accessed 2026-09-28

License: Apache-2.0 code. Factual summary and attribution only; no upstream prose, images, weights, or dataset redistributed.

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

md-libero
Lifelong-Robot-Learning/LIBERO official repository

Lifelong-Robot-Learning · Repository · Publication date not disclosed · Source accessed 2026-09-28

License: MIT code; CC BY 4.0 dataset. Factual summary and attribution only; no upstream prose, images, weights, or dataset redistributed.

Official README inspected live. Repository code terms do not automatically cover model weights, data, or third-party assets.

pr-dep-gxo-digit
GXO multi-year agreement with Agility Robotics

GXO · Customer · Published 2024-06-27 · Source accessed 2026-09-28

Factual summary and attribution only; no upstream prose or dataset redistributed.

Customer describes commercial live-warehouse operation after a pilot. Does not identify current Digit 5 hardware.