Transformer Transformer: A Unified Model for Motion-Conditioned Robot Co-Design
Hacker News · 2026-07-29
Transformer Transformer is a unified model designed for motion-conditioned robot co-design, capable of generating a complete robot, including every link, joint, motor, and inertial property, optimized for a given manipulation demonstration. This model, trained on RoboTokens—a novel, unified tokenization of robot embodiments, states, and and actions—achieved a 73% reduction in cloth tracking error and a 30% reduction in maximum joint speed compared to the original design in a fabricated cloth-flinging robot.
The architecture operates across diverse embodiment spaces (e.g., wheeled bimanual, quadrupeds, humanoids) and use cases, performing as a generator, critic, and cross-embodiment controller. It functions as a reward-agnostic dynamics model, converting predictions into reward-specific value assessments at inference time to steer embodiment diffusion through a process termed Dynamics Self-Guidance. This enables zero-shot optimization for previously unseen rewards and trajectories across three distinct design spaces, outperforming evolutionary baselines in both performance and runtime. The RoboTokens representation allows the model to uniformly describe robots ranging from 0.65 kg dexterous hands to 67.5 kg quadrupeds, using 28–101 tokens, a significantly more compact representation than traditional formats like MJCF.
The full article also explores the architectural details and the dynamics of self-guidance for zero-shot reward optimization.