A maker used Codex and MuJoCo RL to teach a tiny robot to stand back up

The headline sounds like sci-fi until you see the bill of materials: two servos, an M5Atom smaller than a matchbox, and a policy compiled into a C header file.

That is the SelfRisingRobot project from Japanese maker homemadegarbage. The robot falls over. It stands back up. No cloud. No GPU at inference time. The brain was trained in MuJoCo with reinforcement learning, with OpenAI Codex helping scaffold the simulation and training loop.

This is one of the most practical demos in the June 2026 AlphaSignal digest, and it is not a benchmark slide. It is hardware you can print.

The stack in plain language

The repo splits cleanly:

• 3Dmodel/ for printable mechanics
• RL/ for MuJoCo XML, Gym env, trained PPO checkpoint, eval scripts
• Arduino/ for on-device control

The maker's longer write-up lives at homemadegarbage.com/rl13 (Japanese, with photos and sim videos).

Why self-righting is harder than it looks

Standing up from arbitrary fallen poses is a contact-rich control problem. The policy must coordinate two actuators while the body slides, bounces, and catches edges on the ground.

Training in simulation is the economical path. You can run thousands of trial-and-error episodes for pennies of electricity. Flashing a bad policy onto real hardware breaks servos and wastes weekends.

The workflow here is classic sim-to-real:

1. Build a MuJoCo model that is "good enough" on mass, joint limits, and actuator dynamics
2. Train PPO until the agent reliably recovers from random initial tilts
3. Export the policy weights into embeddable C
4. Run on-device with real IMU feedback and tuned motor constants

homemadegarbage's follow-on posts show the same pattern on a four-leg robot: Codex helped explore walking gaits in sim, then the maker ported workable trajectories to hardware with relatively little friction when the model matched reality.

Where Codex actually helped

Codex did not replace physics. It accelerated the glue code around RL:

• MuJoCo XML tweaks and actuator definitions
• Gymnasium environment wrappers (reset distributions, reward shaping, termination)
• Training scripts and evaluation harnesses
• Iteration when the maker asked for gait changes (for example, weight shift during two-leg walking)

That matches how I use coding agents on robotics side projects. The agent is best at boring scaffolding and refactors. You still own reward design and sim fidelity.

If you want a more formal agent skill for this pattern, SimRig packages a CLI plus agent skill that walks from MJCF inspection to Brax PPO training and evaluation. Different stack, same idea: turn a robot model plus behavior spec into a trained policy with guardrails.

What this unlocks for cheap robots

Three implications matter for applied AI engineers:

Train smart behavior for free, deploy on dumb hardware. Simulation episodes scale. M5-class chips are cheap. The expensive part moves to modeling fidelity, not runtime API bills.

Policies as firmware artifacts. Exporting to policy_network.h treats the neural net like any other embedded lookup table. No Docker. No Wi-Fi dependency. That is huge for toys, education kits, and field prototypes.

Coding agents lower the RL floor. You do not need a robotics PhD to stand up a first MuJoCo env. You need taste in rewards and patience to close the sim-to-real gap.

I have written about adjacent ideas in IMLE real-time robot planning and Snap Spectacles controlling a Unitree Go2. SelfRisingRobot is smaller in scale but clearer in lesson: start in sim, ship a header file.

Limits and honest caveats

This is a two-motor toy, not a warehouse humanoid. Sim-to-real breaks when:

• Motor backlash and battery sag are not modeled
• Contact friction on your floor differs from MuJoCo defaults
• The policy overfits to a narrow reset distribution

homemadegarbage documents real hardware tuning, including measured joint constants on later robots. Expect iteration.

Also, RL in Codex-assisted loops still needs human review of reward hacks. An agent can happily optimize a reward that looks like success in sim and snaps servos on hardware.

How I would replicate the pattern

If you are experimenting this month:

1. Clone SelfRisingRobot and run the sim eval scripts before you print plastic
2. Change one variable at a time (torque limits, reset noise, reward weights)
3. Export policies early and test on hardware with current limits before you chase higher success rates in sim
4. Keep Codex (or your agent of choice) on harness code, not on unsupervised reward rewrites

For production robotics clients, I still pair sim work with structured eval suites and logging. Weekend maker loops can be messier. Client work cannot.

Bottom line

The digest framed this as "reinforcement learning in Codex." The sharper frame is sim-to-real micro-robotics with an agent-written training harness.

Codex did not learn to stand up. PPO did. Codex made it faster to iterate the lab setup that made PPO possible.

If you are building embodied agents and want help choosing sim stacks, export paths, or on-device inference, book a free discovery call.