Reinforcement Learning Environment

We now have a Reinforcement Learning Environment which uses Pybullet and OpenAI Gym! It contains a variety of optional terrains, which can be activated using heightfield=True in the environment class constructor.

If you try to launch the vanilla gait on fairly difficult terrain, Spot will fall very quickly:

PyBullet

By training an Augmented Random Search agent, this can be overcome:

PyBullet

If you are new to RL, I recommend you try a simpler example. Notice that if we choose non-ideal parameters for the generated gait, the robot drifts over time with a forward command:

PyBullet

You should try to train a policy which outputs a yaw command to eliminate the robot's drift, like this:

PyBullet

You can choose a PNG-generated terrain:

PyBullet

Or, for more control, you can choose a programmatically generated heightfield:

PyBullet

Notice that when the simulation resets, the terrain changes. What you cannot see is that the robot's link masses and frictions also change under the hood for added training robustness:

PyBullet

Quickstart Reinforcement Learning

pip3 install numpy
pip3 install pybullet
pip3 install gym

cd spot_bullet/src

./spot_ars_eval.py

Optional Arguments

-h, --help          show this help message and exit
-hf, --HeightField  Use HeightField
-r, --DebugRack     Put Spot on an Elevated Rack
-p, --DebugPath     Draw Spot's Foot Path
-a, --AgentNum      Agent Number To Load