Skatebots: Development Log

Legs, Wheels and a Lot of Stairs

Most of the places people actually go have stairs, kerbs, ramps and uneven ground somewhere along the way, and that’s where mobile robots still tend to get stuck. Skatebots is a collaboration between xLab and DEEP Robotics to change that. The work centres on two of DEEP Robotics’ robots: the Jueying X30 Pro, a rugged quadruped, and the M20, a wheeled-legged robot that can walk or roll depending on the ground under it.

The question we keep coming back to is how to make machines like these both safe and fast. In practice that means a robot that notices the stairs before it reaches them, plans a route through a whole building rather than a single room, and switches to the right way of moving without someone holding a joystick. DEEP Robotics brings the platforms and their autonomy software. We build on that with our own work in perception, planning and control.

This page is our working log. So far most of it is about the X30 Pro, and the M20 has made its first steps in simulation. We add to the Progress Log as things get done. Who knows? One day, they might race.

Who Built What

Almost everything that makes the X30 walk, see and navigate comes from DEEP Robotics. Our work so far sits on top of that. Under the hood the robot has three computers (one each for locomotion, perception and navigation) and four LiDARs that give it a view all the way around its body.

We run DEEP Robotics’ software as it ships and talk to it only through its official interfaces, so anything we build can be compared directly with the stock behaviour.

Mapping and Localization

Provided by DEEP Robotics. The X30’s SLAM builds a 3D point-cloud map from the merged LiDAR scans, together with a 2D occupancy grid for path planning. A localization module then tracks the robot on that prior map. Indoor, outdoor, and fast (no loop closure) mapping modes are available.

Our addition. The vendor exposes mapping and localization through shell scripts on the navigation computer. We wrapped them in a ROS node with a Foxglove panel, so an operator can start mapping in a chosen mode, save the map, start localization and switch to the newest map from a laptop. Every map on the robot is recorded in a manifest in the repository, with its creation time, completeness and associated routes.

3D point-cloud map of a hallway intersection in the Levine building, coloured by height, produced by the vendor SLAM.

Waypoint Navigation

Provided by DEEP Robotics. robot_server accepts a list of waypoints, each carrying its pose, gait, speed, navigation mode, terrain mode and obstacle-avoidance setting. It hands each leg to the vendor’s global planner and path tracker. The tracker selects a target 3 m ahead on the global path, scores a library of 1,872 precomputed candidate paths against the live LiDAR cloud, and steers with an LQR controller. Velocities are capped at 1.5 m/s forward, 0.4 m/s lateral and 0.6 rad/s yaw.

Our addition. DEEP Robotics provides its own operator app for routes. For our research workflow we also wanted routes to be scriptable and version-controlled, so we implemented the robot_server protocol in route_manager and built a Foxglove panel, Route Control, around it. Waypoints are clicked in the 3D view or added at the robot’s current pose. Routes are saved per map and committed automatically to the repository, because a route’s coordinates are meaningful only on the map it was drawn on. We also documented the tracker’s pipeline and parameters for our team.

Multi-Floor Routes

Provided by DEEP Robotics. Multi-floor navigation uses one 3D point cloud for the whole building and a separate 2D grid for each floor. Localization runs on the 3D cloud throughout, including on the stairs. robot_server switches the planning grid when the robot arrives at a waypoint whose floor ID differs from the current one.

Our addition. Setting up a multi-floor route involves several careful manual steps. The grid switch happens on arrival, so the landing waypoint must already carry the next floor. The stairs need to be marked traversable in both grids, and the floor configuration must match the grids produced by mapping. We automated these steps:

• Floor and staircase extraction. Floor heights and staircase locations are recovered from the SLAM mapping trajectory and its per-keyframe floor labels.
• Stair corridors. stair_corridor.py clears a robot-width corridor along the walked staircase in each affected grid, with a preview and an undo.
• Route Control. The panel adds a working-floor selector, go up and go down actions that insert the two waypoints crossing a staircase, per-waypoint leg settings (gait, navigation mode, obstacle avoidance, direction), and validation that blocks a run while the route has errors.

The same type of route in Foxglove. Labels give each waypoint's index, floor (F0, F1) and leg settings. Stair legs run with obstacle avoidance off, since the steps would otherwise register as obstacles.

Terrain-Aware Gait Switching

The X30 requires an explicit gait change before stairs or steep slopes. During waypoint navigation the gait is fixed per leg in advance; under teleoperation it is left to the operator. The platform includes its own automatic switcher, switch_mode. For our research we needed a switcher that we could inspect, tune and extend, so we wrote our own: terrain_gait, a heuristic LiDAR-based terrain classifier and gait planner that runs on the perception computer. It is complete and switches gaits automatically on the robot under teleoperation.

Pipeline

1. Point preparation. Each merged scan is de-skewed point by point using leg odometry. It is then filtered: no-return points (about 30–45% of each scan on the X30), the robot’s own body, out-of-range points, and optionally Livox noise flags, a forward view cone and a 4 cm voxel grid.
2. Accumulation. The last second of scans is kept, together with one keyframe per 10 cm travelled over the last 4 m. Stair nosings hide the treads behind them, so ground seen during the approach is kept until the robot reaches it.
3. Elevation map. A 7 × 7 m robot-centred grid of 5 cm cells. Each cell stores a robust surface height (80th percentile), a height spread, and the tilt and RMS error of a plane fitted over its 30 × 30 cm neighbourhood.
4. Path profile and segmentation. The map is sampled along the direction of travel across the robot’s width. An exact dynamic-programming piecewise-linear fit splits the profile into flat pieces, inclines and jumps. These are labelled walk, slope (8–40°, fit error below 2 cm), stairs (two or more risers of 8–30 cm) or obstacle. The fit error separates ramps from stairs: a flight seen from a distance can average to a 30° incline, but leaves a sawtooth residual of roughly riser/√12, compared with about 1 cm for a true ramp. A separate narrow-obstacle check catches posts and legs, which the median across the robot’s width would otherwise ignore.
5. Gait planner. A state machine computes the braking distance at the current speed and places a hold before the terrain edge. Once the robot has been stationary for 0.5 s, it requests the new gait and releases the hold only after the robot confirms the change. The switch back to walk happens while moving, once the footprint and the area just ahead are clear.

Enforcing the Hold

A gait decision is only useful if the robot actually stops. terrain_gait_gate sits between the operator’s joystick and the vendor’s safety_node. It scales the joystick input exactly as the vendor’s assist mode does, then applies the planner’s hold level: free, no forward motion (braking or blocked; turning and reversing are still allowed) or no motion (gait switch in progress). If the hold signal is lost for 0.5 s, the gate treats it as no motion. The vendor’s own obstacle braking in safety_node stays in the loop.

Planner state overlay: state, current and requested gait, speed, braking distance, distance to the stop point and the reason for the decision, followed by per-cycle timing and the per-filter point counts for the latest scan.

Validation

• Simulation. A purpose-built simulator drives a model of the robot over a test course: 17 cm stairs up and down, 15° and 22° ramps, a 5 cm kerb, a 45 cm step that no gait can climb, and an optional 12 cm post. It uses ray-cast LiDAR from the four real sensor mounts. The simulated world is deliberately imperfect: drifting leg odometry, body bob, sensor misalignment, ghost points at edges, dust, and uneven ground. A lap fails if the robot is on stairs or a slope in the wrong gait, switches gait while moving, or touches the tall step.
• Tests. Unit tests cover the classifier, planner and filters. A closed-loop test runs five noisy laps without ROS, and end-to-end tests run the ROS graph and the gate in a ROS Noetic container.
• On-robot performance. With 4 cm voxels, the decision loop runs at 10 Hz in roughly 20 ms per cycle on the RK3588.
• ROS 1 port. The first version ran in ROS 2 behind ros1_bridge. Copying every 80,000-point scan across the bridge used 58% of a CPU core and dropped 1–3 scans per second. The node now runs natively in ROS 1 and subscribes to the LiDAR directly.
• On the robot. Automatic gait switching works on the X30. The robot detects stairs ahead, stops short of the first riser, switches to the stair gait, and returns to walk once it is back on level ground, with no operator input for gait selection. So far this has been tested under manual teleoperation only. It has not yet been combined with the vendor’s autonomous navigation, which requires each leg’s gait to be fixed before the robot starts moving.

Our Own Controller: Pure Pursuit

The stock stack is built for dependable point-to-point navigation: at waypoints the robot typically stops, turns in place and then continues. For our research we want to study continuous motion through waypoints, with the gait chosen on the fly by terrain_gait rather than fixed in the route. That is what nav3d does. It is our own route follower for 3D waypoints and needs no 2D maps.

A route says only where to go. How to get there comes from what terrain_gait sees: when to change gait, when to turn round before stairs going down (the X30 descends facing up the stairs) and when to walk straight on a flight. The controller is a swappable module behind one interface, while the route, pose, terrain, stair rules, speed limits and logging stay outside it. That way pure pursuit, MPC and later learned controllers can be compared on equal terms. Every command still passes through terrain_gait_gate and the vendor’s safety_node.

The first controller is pure pursuit. It steers along an arc through a look-ahead point that moves further out as the robot speeds up, and slows down in curves and towards the end of each run. It also knows how tightly the current gait can turn while walking: it arcs through corners the gait can take, and turns on the spot only for corners that are too sharp.

Language-Guided Navigation

As an early look at the learning-based side, we let a general-purpose vision-language model, Claude, decide where the robot goes. The model never drives the legs. It is given a task in plain English and works through a small set of tools: look (the camera image plus a map of the area around the robot, with lettered places it can go), go to one of those places, turn, take the stairs, remember a sign on the map, and done. Each move is a single target that the map shows to be free and reachable, and the vendor’s navigation stack walks the robot there with its own obstacle avoidance.

Reading signs. A separate text-recognition (OCR) model reads the camera twice a second the whole time, much faster than the language model could. Anything it reads is pinned to the map as a landmark. A move can be told to stop as soon as a given word is read, and when it ends the model gets a digest of what was read on the way. That digest is how the model decides whether it has reached the goal; it can only finish by quoting the goal’s sign.

In the run below it was asked to find Skirkanich Hall on the same floor, with a hint: “go straight to the dead end, take a right, then go straight and take a left”. It reached the hall in four moves, about 34 m in under three minutes, and finished when it read the hall’s hanging sign.

Simulation

X30 Policies in MuJoCo

DEEP Robotics’ locomotion policies for the X30 can also be run in MuJoCo through the robot’s SDK. That gives us a safe place to try new commands and controllers before they reach the real robot.

M20

Work on the lab’s second robot, the M20, has started in simulation. The robot runs in MuJoCo on a test scene with a ramp, a flight of stairs and a raised platform.

Next Steps

1. Fix pure pursuit on the stairs. Track down the gait misjudgement behind the fall in the first pure pursuit run, add the case to the simulator’s test course, and repeat the on-robot runs until whole multi-floor routes finish cleanly.
2. Language-guided navigation on our controller. Let the language model’s moves run on nav3d and pure pursuit instead of the vendor’s planner, with terrain_gait choosing the gaits.
3. MPC baseline. In parallel, a model-predictive tracker over a planar model of the X30 (forward, lateral and yaw velocity). It respects the robot’s velocity and acceleration limits and treats terrain_gait’s stop points as constraints, so the robot slows smoothly into gait switches instead of braking late. It plugs into nav3d behind the same interface as pure pursuit.
4. Learning-based gait control. Alongside MPC, a learned alternative to the heuristic gait switcher.
5. Evaluation. Compare the vendor tracker, pure pursuit and MPC on the same routes, including stair legs. Metrics: time to complete, number of full stops, cross-track error, and command smoothness.
6. Full VLM control. Move from a model that picks targets to vision-language(-action) models that take over more of the driving, and evaluate them against the classical baselines.

Progress Log

Contributors

John Tappen, Mann Bhanushali, Nandagopal Vidhu, Rahil Parikh, Rahul Mangharam, and Shriyash Bajaj