SELSUS
Autonomous drone racing brings perception and physics together: a drone must find its way through race gates using onboard sensing and software.
Our stack connects camera observations, known geometry, and inertial sensing to gate-aware flight commands. Vision finds the target; geometry places it in space; control turns that estimate into motion.
Designed to fly without human steering. The autonomy loop interprets sensor data, selects the next gate, and sends the commands that guide the simulated drone.
Architecture README ↗A high-level visual overview of our autonomous drone stack, from seeing a gate to moving toward it. Start anywhere; the illustrations make the ideas tangible.
A camera turns a three-dimensional scene into a flat image. Camera intrinsics describe that projection; known gate dimensions help translate the picture back into meters.
A pixel gives a direction, not a distance. This ray has a forward component of 1, rather than unit length. A point at forward depth Z is pC = Zr; its straight-line range can be longer.
Four solid rails surround a square opening. Their known size supplies scale, while their depth changes the shape we see when a gate tilts.
The specification conflicts. Official drawing: 140 mm; specification text: 260 mm. This illustration uses the current 260 mm model.
Distance changes size. Rotation changes the visible rails. Move the gate in camera coordinates to explore both.
Knowing where a gate is only starts the journey. The drone must tilt, accelerate, and manage its momentum to reach the opening with room to continue.
Position changes along three axes. Roll, pitch, and yaw describe orientation. These motions are coupled: tilting the drone redirects its thrust.
Thrust must counter gravity and change velocity. Drag resists motion; momentum carries the drone forward even after its aim changes. Control balances these effects continuously.
| Control concept | What it asks for | Place in this stack |
|---|---|---|
| Rate / ACRO | How quickly to rotate about each body axis. | A useful control concept; not the active command path described here. |
| Attitude + thrust | How to orient the drone and how much thrust to apply. | The active gate-aware controller turns target and vehicle estimates into these commands. |
| MAVLink | A common message format for telemetry and commands. | Carries the attitude-target message to the simulator bridge. |
Rate control · PX4 ↗ · Attitude targets · MAVLink ↗ · Simulator physics · specification page 5 ↗
Classical vision starts with target pixels. Known gate geometry and camera intrinsics turn those clues into repeatable estimates of individual gates.
Selected stages from our vision development tools. Choose a frame, then follow its clues from the original image to gate candidates.
The camera captures the gates and the scene around them.
We explored many classical extraction methods. These five views show how target pixels become evidence for a geometric prediction. Contours & fitting · OpenCV ↗
Start with the target pixels that survived detection. A connected patch may contain several gates, and one gate may appear in separate pieces.
Visible openings and nearby fragments offer starting clues. Missing rails remain unknown.
Move and rotate a known gate until its projection agrees with the visible clues.
A missing pixel may reflect lighting, detection, or an obstruction. It does not prove that a rail is absent.
The fit can extend beyond the picture, but the image border is never treated as a physical gate edge.
One patch of pixels may contain more than one gate. Comparing candidate gates together helps explain the scene while keeping other plausible positions and orientations available.
Full rings, missing rails, cropped images, and overlapping gates offer different clues. A missing rail can be estimated, but it has not been seen.
The outer edge and opening help estimate size and tilt. A nearly face-on gate can still hide small rotations.
Useful evidence ≠ unique orientation.Three visible rails help locate a gate, but more than one orientation may still fit.
The missing rail is inferred.A gate can continue beyond the picture. The edge of the image is a crop, not part of the gate.
The image border is not a rail.Touching shapes may belong to different gates. Their shared pixels do not reveal which gate is closer.
Connected ≠ one physical instance.One rail or a tiny distant shape may not reveal enough to place and orient a gate reliably.
Unresolved is a valid result.A square ring looks the same after a quarter turn. With identical front and back faces, it can also face the other way.
The camera cannot always tell these orientations apart. The flight system needs a consistent way to interpret the gate relative to the drone’s approach.
The drone can roll while the gate stays still. Its IMU helps relate the camera’s view to the vehicle’s attitude.
That context helps interpret tilt; it does not make the gate’s identical sides visually distinct.
A useful target must stay meaningful as the camera moves. Inertial sensing estimates vehicle motion; a persistent gate map connects successive observations to the next flight decision.
Rotation and acceleration over time.
Position relative to the camera.
Account for camera mounting and the drone’s pose.
Associate observations with remembered gates.
Track the next uncrossed gate and the one beyond it.
Use target and vehicle estimates to compute a command.
Small sensor errors accumulate. IMU bias calibration and well-aligned observations matter because a drifting vehicle estimate can also move the mapped target.
The active controller uses the target and next gate to shape its approach. This is gate-aware target selection; broader path-planning experiments remain separate workbenches.
One lens can observe the same gate from several positions. Agreement across those views helps test an estimate and, with enough sideways motion, constrain its depth.
Keep observations tied to a consistent target.
Translation creates a new line of sight.
Estimated relative motion connects the two views.
Strengthen a consistent estimate; reject weak or conflicting views.
The same loop that guides a drone through gates can grow into a reusable foundation for fast, vision-guided robotics: perception, mapping, planning, control, logging, and review working as one system.
Turn the prototype pipeline into a repeatable runtime with clearer interfaces, stronger validation, and better simulation-to-flight checks. Each module should be easy to test on its own and predictable when connected.
Generalize the vision and mapping layers so the system can handle new gate layouts, lighting, camera positions, and obstacle-rich environments. The goal is not one memorized route, but adaptable navigation.
The architecture can support more than racing: inspection paths, indoor autonomy, field robotics, synthetic-data generation, controller benchmarking, and interactive analysis tools for future vehicle experiments.
A platform for future autonomous flight work.
From here, the project can grow from a gate-racing demonstrator into a reusable testbed for perception-driven navigation, aggressive control, and robotics research.