Demos.
Papers describe the method. These show what it actually looks like, some of it running live in your browser.
Guerickestraße, rebuilt in three splat segments
About 8–16 MB per segment, loaded one at a time as you drive.
Real drives, replayed in simulation.
Cam2Sim started from a practical problem. We had many hours of real driving recordings, but once an experiment was finished we could only replay the recorded sensor data. We wanted to turn these recordings into environments in which the driving system could interact again. Cam2Sim reconstructs the recorded environment in CARLA and uses Gaussian Splatting for the camera view. The reconstruction is not a complete replacement for the real environment, but it allows us to reuse real experiments for closed-loop simulation.
Drive one of those recordings above: a simple simulator on Guerickestraße in Munich, with the road traced from the recorded drive, and beside it the camera view for the car's current pose, generated from a Gaussian-splat reconstruction trained on that drive's camera frames, in three segments. Below it is the Cam2Sim demo video.
Not loaded yet.
Click the scene, then drive with the arrow keys or WASD. On a phone, use the buttons. The car stays near the recorded path, where the reconstruction has views to draw from. The small inset is the nearest recorded camera frame.
The same reality gap, across four execution levels.
An open-source small-scale vehicle and its Unity digital twin, compared against Fortuna at full scale, across simulation, vehicle-in-the-loop, mixed-reality, and real-world testing.
Loading the scan…
This demo needs WebGL, which your browser has turned off.
The same perturbation, any simulator.
PerturbationDrive generates camera perturbations—like rain, or a corrupted feed—and applies them through a simulator adapter, so the same scenario runs unchanged across Udacity's simulator, a small-scale digital twin, or a full driving stack. Here it runs on the camera following a scan of Fortuna round a test track: pick any of the library's perturbations and the scale it is applied at.
- < 0.5 m
- < 1.2 m
- < 2.5 m
- more
The same perturbations, on a real vehicle.
We injected PerturbationDrive's camera perturbations online while the driving model steered the vehicle along Guerickestraße in Munich. Pick a perturbation and the intensity it was driven at to see the recorded trajectory against the nominal run and how the run ended.
Forecasting near misses to focus fuzzing.
Foresee monitors nearby agents, classifies and ranks the riskiest ones, then clips and fuzzes the scenario around them—so testing time goes where failures are most likely to surface.
Do image-quality metrics track what actually matters?
The ICST 2024 talk on assessing quality metrics for neural reality-gap input mitigation—testing whether standard image-quality metrics track how neural image translation affects a driving system's perception and control.
Fitting a neural network onto a 40-gram nano-drone.
Before autonomous cars, I worked on autonomous nano-drones. At the Dalle Molle Institute for Artificial Intelligence (IDSIA) in Lugano, I helped fit a relative-localization neural network onto a 40-gram nano-quadrotor: a single onboard camera and under 100 mW of power, enough to track a peer drone in real time (IEEE ICRA 2023). The platform changed; the question didn’t: what has to be true for a learned model to work outside the lab it was trained in?