Closing the Sim2Real Gap in Offroad Lidar Simulation
- clemenslinnhoff
- Jun 10
- 4 min read
When we talk about lidar simulation, most discussions focus on sensor models, ray tracing, or point cloud generation. But in many real-world applications, the biggest challenge lies elsewhere: the materials.
This becomes especially clear in agricultural environments, where the scene is dominated by soil, and where that soil is anything but uniform.
The challenge: simulating soil is harder than it looks
In offroad scenarios, e.g. in agriculture, perception systems operate in an environment that is fundamentally different from structured road scenarios. Instead of asphalt and well-defined objects, you are dealing with:
unstructured terrain
varying soil compositions
changing moisture levels
dynamic surface conditions
All of these factors directly influence how lidar signals are reflected. As outlined in a recent scientific publication we co-authored [1], this makes realistic simulation particularly challenging, since the physical material properties, and therefore the sensor response, are often unknown. At the same time, these properties are crucial. Especially when developing AI-based perception functions such as point cloud semantic segmentation, small inaccuracies in reflectivity can lead to significant performance gaps when transferring from simulation to reality.
From generic materials to measured reality
To address this, the work focuses on a simple but powerful idea: measure the materials instead of guessing them.
The application example is the agricultural use case light tillage. In this operational design domain (ODD), the soil is the dominant surface and therefore the most critical element for simulation accuracy. Instead of treating soil as a generic material, the approach breaks it down into its fundamental components:
sand
silt
clay
In addition, an often-overlooked factor is explicitly considered: soil moisture. Different moisture levels significantly change reflectivity and therefore the lidar response. To capture this, soil samples are systematically prepared and evaluated across realistic moisture levels defined together with agricultural experts. This results in a set of well-defined, physically grounded material configurations representing real agricultural conditions.
Measuring reflectivity with BRDFs
The key step is the measurement itself. Using our Reflectivity Measurement Device, the bidirectional reflectance distribution function (BRDF) is captured for each soil and moisture combination. This allows us to describe how light is reflected depending on:
incidence angle
observation angle
wavelength
Rather than assigning a single reflectivity value, the full angular behavior of the material is captured. The measured BRDFs are then used as lookup tables in the simulation. This is a crucial difference: the simulation no longer approximates reflectivity. It reproduces measured light-material interaction.
Integrating measurements into lidar simulation
The measured material data is integrated into a full sensor simulation pipeline.
In this case, the environment and sensor simulation is performed using dSPACE AURELION, where:
the scene geometry defines where rays hit
the material IDs define which BRDF is applied
the BRDF determines how much energy is reflected
At runtime, the lidar model evaluates the BRDF for each interaction between a ray and the surface, resulting in physically consistent reflectivity values. Further effects such as surface roughness are incorporated through additional modeling in the sensor pipeline. The outcome is a synthetic point cloud that closely mimics the behavior of a real sensor, not just geometrically, but also in terms of signal intensity.
Validation: does it actually make a difference?
The impact of measured material data becomes visible in both direct and indirect validation.
First, simulated and real lidar data are compared directly using statistical metrics with Persival Avelon. The results show:
very low bias in reflectivity
low scattering error
This indicates that the measured material models provide a very accurate fit to real-world sensor behavior.
Indirect validation is performed on application level.
A semantic segmentation network, trained purely on real data, is evaluated on simulated point clouds. Here, the choice of BRDF has a direct influence on performance. Different soil types and reflectivity characteristics lead to noticeable differences in segmentation accuracy. In fact, the simulation can even be tuned to better represent the real-world scenario than the original dataset, revealing insights about the underlying soil composition.
Why this matters for agricultural automation
The key takeaway is not just that simulation becomes more accurate. It’s that simulation becomes usable as a validation tool. By integrating measured reflectivity:
the sim2real gap is significantly reduced,
AI models can be pre-validated before field deployment,
rare or difficult conditions can be tested virtually.
This is particularly important in offroad environments, such as in agriculture, where:
data collection is expensive and time-consuming,
environmental conditions vary widely,
edge cases are hard to reproduce.
As shown in the study, measured BRDFs make it possible to bring simulation fidelity to a level where it can support real development workflows.
From agriculture to broader applications
While this work focuses on soil and agricultural machinery, the underlying principle applies far beyond this domain. Whenever lidar perception depends on material properties, and it always does, accurate reflectivity data becomes a key enabler for realistic simulation. Agriculture simply makes this especially visible. Because when your environment is literally defined by its material, you can no longer afford to approximate it.
References
[1] M. Felske, J. Redenius, P. Rosenberger, N. Rüddenklau: "Optimizing Real-Time LiDAR Simulation with Reflectivity Measurements for Deep Learning Semantic Segmentation in Autonomous Light Tillage", LAND.TECHNIK AgEng 2025 (pp.125-132), 2025, DOI: 10.51202/9783181024652-125
Comments