Module 3: Mapping, SLAM & World Reconstruction

Module 3 focuses on where the robot is and what the world looks like. You will learn SLAM fundamentals, build maps from camera and LiDAR data, and run the same pipelines inside your digital twin before deploying to hardware.

3.1 SLAM Fundamentals

What is SLAM?

SLAM (Simultaneous Localization and Mapping) solves two problems at once:

• Localization: Where am I?
• Mapping: What does my environment look like?

SLAM operates in a feedback loop:

• As the robot moves, it collects sensor data (images, depth, LiDAR)
• It estimates its pose relative to a map
• It updates the map with new observations

VSLAM vs LiDAR SLAM

• VSLAM (Visual SLAM):

  • Uses cameras (RGB or RGB-D), often fused with IMU
  • Works in unstructured environments where texture is available
  • Can be more compact (just a camera + IMU)

• LiDAR SLAM:

  • Uses LiDAR scans and sometimes IMU
  • Often more robust to lighting conditions
  • Produces accurate geometric maps

Both approaches:

• Extract features or perform direct alignment of frames
• Use loop closure to correct accumulated drift

3.2 Building Maps in Real Time

VSLAM Pipeline

A typical visual SLAM pipeline includes:

• Feature detection and matching:
  • Identify keypoints (e.g., ORB) in each frame
  • Match them between frames to estimate motion
• Pose estimation:
  • Estimate camera motion using epipolar geometry or direct alignment
  • Fuse with IMU for stability and scale
• Keyframes and landmarks:
  • Select key frames to store in a global graph
  • Maintain 3D landmarks observed across multiple views
• Loop closure:
  • Detect when the robot revisits a place
  • Adjust the pose graph to reduce drift

Map Representations

You will work with:

• Occupancy grids:
  • 2D grids marking free, occupied, or unknown cells
  • Common input for navigation planners (e.g., Nav2)
• TSDF and 3D meshes:
  • TSDF (Truncated Signed Distance Function) stores distances to nearest surfaces
  • Meshes provide continuous 3D surfaces

These maps support:

• Navigation (collision avoidance, path planning)
• High-level reasoning about rooms and obstacles

3.3 SLAM Inside the Digital Twin

Why Run SLAM in Simulation?

Running SLAM inside your Gazebo or Isaac Sim environments allows you to:

• Test algorithms safely before hardware
• Compare estimated trajectories to simulation ground truth
• Stress-test failure cases (poor lighting, repetitive textures, dynamic obstacles)

Workflow

1. Simulate sensors:
   • Use Chapter 3 setups to provide camera/depth/LiDAR in Gazebo or Isaac Sim
2. Run the same SLAM nodes:
   • Use identical ROS 2 nodes and configurations as you will on the real robot
3. Record ground truth and SLAM outputs:
   • Ground truth poses from simulation
   • SLAM-estimated poses and maps
4. Compare and tune:
   • Compute trajectory errors
   • Inspect maps visually and quantitatively

By the end of Module 3, you should have:

• A SLAM-based mapping pipeline running in ROS 2
• Maps that can be used by planners and controllers
• A validated path from sim to real for localization and mapping