Topic 3: Task Execution & Action Sequencing

Topic 3 focuses on how to organize and execute complex tasks using task graphs, behavior trees, and a skill library. This is where navigation, perception, and manipulation are combined into coherent behaviors.

3.1 Task Graphs & Behavior Trees

Task Graphs

Task graphs represent tasks as nodes (skills, decisions) with edges indicating flow:

• Nodes:
  • Atomic skills (navigate, pick, place, follow).
  • Decision points (if/else based on sensor state).
• Edges:
  • Define ordering, branching, and loop structures.

Task graphs make it easier to:

• Visualize the overall behavior.
• Modify and extend tasks without rewriting low-level code.

Behavior Trees

Behavior trees are a structured form of task graphs with well-defined node types:

• Root: Entry point of the behavior.
• Composite nodes:
  • Sequence (run children in order).
  • Selector (try children until one succeeds).
  • Parallel (run multiple children concurrently).
• Decorator nodes:
  • Modify behavior (retry, timeouts, conditions).
• Leaf nodes:
  • Actions (call skills, send ROS 2 actions).
  • Conditions (check sensor state, task status).

Benefits:

• Clear semantics and reusability.
• Built-in support for failure recovery and retries.
• Easy integration with Nav2 and other ROS 2 tooling.

3.2 Skill Library Construction

You will define a skill library—a set of reusable, well-tested capabilities:

Example skills:

• Pick up object:
  • Input: target object pose.
  • Behavior: approach, reach, grasp, lift.
• Place to target:
  • Input: target surface/pose.
  • Behavior: move, lower, release, retreat.
• Follow human:
  • Input: human pose estimate.
  • Behavior: maintain distance and orientation, avoid obstacles.
• Open drawer / turn knob (if hardware allows):
  • Input: pose of handle/knob.
  • Behavior: reach, apply torque, monitor force.
• Deliver object to coordinate:
  • Input: target room or coordinate.
  • Behavior: navigate, position, present object.

Each skill:

• Has clear preconditions (what must be true before it runs).
• Has success/failure outcomes.
• Publishes status to ROS 2 topics for monitoring and debugging.

3.3 Chaining Skills into Tasks

Complex tasks are composed by chaining skills:

Examples:

• Find object → navigate → grasp → deliver:
  • Perception: detect and localize object.
  • Navigation: move to vicinity.
  • Manipulation: pick up object.
  • Navigation: move to delivery location.
  • Manipulation: place or hand over.
• Track person → follow → maintain distance:
  • Perception: detect and track human pose.
  • Navigation: follow path while avoiding obstacles.
  • Control: maintain safe following distance and orientation.
• Scan room → detect changes → report findings:
  • Navigation: visit waypoints around the room.
  • Perception: capture observations and compare to baseline.
  • Reasoning: summarize differences.
  • Communication: report via speech or console.

Behavior trees or task graphs orchestrate these chains, ensuring:

• Proper error handling and retries.
• Graceful fallback if sub-skills fail.
• Logging of task progress and outcomes.

By the end of Topic 3, you will have:

• A basic skill library implemented as ROS 2 nodes and actions.
• At least one behavior tree or task graph that chains navigation and manipulation into a meaningful humanoid task.