Overview

A pick-and-place system built on a novel self-recovering Behavior Tree developed in-house by the WIM Research Team, letting the robot recover on its own from collisions, grasp failures, and environmental changes in unstructured, dynamic environments.

Project Background & Purpose

Developed to validate an intelligent control logic that detects and self-recovers from exceptions — such as an object's position unexpectedly changing or a grasp failing — so that work can continue without interruption in real-world conditions. A paper covering this Behavior Tree will be presented at ICCAS 2026 (October 2026).

Components

• Neuromeka Indy7 collaborative robot
• NVIDIA Jetson Orin AGX running the WIM Controller
• ZED X Mini vision camera
• Custom-trained RT-DETR object detection + FoundationPose pose estimation

Process Steps

1. 1ZED X Mini and RT-DETR detect the target object in real time in a dynamic environment
2. 2FoundationPose estimates the object's 3D pose to compute the pick position
3. 3The Behavior Tree executes the pick-and-place task while detecting exceptions in real time — collisions, grasp failures, or object displacement
4. 4Smart Recovery (object-displacement recovery) or Grasp Failure Recovery kicks in to self-recover before resuming the task