Overview

An MLOps console that monitors VLA policy retraining → evaluation → canary rollout → promotion or rollback, plus edge-fleet status, on one screen.

Project Background & Purpose

Every field spec change meant days of hand-stitched steps from retraining to rollout, with no criterion for reverting a regression after deployment. The project chains dataset version → training config → checkpoint → evaluation → deployment version under hashes, sends only candidates that pass 1,000 sim and 50 hardware evaluation episodes to canary, and requires all six gates before full promotion. The console is an internal tool of the customer's MLOps team built on the logging and lineage API.

Components

• 3B-class dual-arm VLA, LoRA r16 fine-tuning (H200 ×2, 4 h 20 m), model registry
• Evaluation pipeline (1,000 sim episodes at 94.1 %, 50 hardware episodes at 92 %), six gates
• Edge fleet (onboard inference board, INT8, 38 ms), rollback to v2.6, cell real-time controller (W-RC at 1 kHz), logging and lineage API

Process Steps

1. 1Triggered by an 8 % rise in grasp failures, collect 214 teleop episodes plus 40k synthetic frames and retrain with LoRA (4 h 20 m)
2. 2Canary-deploy ckpt-41200, which passed 1,000 sim and 50 hardware episodes, to one of four edge devices (25 %)
3. 3Check success, alignment, safety-stop, collision, and latency gates over 24 h; promote fully on pass, roll back to v2.6 on miss