Skip to main content

Hand-eye calibration

This measures where on the robot the camera is mounted and at what angle. With it, obstacles the camera sees land in the right place in robot coordinates, and obstacle perception turns on when the stack starts.

Calibration in the left rail shows only when the current session reports a camera model: a camera is set up and the stack was started with the camera on.

The panel has two views, Current value and New measurement. It opens on Current value. Measure again goes to New measurement, and Current values at the head of the New measurement panel goes back. Reopening the panel shows the view you used last.

New measurement​

Fix a calibration board (ChArUco or checkerboard) in front of the robot first. Support gives you the board spec along with the board.

New measurement puts the Calibration stage where the canvas usually is.

  • From aiming on, the camera view fills the stage. While a measurement runs, the step, the detection guide and the End measurement icon button are on that view too.
  • The tabs on the left handle the board spec, aiming, planning, the result and saving.
  • The left panel always floats over the stage, whatever the screen width.

Calibration › New measurement, the board spec form. Under the active camera row: capture mode Manual or Autonomous (selected) with its description, target samples, board type ChArUco or Checkerboard, squares, square side, marker side, the ArUco dictionary field and the "Next: aim at the board" button.

  1. Choose the Capture mode.
    • Manual: you move the camera by hand in FREEDRIVE and make about 18 to 26 poses yourself.
    • Autonomous (default): you aim at the board once and approve a move; the robot then takes the second seed and drives the planned poses on its own. Right after a camera is remounted, a large gap between the saved value and the real mount can stop it at the mount check. Measure manually first in that case.
  2. Enter the board type, squares, square side (mm) and target samples. A ChArUco board also needs the marker side (mm) and the ArUco dictionary. If the robot has a current value, the board spec used for it is filled in. A spec that does not fit stops you from going on.
  3. In the aiming step, Jog the robot until the board sits in the centre of the camera view at 0.5–0.7 m, using the guide lines. The Jog panel is under the cell 3D view; if you cannot see it, press split view at the top.
  4. For Autonomous, press Start seed capture and follow the flow below. The seed is taken from two viewpoints because one viewpoint alone cannot tell a camera mount fault from a moved board.
  5. The stage shows the steps Waiting for seed → Collecting samples → Solving, with the board detection state and what to do next. The detection guide is the calibration engine's own text, shown as is (in English).
  6. Check the residuals and the verdict on the result screen. The verdict is Passed or Below bar; a measurement you stopped reads Stopped. You can save a below-bar result after confirming it, but measuring again is better, and a below-bar record cannot be checked on the robot.
  7. If you like, enter Label (optional), then press Apply (after restart). Above the button you see this robot's value now and what it becomes (previous → new measurement).

The steps of the autonomous flow:

  • First seed: taken at the aimed pose without moving. It checks the distance to the board and the angle off the view centre with the same bar the drive start uses. If it falls short, the guide says which way to adjust, and you can Jog meanwhile. The pose must pass and hold still within 150 s.
  • Second seed: an approval card shows the TCP travel, the up/down direction, the turn and the expected time. The camera keeps looking at the board centre while turning a little. Press Approve move and then Confirm move within 60 s. After a countdown the robot moves slowly, takes the second seed, and after another countdown returns along the same path.
  • Back to aiming: if the TCP comes back more than 5 mm or 5° from the first pose, or the second viewpoint gave no usable board view, the robot returns and stops. Typical causes are a mount far from its setting, a wrong board size, reflections or occlusion, and a dropped video stream.
  • Build plan: up to 3 minutes. It does not start the drive on its own; the robot drives only when you press Start drive. Back in the planning step returns to aiming.
  • During seed capture, the second-seed move and plan building, Cancel (next to the progress and under the centre view) stops it. Cancelling during the countdown means no move. Cancelling mid-move halts the robot where it is.
  • End measurement on the stage ends a measurement at any time. In Autonomous it halts the robot where it is, even mid-move.

Neither Cancel nor End measurement applies the brakes. For that, use Stop at the right end of the bottom robot control bar. Emergency Stop is the physical E-Stop button.

Label and name

With no label, the name is the camera model and the date (for example zedxm-2026-10-01-v1); a second one on the same day is -v2. Only the label's ASCII letters and digits go into the name. A label in Korean only is kept in the record, and the name is made from the date as if the label were empty. You can rename it after saving, under Current value.

After saving:

  • The value takes effect from the next stack start. The saved screen reads Saved. Applies after restart with the change (previous → new).
  • Restart robot stack on the saved screen restarts the stack after you confirm, and Current value goes from Restart needed to Running. The on-robot check is refused before the restart, so the saved screen has no check button; use Show current value to go to the card.
  • The result also shows as a notification. If saving fails, the reason stays above the button.
  • Copy result still copies the full result after saving.
  • If the connection drops while saving, you cannot tell whether it saved. Check under Current value.
  • A result you have not applied survives a stack restart. Apply it from Unapplied result under Current value.
Restart robot stack

Restart robot stack is the same as pressing S twice on the plem screen: every robot of this plem session brakes, stops and starts again, so it asks first. Meanwhile the Pendant disconnects and shows Restarting, and it comes back when the server does. If it is not back within 2 minutes, check the plem screen.

It refuses, with the reason, while a Job, a Jog session or a calibration drive runs. It cannot see Freedrive or another ROS program moving the robot, so press it only once the robot has stopped. In a multi-robot setup it watches only the first robot. On a stack started without the plem screen the button is off with the reason; restart from where the stack was started.

Retry after a failure​

When a measurement fails, the result screen's button picks where to start again from the cause, skipping steps that need no repeat.

ButtonWhenGoes to
Drive the same plan againThe plan is still usable: stopped, busy, move failed, seed wait timed out, below bar and similar. For below bar it measures once more at the same posesJust before Start drive in the planning step. On start the robot checks the board position again and does not drive if the board moved much since planning
Start again from aimingThe plan itself does not fit: the board moved, too few reachable poses or turn range, no solution from the samples, no planThe aiming step. A new seed builds a new plan
Start again from the board specThe board spec does not match the real boardThe board spec form

A manual measurement has no plan, so it offers only Start again from aiming or Start again from the board spec.

If the planning step fails before the drive (seed capture, second-seed move, plan building), its button depends on where the robot is. After a cancel, the message says where the robot is.

SituationRobot positionButton
Any failure not in the rows below, including a dropped connection during seed capture or plan buildingThe system did not move the robotTry again: takes the first seed again from the current pose
The second-seed move did not finish · the return pose differs from the first or could not be confirmed · the connection dropped during that moveMay not be at the first poseBack to aiming
You cancelledThe message saysBack to aiming

On Back to aiming, check the robot state and the camera cable, Jog to aim again, then start.

When the drive end is not confirmed​

If the connection drops during the drive, or the robot's move result is not confirmed, the stage says so with possible causes: E-Stop, robot mode change, controller error, move timeout, server disconnected. The robot may move again.

  1. Block the drive with the E-Stop.
  2. Press I blocked the drive with the E-Stop button. Until then the screen stays locked and Start drive in the planning step stays locked too.
  3. On the plem screen, stop the robot stack with S and start it again with S. Step 2 only unlocks the screen; it does not check whether the previous session ended.
  4. Release the E-Stop.
The autonomous drive moves the robot

Clear the area around the board and stay within reach of the Emergency Stop. During the drive and the second-seed move most of the screen locks; it unlocks when you stop or cancel, or when it ends.

  • Locked: the other groups and tabs of the sidebar, help, Job run · Run again · Continue run in the bottom bar, the robot mode (BRAKED · TRAJ · FREE) in the bottom bar, and the Jog panel. On a narrow screen the sidebar closes.
  • Not locked: the Program and Cell tabs and split view at the top, and Stop at the right end of the bottom bar. Asking for the Cell tab alone switches to split so the camera view stays. The physical E-Stop does not depend on the screen lock.
If you leave the page

During the drive, seed capture, the second-seed move or plan building, or with an unsaved result, the browser asks once more before a reload or close. A reload restarts this screen from the beginning. The server cancels any seed capture, move or plan building in progress, and a moving robot stops where it is.

A running measurement is cancelled only when no screen is left connected to the server. With another screen connected (another Pendant tab, for example) the measurement can go on. If the robot moves, press the E-Stop.

Current value​

The Hand-eye calibration card of the connected robot. Current value at the top is the value this robot uses; Records below lists other values you can switch to, newest measurement first. Another robot's value is not shown or changed here; change it in the robot settings on the plem screen.

The Current value screen. The Hand-eye calibration card shows the value in use with Current and Running marks, then the Measure again, Check on robot, Clear and Import buttons and a list of other records. A record measured with another camera is dimmed with its reason.

If a measurement is not applied yet, Unapplied result at the top of the card shows its time, residuals, quality (Passed or Below bar) and an Apply button.

  • The result stays on the robot device; a page reload or stack restart does not remove it.
  • Apply is the same as Apply (after restart) in New measurement: it saves a new record and makes it this robot's current value.
  • Starting a new measurement, or applying joint mastering with Apply fine mastering, discards the result, since it was measured on the old joint values.
  • Saving is refused if the camera model, robot model, robot serial, tool rotation or F/T sensor model differs from the measurement. Measure again.
  • Cameras are compared by model only, so swapping in another camera of the same model goes unnoticed.

The mark next to the current value tells whether it is in effect.

MarkMeaning
RunningThe running robot stack uses this value for the camera position
Restart neededThe value was changed or cleared and the stack has not restarted yet. It takes effect on restart
(none)Not known yet, for example before the camera position has arrived

If something is wrong with the current value, the card says so first, with the reason below it.

MessageCauseWhat to do
The next restart will fail to startThe value does not fit this robot (a different robot model, or a damaged record)Clear it or use another value
From the next restart the default position is used instead of this valueThe camera or tool rotation differs from the measurement. Start-up is not blocked; the camera falls back to its default position, which follows the tool rotation and F/T sensor. Perception (nvblox and others) stays off until you measure againMeasure again in the current setup
This record was not measured on this robotAnother robot's record, or one naming no robot. The on-robot check is refused; start-up is not affectedCalibrate this robot again, or use another value of this robot
Record file missing · Record damagedThe record it points to cannot be readClear it
Measuring again is recommendedThe robot's joint offsets or kinematics were updated after this measurement. It is judged by file write time, so it can show even when nothing changed. Records in the same case get a Remeasure recommended badgeOnly a warning; nothing is blocked
The camera settings are missing or do not matchThe camera is not set up, or its settings disagreeSet up or save the camera again with Robot settings
If you changed only the gripper or F/T sensor

If you change the gripper or F/T sensor and keep the tool rotation, the old measurement still applies and perception turns on. With the camera on the tool, that value no longer matches the real position. Measure again in the current setup before you use the camera.

ActionWhereWhat it does
ApplyUnapplied resultSaves the unapplied result as a new record and makes it the current value. Takes effect from the next start
Measure againCardGoes to New measurement
Check on robotCardOpens the check below
ClearCardRuns with no calibration (factory default). Takes effect from the next start
ImportCardAdds an exported YAML as a record. An imported value is not used until you choose it
Use this valueA record's ⋯Makes that value this robot's current value. Takes effect from the next start
Rename · ExportA record's ⋯ (current value too)Renames it, or downloads it as a YAML file
RetireA record's ⋯Deletes it after a second confirmation in the menu. A value any robot uses cannot be retired (clear the current value first)

Each record shows its quality (Passed or Below bar), measurement time and residuals.

  • Used by: robot shows when another robot uses it.
  • Measured on another robot (serial) shows when another robot measured it.
  • Records this robot cannot use (another camera or robot model, for example) are dimmed with the reason, not hidden, so you can retire or export them.

Check on the robot​

Open it with Check on robot on the Current value card. The button shows only when the current value is a sound record. Close it with Close at the top right.

  • The top of the window shows the current value and its mark (Running or Restart needed).
  • Next to it are the optical frame and the translation and rotation residuals.
  • A below-bar, missing or damaged record shows its reason too.
  • It works with only the camera (Camera) on, even in setups with collision avoidance (cuMotion) and the environment map (nvblox) off.

Pick a point on the board in the camera view and move the TCP over it.

  1. Click a point on the board. The robot prepares to move the TCP 2 cm above it.
  2. Press Move. It moves after the countdown ("Moving in N s"). Cancel during the countdown means no move.
  3. Check that the TCP sits right above the point. The sideways offset is the calibration error; use the board squares as a ruler.
  4. Raise the tool a few cm before the next point. With the tool close to the board, the path to the next point runs too close to the board and the move is refused.
  5. Repeat at 2 or 3 diagonal corners.
  • Guides and refusal reasons are in the screen language. Raw text from the planner or camera, if any, follows the reason as is under Details.
  • It works only for a value that passed and has its board spec recorded. Otherwise the button is locked with the reason.
  • After a save or Use this value, the stack must restart.
  • If the connection drops mid-move, the cancel may not have arrived. If the robot moves, press the E-Stop.