A Pick and Place Task Using a Calibrated Camera

Once you have calibrated a camera, you can use it to select 3D points using pairs of 2D images. Each 2D image point corresponds to a line in the 3D world; by intersecting the lines resulting from two images of the same feature, we can obtain a point in the 3D world.

One popular application of this procedure is "pick and place" tasks. A robot is instructed to move to a point in the world, pick up an object there, move to another point in the world, and place the object at the new location. This is widely used in assembly lines to move parts. In our environment, the specific task is to move a rocket (the red object in the lower half of the image below) from a truck to a launchpad (the white cylinder in the upper half of the image below).

From both of the camera stations that you selected during camera calibration, you will be asked to identify a feature (the white part of the rocket). You will then have to identify the launch pad feature from both stations.

  1. Identifying the pick feature on the rocket
  2. Identifying the place feature on the launchpad
  3. Computing the 3D points
  4. Moving the robot through the path
  5. Possible reasons for failure
Up to Camera Calibration

Back to Computing the Calibration Matrix