A central function in the robot is detection of ripe crops. For successful operation, the 3D location of each crop must be determined with high accuracy. The chosen solution is based on an RGB-D camera that simultaneously reports color and depth information. Using this camera and a custom built LED-based flash-light illumination system, RGB images of the plant are acquired from both overview distance and close range. In order to facilitate high frame-rate operation, a straight forward shape- and color-based detection algorithm was implemented using Mstar. The algorithm scans each acquired image for regions matching the target color thresholds. Detected regions are refined by removing detections exceeding predefined minimum/maximum sizes. To further remove misdetections additional shape parameters are calculated. Finally, depth information from the camera is used to compute the volume of the detected regions. This information is then used to further prune false detections, avoid non-harvestable crop clusters, and define harvest priorities. The exact 3D location of the point of mass is calculated using the depth information extracted from the detected region and a standard procedure of pixel-to-world transformation of the region. Given the subsets of regions that are classified as vegetable to be harvested, a methodology for harvesting sequencing was defined. The robot arm then approaches the target by visual-servo control that keeps the target in the middle of the images until it is reached.