Automatic Fruit Harvesting Device Based on Visual Feedback Control

With aging populations, and people′s demand for high-quality or high-unit-price fruits and vegetables, the corresponding development of automatic fruit harvesting has attracted significant attention. According to the required operating functions, based on the fruit planting environment and harvesting requirements, this study designed a harvesting mechanism to independently drive a gripper and scissor for individual tasks, which corresponded to forward or reverse rotation using a single motor. The study utilized a robotic arm in combination with the harvesting mechanism, supported by a single machine vision component, to recognize fruits by deep-learning neural networks based on a YOLOv3-tiny algorithm. The study completed the coordinate positioning of the fruit, using a two-dimensional visual sensing method (TVSM), which was used to achieve image depth measurement. Finally, impedance control, based on visual feedback from YOLOv3-tiny and the TVSM, was used to grip the fruits according to their size and rigidity, so as to avoid the fruits being gripped by excessive force; therefore, the apple harvesting task was completed with a 3.6 N contact force for an apple with a weight of 235 g and a diameter of 80 mm. During the cutting process, the contact point of the metal scissors of the motor-driven mechanism provided a shear force of 9.9 N, which was significantly smaller than the simulation result of 94 N using ADAMS and MATLAB software, even though the scissors were slightly blunt after many cuts. This study established an automatic fruit harvesting device based on visual feedback control, which can provide an automatic and convenient fruit harvest by reducing harvesting manpower.