IDEAS home Printed from https://ideas.repec.org/a/gam/jagris/v12y2022i8p1170-d881508.html
   My bibliography  Save this article

An Algorithm for Calculating Apple Picking Direction Based on 3D Vision

Author

Listed:
  • Ruilong Gao

    (College of Metrology and Measurement Engineering, China Jiliang University, Hangzhou 310018, China)

  • Qiaojun Zhou

    (Engineering Training Center, China Jiliang University, Hangzhou 310018, China
    Faculty of Mechanical Engineering & Automation, Zhejiang Sci-Tech University, Hangzhou 310018, China)

  • Songxiao Cao

    (College of Metrology and Measurement Engineering, China Jiliang University, Hangzhou 310018, China)

  • Qing Jiang

    (College of Metrology and Measurement Engineering, China Jiliang University, Hangzhou 310018, China)

Abstract

Research into autonomous (robotic) apple picking has not yet resolved the problem of finding the optimal picking orientation. Robotic picking efficiency, in terms of picking all available apples without loss or damage, remains low. This paper proposes a method of determining the optimal picking orientation relative to the target fruit and adjacent branches from the point cloud of the apple and the surrounding space. The picking mechanism is then able to avoid branches and accurately grasp the target apple in order to pick it. The apple is first identified by the YOLOv3 target detection algorithm, and a point cloud of the fruit and the space surrounding it is obtained. The random sample consensus algorithm RANSAC is used for sphere fitting, and the fruit is idealized as a sphere. RANSAC also idealizes the branch as a line that is fitted to the branch bearing the target apple in the point cloud around it. The distance between the line of the branch and the fruit centroid is constrained in fitting to ensure identification of the branch/line closest to the apple/sphere. The best apple picking orientation is determined from the positional relationship between the straight branch/line and the center of the apple/sphere. The performance of the algorithm was evaluated using apples with various orientations on growing trees. The average angle error between the calculated picking direction vector and the expected direction vector was 11.81°, and the standard deviation was 13.65°; 62.658% of the determinations erred by ≤10°, and 85.021% erred by ≤20°. The average time for estimating the orientation of an apple was 0.543 s. The accuracy and speed of the algorithm enabled the robotic picker to operate at a speed that matches that of a human apple picker.

Suggested Citation

  • Ruilong Gao & Qiaojun Zhou & Songxiao Cao & Qing Jiang, 2022. "An Algorithm for Calculating Apple Picking Direction Based on 3D Vision," Agriculture, MDPI, vol. 12(8), pages 1-19, August.
    • Handle: RePEc:gam:jagris:v:12:y:2022:i:8:p:1170-:d:881508
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2077-0472/12/8/1170/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2077-0472/12/8/1170/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Sajad Sabzi & Yousef Abbaspour-Gilandeh & Jose Luis Hernandez-Hernandez & Farzad Azadshahraki & Rouhollah Karimzadeh, 2019. "The Use of the Combination of Texture, Color and Intensity Transformation Features for Segmentation in the Outdoors with Emphasis on Video Processing," Agriculture, MDPI, vol. 9(5), pages 1-14, May.
    2. Pan Fan & Guodong Lang & Pengju Guo & Zhijie Liu & Fuzeng Yang & Bin Yan & Xiaoyan Lei, 2021. "Multi-Feature Patch-Based Segmentation Technique in the Gray-Centered RGB Color Space for Improved Apple Target Recognition," Agriculture, MDPI, vol. 11(3), pages 1-18, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pan Fan & Guodong Lang & Pengju Guo & Zhijie Liu & Fuzeng Yang & Bin Yan & Xiaoyan Lei, 2021. "Multi-Feature Patch-Based Segmentation Technique in the Gray-Centered RGB Color Space for Improved Apple Target Recognition," Agriculture, MDPI, vol. 11(3), pages 1-18, March.
    2. Huawei Yang & Yinzeng Liu & Shaowei Wang & Huixing Qu & Ning Li & Jie Wu & Yinfa Yan & Hongjian Zhang & Jinxing Wang & Jianfeng Qiu, 2023. "Improved Apple Fruit Target Recognition Method Based on YOLOv7 Model," Agriculture, MDPI, vol. 13(7), pages 1-21, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jagris:v:12:y:2022:i:8:p:1170-:d:881508. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.