基于机器视觉的苹果园果实识别研究综述

金保华; 殷长魁; 张卫正; 张伟伟

doi:10.3969/j.issn.2096-1553.2019.02.010

基于机器视觉的苹果园果实识别研究综述

郑州轻工业大学计算机与通信工程学院, 河南郑州 450002

作者简介: 金保华(1966-),男,河南省郑州市人,郑州轻工业大学教授,主要研究方向为人工智能.;

基金项目: 国家自然科学基金项目（61403349）；河南省科技攻关项目（182102110399）；河南省高等学校重点科研项目（18A210025）
中图分类号: TN959.4;S758.4;TS255.1

Review on apple garden fruit recognition based on machine vision

College of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450002, China
Received Date: 2018-11-08

CLC number: TN959.4;S758.4;TS255.1

摘要: 从基于颜色阈值、形状和纹理的果实识别，三维果实形态识别，夜间果实识别，基于机器学习的果实识别，阴影和遮挡影响下的果实识别5个方面，对基于机器视觉的苹果园果实识别研究现状进行了综述，认为上述研究所涉及的算法较为复杂，功能也很强大.但鉴于视觉理论、图像处理技术和硬件条件等限制，以及苹果园复杂多变的环境，基于机器视觉的果实识别目前尚无理想的方法，未来的研究重点应包括：1）加强更有效的图像增强、图像分割和特征提取等算法的研究，有效解决果实重叠、遮挡、颜色和光线变化的影响；完善白天和夜间果园现场作业的识别算法，建成全天候作业采摘机器人.2）加强基于自监督学习的果实识别的研究，以增加模型接收的反馈信息和模型表征的复杂的适用任务类型，减少任务中涉及的人类手工劳动比重，提高自动化程度.3）加强图像的自动获取与果实识别的研究，结合计算机视觉与近红外、激光雷达等检测技术，集成多模态的图像和非图像信息进行果实识别，提高处理速度和实时性，以及识别的准确度及系统的稳健性，为苹果自动采摘、果园的精准管理提供借鉴.
- 机器视觉 /
- 苹果园 /
- 果实识别 /
- 图像处理 /
- 机器学习
Abstract: The current situation of fruit recognition based on machine vision was reviewed from fruit recognition based on color threshold, shape and texture, three-dimensional fruit shape recognition, nocturnal fruit recognition, fruit recognition based on machine learning, fruit recognition under the influence of shadow and occlusion. It's thought that the algorithms involved in the above research were more complicated and features were very powerful. However, in view of the limitations of visual theory, image processing technology and hardware conditions, as well as the complex and varied environment of apple garden, there was no more ideal technology for machine vision-based fruit recognition, and it needed to be improved. Future research focuses include:1) Strengthening more effective algorithms for image enhancement, image segmentation, and feature extraction to effectively address the effects of fruit overlap, occlusion, color, and light changes; and improving the identification algorithms for day and night orchard field operations for the construction of an all-weather operation picking robot. 2) Strengthening the research on fruit recognition based on self-supervised learning to increase the feedback information received by the model and the complex applicable task types of model representation, reduce the proportion of human manual labor involved in the task, and improve the degree of automation. 3) Strengthening the research of automatic image acquisition and fruit recognition, combined with computer vision and near-infrared, laser radar and other detection technologies, integrating multi-modal image and non-image information for fruit recognition, improving processing speed and real-time, and identifing accuracy and system robustness to provide reference for apple's automatic picking and precise management of orchard.
- machine vision /
- apple garden /
- fruit recognition /
- image processing /
- machine learning
1. [1]
  段峰,王耀南,雷晓峰,等.机器视觉技术及其应用综述[J].自动化博览,2002,19(3):59.
2. [2]
  孙碧亮.基于机器视觉的检测算法研究及其在工业领域的应用[D].武汉:华中科技大学,2006.
3. [3]
  徐铭辰,牛媛媛,余永昌.果蔬采摘机器人研究综述[J].安徽农业科学,2014(31):11024.
4. [4]
  李素云,唐先进.苹果采摘机器人的研究现状、进展与分析[J].装备制造技术,2016(1):185.
5. [5]
  李卓,杨子敬,郝建唯,等.苹果幼果图像的分割与识别算法研究[J].沈阳理工大学学报,2015,34(1):34.
6. [6]
  沈甜.苹果采摘机器人重叠果实快速动态识别及定位研究[D].镇江:江苏大学,2016.
7. [7]
  王晋.自然环境下苹果采摘机器人视觉系统的关键技术研究[D].秦皇岛:燕山大学,2014.
8. [8]
  陈鸥.苹果采摘机器人视觉系统识别基础方法的研究[D].秦皇岛:燕山大学,2016.
9. [9]
  ZHOU R,DAMEROW L,SUN Y,et al.Using colour features of cv "Gala" apple fruits in an orchard in image processing to predict yield[J].Precision Agriculture,2012,13(5):568.
10. [10]
  钱建平,杨信廷,吴晓明,等.自然场景下基于混合颜色空间的成熟期苹果识别方法[J].农业工程学报,2012,28(17):137.
11. [11]
  李莎,钱建平,赵春江,等.不同颜色模型识别成熟苹果图像的比较分析[J].计算机工程与设计,2015,36(6):1545.
12. [12]
  SI Y,LIU G,FENG J.Location of apples in trees using stereoscopic vision[J].Computers and Electronics in Agriculture,2015,112:68.
13. [13]
  PATEL H N,JAIN R K,JOSHI M V.Automatic segmentation and yield measurement of fruit using shape analysis[J].International Journal of Computer Applications,2012,45(7):19.
14. [14]
  崔淑娟,李健.基于色差信息的成熟苹果识别[J].西北大学学报(自然科学版),2011,41(6):993.
15. [15]
  吕继东,赵德安,姬伟.苹果采摘机器人目标果实快速跟踪识别方法[J].农业机械学报,2014,45(1):65.
16. [16]
  LV J,XU L.Method to acquire regions of fruit,branch and leaf from image of red apple in orchard[J].Modern Physics Letters B,2017,31(19):1.
17. [17]
  LV J,WANG F,MA Z,et al.Yellow apple recognition method under natural environment[C]//Intelligent Human-Machine Systems and Cybernetics (IHMSC),20157th International Conference.Piscataway:IEEE,2015:46.
18. [18]
  LI B,LONG Y,SONG H.Detection of green apples in natural scenes based on saliency theory and Gaussian curve fitting[J].International Journal of Agricultural and Biological Engineering,2018,11(1):192.
19. [19]
  冯娟,刘刚,司永胜,等.基于激光扫描三维图像的树上苹果识别算法[J].农业机械学报,2013,44(4):217.
20. [20]
  孙贤刚,伍锡如,党选举,等.基于视觉检测的苹果采摘机器人系统设计与实现[J].农机化研究,2016(9):151.
21. [21]
  张宾,宿敬肖,张微微,等.基于激光视觉的智能识别苹果采摘机器人设计[J].农机化研究,2016,38(7):60.
22. [22]
  ZHANG B,HUANG W,WANG C,et al.Computer vision recognition of stem and calyx in apples using near-infrared linear-array structured light and 3D reconstruction[J].Biosystems Engineering,2015,139:25.
23. [23]
  TAO Y,ZHOU J.Automatic apple recognition based on the fusion of color and 3D feature for robotic fruit picking[J].Computers and Electronics in Agriculture,2017,142:388.
24. [24]
  李强.苹果采摘机器人视觉系统研究[D].兰州:兰州理工大学,2017.
25. [25]
  赵德安,刘晓洋,陈玉,等.苹果采摘机器人夜间识别方法[J].农业机械学报,2015,46(3):15.
26. [26]
  刘晓洋,赵德安,陈玉,等.夜间低照度条件下苹果采摘机器人的图像识别[J].华中科技大学学报(自然科学版),2015,43(S1):525.
27. [27]
  姬伟,吕兴琴,赵德安,等.苹果采摘机器人夜间图像边缘保持的Retinex增强算法[J].农业工程学报,2016,32(6):189.
28. [28]
  JI W,QIAN Z J,XU B,et al.A nighttime image enhancement method based on Retinex and guided filter for object recognition of apple harvesting robot[J].International Journal of Advanced Robotic Systems,2018,15(1):1.
29. [29]
  吕兴琴.苹果采摘机器人夜间图像降噪和增强技术研究[D].镇江:江苏大学,2016.
30. [30]
  贾伟宽,赵德安,阮承治,等.苹果采摘机器人夜间图像降噪算法[J].农业工程学报,2015,31(10):219.
31. [31]
  贾伟宽.基于智能优化的苹果采摘机器人目标识别研究[D].镇江:江苏大学,2016.
32. [32]
  RUAN C,ZHAO D,JIA W,et al.Night vision image de-noising of apple harvesting robots based on the wavelet fuzzy threshold[J].International Journal of Advanced Robotic Systems,2015,12(12):169.
33. [33]
  JI W,ZHAO D,CHENG F,et al.Automatic recognition vision system guided for apple harvesting robot[J].Computers & Electrical Engineering,2012,38(5):1186.
34. [34]
  陈珂,许林峰,柯文德.基于HOG与支持向量机的成熟苹果自动识别[J].江苏农业科学,2017,45(7):211.
35. [35]
  夏雪,周国民,丘耘,等.自然环境下果实作业机器人幼果期苹果侦测方法[J].中国农业科技导报,2018,20(5):64.
36. [36]
  贾伟宽,赵德安,刘晓洋,等.机器人采摘苹果果实的K-means和GA-RBF-LMS神经网络识别[J].农业工程学报,2015,31(18):175.
37. [37]
  马晓丹,刘刚,周薇,等.基于量子遗传模糊神经网络的苹果果实识别[J].农业机械学报,2013,44(12):227.
38. [38]
  麦春艳,郑立华,肖昌一,等.自然光照条件下苹果识别方法对比研究[J].中国农业大学学报,2016,21(11):43.
39. [39]
  XU L,LV J.Recognition method for apple fruit based on SUSAN and PCNN[J].Multimedia Tools and Applications,2018,77(6):7205.
40. [40]
  王慧,季雪.图像识别处理技术在农业工程中的应用[J].传感器与微系统,2018(6):158.
41. [41]
  BARGOTI S,UNDERWOOD J P.Image segmentation for fruit detection and yield estimation in apple orchards[J].Journal of Field Robotics,2016,34(6):1039.
42. [42]
  王丹丹.重叠及遮挡影响下的苹果目标识别与定位方法研究[D].杨凌:西北农林科技大学,2016.
43. [43]
  JI W,MENG X,TAO Y,et al.Fast segmentation of colour apple image under all-weather natural conditions for vision recognition of picking robots[J].International Journal of Advanced Robotic Systems,2016,13(1):24.
44. [44]
  SUN S,WU Q,JIAO L,et al.Recognition of green apples based on fuzzy set theory and manifold ranking algorithm[J].Optik,2018,165:395.
45. [45]
  CHEN S W,SHIVAKUMAR S S,DCUNHA S,et al.Counting apples and oranges with deep learning:a data-driven approach[J].IEEE Robotics and Automation Letters,2017,2(2):781.
46. [46]
  廖崴,郑立华,李民赞,等.基于随机森林算法的自然光照条件下绿色苹果识别[J].农业机械学报,2017,48(S1):86.
47. [47]
  李娜,陈宁.自然场景下苹果采摘机器人视觉系统研究[J].计算机技术与发展,2018(11):1.
48. [48]
  宋怀波,张卫园,张欣欣,等.基于模糊集理论的苹果表面阴影去除方法[J].农业工程学报,2014,30(3):135.
49. [49]
  WANG D,SONG H,TIE Z,et al.Recognition and localization of occluded apples using K-means clustering algorithm and convex hull theory:a comparison[J].Multimedia Tools and Applications,2016,75(6):3177.
50. [50]
  MENG J,WANG S.The recognition of overlapping apple fruits based on boundary curvature estimation[C]//2015 Sixth International Conference on Intelligent Systems Design and Engineering Applications (ISDEA).Piscataway:IEEE,2015:874.
51. [51]
  孙飒爽,吴倩,谭建昌,等.枝条遮挡下单个苹果目标识别与重建的研究[J].西北农林科技大学学报(自然科学版),2017(11):1.
52. [52]
  段延娥,李道亮,李振波,等.基于计算机视觉的水产动物视觉特征测量研究综述[J].农业工程学报,2015,31(15):1.
1. [1]
  张建栋 , 杨忠泮 , 吴恋恋 , 徐大勇 , 朱萍 , 张雯晶 , 堵劲松 . 基于高光谱成像及机器学习的烟叶糖料液施加量判别模型. 轻工学报, 2024, 39(5): 86-94. doi: 10.12187/2024.05.010
2. [2]
  卢晓波 , 徐海 , 朱俊召 , 张宇 , 谭健 , 高冠男 , 胡军华 , 林龙 . 基于机器视觉的加热卷烟烟支端部质量检测系统设计. 轻工学报, 2024, 0(0): -.
3. [3]
  张伟伟 , 姬远鹏 , 元春波 , 王君婷 , 齐晓任 , 张卫正 , 李萌 , 饶智 . 基于改进Mask R-CNN模型的粘连烟丝识别方法. 轻工学报, 2024, 39(5): 78-85. doi: 10.12187/2024.05.009
4. [4]
  贾尚羲 , 张怡雪 , 石盼盼 , 王昱 , 李可 . 不同时长超声波处理对鹰嘴豆分离蛋白乳化液稳定性的影响. 轻工学报, 2024, 39(5): 40-49. doi: 10.12187/2024.05.005
5. [5]
  倪众 , 楚鹏飞 , 林颖 , 刘玉欣 . 低温长时间热处理过程中氧化和加热对海参体壁胶原纤维结构的影响. 轻工学报, 2024, 0(0): -.

点击查看大图

计量

PDF下载量: 107
文章访问数: 4368
引证文献数: 0

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

基于机器视觉的苹果园果实识别研究综述

作者简介: 金保华(1966-),男,河南省郑州市人,郑州轻工业大学教授,主要研究方向为人工智能.;

Review on apple garden fruit recognition based on machine vision

计量

通讯作者: 陈斌, bchen63@163.com

基于机器视觉的苹果园果实识别研究综述

作者简介:金保华(1966-),男,河南省郑州市人,郑州轻工业大学教授,主要研究方向为人工智能.

English Abstract

Review on apple garden fruit recognition based on machine vision

目录

基于机器视觉的苹果园果实识别研究综述

作者简介: 金保华(1966-),男,河南省郑州市人,郑州轻工业大学教授,主要研究方向为人工智能.;

Review on apple garden fruit recognition based on machine vision

计量

文章相关

通讯作者: 陈斌, bchen63@163.com

基于机器视觉的苹果园果实识别研究综述

作者简介:金保华(1966-),男,河南省郑州市人,郑州轻工业大学教授,主要研究方向为人工智能.

English Abstract

Review on apple garden fruit recognition based on machine vision

目录