基于级联融合网络的密集型抓取位姿检测*

doi:10.16731/j.cnki.1671-3133.2024.07.016

摘要/Abstract

摘要： 针对机械臂在密集场景下对未知物体进行平面抓取时检测精度低、耗时长的问题,提出了一种新的抓取网络结构,称为级联融合抓取检测(Cascade Fusion Grasp Detection,CFGD)网络,用于预测密集场景下物体的抓取位姿。在提出的用于提取初始高分辨率特征的主干和几个块的基础上,引入几个级联阶段来生成CFGD网络中的多尺度特征。每个阶段包括1个用于特征提取的子主干和1个用于特征集成的极其轻量级的过渡块,该设计使得整个主干参数比例大,特征融合更深入、更有效。所提算法在康奈尔抓取数据集、提花抓取数据集和自定义数据集上在检测精度与速度上较现有算法有明显提升。在真实抓取场景下,单目标场景下的抓取成功率为98.6 %,在密集场景下的抓取成功率为94.6 %。试验结果表明,所提算法以较高的准确率预测和抓取了密集场景下的未知物体。

关键词: 机械臂, 平面抓取, 位姿估计, 级联融合, 机器视觉

Abstract: In order to solve the problem of low detection accuracy and long time when robot arms grasp unknown objects in dense scenes,a new grasping network structure called Cascade Fusion Grasp Detection (CFGD) was proposed,which was used to predict the grasping pose of objects in dense scenes. Based on the proposed backbone and several blocks for extracting initial high-resolution features,several cascade stages were also introduced to generate multi-scale features in CFGD network. A sub-backbone for feature extraction and an extremely lightweight transition block for feature integration were included at each stage.Such a design allowed for a larger proportion of parameters throughout the backbone and a deeper and more efficient feature fusion. Compared with the existing algorithms,the proposed algorithm has significantly improved the detection accuracy and speed on Cornell grasp data set,Jacquard grasp data set and custom data set. In the real grasping scene,the grasping success rate is 98.6 % in the single target scene and 94.6 % in the dense scene. Experimental results showed that the proposed algorithm can predict and grasp unknown objects in dense scenes with high accuracy.

Key words: mechanical arm, plane grasping, pose estimation, cascade fusion, machine vision

中图分类号:

TP242

邓鹏, 唐文涛, 黄开明. 基于级联融合网络的密集型抓取位姿检测^*[J]. 现代制造工程, 2024, 526(7): 126-134.

DENG Peng, TANG Wentao, HUANG Kaiming. Intensive grasp pose detection based on cascaded fusion network[J]. Modern Manufacturing Engineering, 2024, 526(7): 126-134.

参考文献

[1] 张健,刘韦,宋丽.基于力传感器重力补偿的机器人主动柔顺插孔控制方法[J].制造业自动化,2022,44(3):197-200.
[2] 刘许亮.智能制造机器人多手臂自适应协同控制方法研究 [J].制造业自动化,2022,44(1):110-113.
[3] LIN Y,SUN Y.Task-based grasp quality measures for grasp synthesis[C]//2015 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).Hamburg:IEEE,2015:485-490.
[4] LIN Y,SUN Y.Task-oriented grasp planning based on disturbance distribution[J].Springer,2016,114:577-592.
[5] KOKIC M,STORK J A,HAUSTEIN J A,et al.Affordance detection for task-specific grasping using deep learning[C]//2017 IEEE-RAS 17th International Conference on Humanoid Robotics.Humanoids:IEEE,2017:91-98.
[6] DETRY R,PAPON J,MATTHIES L.Task-oriented grasping with semantic and geometric scene understanding[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).Vancouver:IEEE,2017:3266-3273.
[7] FANG K,ZHU Y,GARG A,et al.Learning task-oriented grasp for tool manipulation from simulated self-supervision[J].The International Journal of Robotics Research,2020,39(2/3):202-216.
[8] BOHG J,MORALES A,ASFOUR A,et al.Date-driven grasp synthesis-a survey[J].IEEE Trans.Robot,2014,30(2):289-309.
[9] VARLEY J,DECHANT C,RICHARDSON A,et al.Shape completion enabled robotic grasping[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).Humanoids:IEEE,2017:2442-2447.
[10] GUO D,SUN F,LIU H,et al.A hybrid deep architecture for robotic grasp detection[C]//2017 IEEE International Conference on Robotics and Automation(ICRA).Singapore:IEEE,2017:1609-1614.
[11] MAHLER J,LIANG J,NIYAZ S,et al.Dex-Net 2.0:Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics[J].RSS,2017,29(22):12-26.
[12] LEVINE S,PASTOR P,KRIZHEVSKY J,et al.Learning hand-eye coordination for robotic grasping with deep learning and largescale data collection[J].The International Journal of Robotics Research,2018,37(4/5):421-436.
[13] MILLER A T,KNOOP S,CHRISTENSEN H I,et al.Automatic grasp planning using shape primitives[C]//2003 IEEE International Conference on Robotics and Automation. Taipei:IEEE,2003:1824-1829.
[14] MAHLER J,PATIL S,KEHOE B,et al.GP-GPIS-OPT:grasp planning with shape uncertainty using gaussian process implicit surfaces and sequential convex programming[C]//2015 IEEE International Conference on Robotics and Automation(ICRA).Stockholm:IEEE,2015:4919-4926.
[15] GOLDFEDER C,ALLEN P K,LACKNER C,et al.Grasp planning via decomposition trees.[C]//2007 IEEE International Conference on Robotics and Automation.Torun:IEEE,2007:4679-4684.
[16] SAXENA A,DRIEMEYER J,NG A Y.Robotic grasping of novel objects using vision[J].Int.J.Robot.Res,2008,27(2):157-173.
[17] LE Q V,KAMM D,KARA A F,et al.Learning to grasp objects with multiple contact points[C]//2010 IEEE International Conference on Robotics and Automation.Anchorage:IEEE,2010:5062-5069.
[18] JIANG Y,MOSESON S,SAXENA A.Efficient grasping from RGBD images:learning using a new rectangle representa-tion[C]//2011 IEEE International Conference on Robotics and Automation.Shanghai:IEEE,2011:3304-3311.
[19] GUO D,SUN F,FANG B,et al.Robotic grasp using visual and tactile sensing[J].Information Sciences,2017,417:274-286.
[20] ZHOU X,LAN X,ZHANG H,et al.Fully convolutional grasp detection network with oriented anchor box[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).Madrid:IEEE,2018:7223-7230.
[21] REN S,HE K,GIRSHICK R,et al.Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(6):1137-1149.
[22] ASIF U,TANG J,HARRER S.Densely Supervised Grasp Detector(DSGD)[C]//AAAI Conference on Artificial Intelligence.Hawaii:IEEE,2019:8085-8093.
[23] HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:770-778.
[24] LIU Z,MAO H,WU C Y,et al.A ConvNet for the 2020s[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans:IEEE,2022:11976-11986.
[25] YU W,LUO M,ZHOU P,et al.Metaformer is actually what you need for vision[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans:IEEE,2022:10819-10829.
[26] DONG X,BAO J,CHEN D,et al.Cswin transformer:a general vision transformer backbone with cross-shaped windows[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans:IEEE,2022:12124-12134.
[27] HE K,GKIOXARI G,DOLLÁR P,et al.Mask R-CNN[C]//IEEE International Conference on Computer Vision.Venice:IEEE,2017:2961-2969.
[28] LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//IEEE International Conference on Computer Vision.Venice:IEEE,2017:2980-2988.
[29] TIAN Z,SHEN C,CHEN H,et al.Fcos:fully convolutional one-stage object detection[C]//IEEE/CVF International Conference on Computer Vision.Seoul:IEEE,2019:9627-9636.
[30] TAN M,PANG R,LE Q V.Efficientdet:scalable and efficient object detection[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.Venice:IEEE,2020:10781-10790.
[31] JIANG Y,MOSESON S,SAXENA A.Efficient grasping from RGBD images:Learning using a new rectangle representation[C]//IEEE International Conference on Robotics and Automation.Shanghai:IEEE,2011:3304-3311.
[32] MORRISON D,CORKE P,LEITNER J.Learning robust,real-time,reactive robotic grasp[J].The International Journal of Robotics Research,2020,39(2/3):183-201.
[33] DEPIERRE A,DELLANDREA E,CHEN L.Jacquard:a large scale dataset for robotic grasp detection[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems.Madrid:IEEE,2018:3511-3516.
[34] JEFFREY M,KEN G.Learning deep policies for robot bin picking by simulating robust grasping sequences[C]//Conference on Robot Learning.California:PMLR,2017:515-524.
[35] PAS T A,GUALTIERI M,SAENKO K,et al.Grasp pose detection in point clouds[J].The International Journal of Robotics Research,2017,36(13/14):1455-1473.
[36] WU B,AKINOLA I,GUPTA A,et al.Generative Attention Learning:A “GenerAL” Framework for High-performance Multi-fingered Grasp in Clutter[J].Autonomous Robots,2020,44(6):971-990.

基于级联融合网络的密集型抓取位姿检测^*

Intensive grasp pose detection based on cascaded fusion network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	贾英霞, 王东辉. 基于自适应神经网络的工业机器人双臂协同鲁棒控制^*[J]. 现代制造工程, 2024, 525(6): 61-68.
[2]	杨杰杰, 秦志杰. 基于非奇异快速积分终端滑模的机械臂鲁棒控制^*[J]. 现代制造工程, 2024, 523(4): 73-79.
[3]	金宇杰, 龚堰珏, 赵罘. 基于模拟退火量子遗传算法的焊接机器人轨迹规划^*[J]. 现代制造工程, 2024, 520(1): 33-38.
[4]	黄广伟, 陈军杰. 受限空间机械臂避障路径与能耗最优研究^*[J]. 现代制造工程, 2024, 520(1): 39-44.
[5]	潘俊杰, 夏如艇. 基于机器视觉的E型卡簧尺寸检测方法[J]. 现代制造工程, 2024, 520(1): 95-102.
[6]	李敏. 变论域模糊补偿的机械臂积分终端滑模控制[J]. 现代制造工程, 2023, 514(7): 37-44.
[7]	王笑;王磊. 基于新几何子问题的6R机械臂逆运动学分析[J]. 现代制造工程, 2023, 514(7): 45-53.
[8]	任文硕;邢关生. 全向移动机械臂的混合视觉伺服与参数优化[J]. 现代制造工程, 2023, 511(4): 59-68.
[9]	杨泽青;张明轩;陈英姝;平恩旭;方勇;吕雅丽;高岩. 基于机器视觉的表面缺陷检测方法研究进展[J]. 现代制造工程, 2023, 511(4): 143-156.
[10]	惠翔禹;金隼;顾同成;何心;李冰;高宇星. 装载机动臂孔组同轴度视觉检测系统设计[J]. 现代制造工程, 2023, 510(3): 112-118.
[11]	钱乐平;邵中魁;沈小丽. 基于3D机器视觉的离心泵叶轮智能无序上料技术应用研究[J]. 现代制造工程, 2023, 509(2): 27-35.
[12]	刘阳，徐巧玉，陈翰文，张正. 凿岩机械臂轨迹规划及电液跟踪控制研究[J]. 现代制造工程, 2023, 519(12): 61-67.
[13]	李明;罗江涛;雒瑞民;赵昊宁;荣学文. 液压机械臂复合连杆机构优化设计[J]. 现代制造工程, 2023, 518(11): 129-136.
[14]	聂鹏，王家明，李海伟，纪俐. 基于机器视觉的铆接高度差检测方法研究[J]. 现代制造工程, 2022, 504(9): 107-112.
[15]	谭丹丹. 基于自适应变阻抗的工业机器人双机械臂控制[J]. 现代制造工程, 2022, 503(8): 39-45.