对于机械臂视觉抓取功能来说,比较传统的做法:由于之前的物体识别模块已经确定了这片点云对应的物体,如果我们有物体的3D模型,便可以直接用ICP算法将这片点云与物体的3D模型对齐。既然知道3D模型的位姿那就好办了。之后就是各种抓取姿态生成、力封闭之类的东西了。其他方法:利用RCNN在图像中定位了物体的大概位置,然后根据一堆点云做抓取,而不用知道物体的3D模型。   1、Using Geometry to Detect Grasp Poses in 3D Point Clouds 上面这个是ICRA 2015中的一篇论文,他们利用SVM分类器,直接对3D点云进行训练,判断其可抓取方向:感觉效果并不是特别好,需要较为完整的点云,同时训练库对结果影响也很大。   2、High precision grasp pose detection in dense clutter 这篇论文采用卷积神经网络训练一小片点云的可抓取性,上图按顺序依次是:彩色图片、点云图片、随机生成的抓取姿态、CNN判断可行的抓取姿态。据作者说抓取成功率相比于第一种方法提升了20%。   这两种方法,作者都已经将程序开源出来了,第一种方法是agile_grasp,在ROS中编译安装很简单,第二种方法是gpd_ros,这个包安装编译就有很多坑,首先gpd_ros依赖于gpd库,而作者提供的gpd2.0.0版本是基于c++14和PCL1.9的版本写的,所以对于ROS kinetic用户来说配置起来错误很多。最终要想编译gpd_ros成功,需要以下几个步骤:  
  1. 下载编译PCL1.9
  2. 修改cmake版本
  3. 下载编译gpd
  4. 添加opencv依赖
  5. 编译gpd_ros
 

1. ROS环境下载编译PCL1.9

  https://blog.csdn.net/czychen1997/article/details/107306674/?utm_medium=distribute.pc_relevant.none-task-blog-utm_term-3&spm=1001.2101.3001.4242   现在在官网好像没法下载源码了,要到github下载才行:https://github.com/PointCloudLibrary/pcl/releases/tag/pcl-1.9.0 ,下载源码之后,放到你想要的安装目录下,然后安装依赖  
$ sudo apt-get update
$ sudo apt-get install git build-essential linux-libc-dev
$ sudo apt-get install cmake cmake-gui 
$ sudo apt-get install libusb-1.0-0-dev libusb-dev libudev-dev
$ sudo apt-get install mpi-default-dev openmpi-bin openmpi-common  
$ sudo apt-get install libflann1.8 libflann-dev
$ sudo apt-get install libeigen3-dev
$ sudo apt-get install libboost-all-dev
$ sudo apt-get install libvtk5.10-qt4 libvtk5.10 libvtk5-dev
$ sudo apt-get install libqhull* libgtest-dev
$ sudo apt-get install freeglut3-dev pkg-config
$ sudo apt-get install libxmu-dev libxi-dev 
$ sudo apt-get install mono-complete
$ sudo apt-get install qt-sdk openjdk-8-jdk openjdk-8-jre
  然后进入目录编译:  
$ cd pcl_1.9
$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=None -DCMAKE_INSTALL_PREFIX=/usr \ -DBUILD_GPU=ON-DBUILD_apps=ON -DBUILD_examples=ON \ -DCMAKE_INSTALL_PREFIX=/usr ..
$ make -j
  编译过程中,由于系统的VTK版本比较低,可能会出现以下warning像下面这样的,不用管:  
In file included from /home/redwall/catkin_ws/pcl-1.9.0/visualization/include/pcl/visualization/impl/pcl_visualizer.hpp:48:0,
                 from /home/redwall/catkin_ws/pcl-1.9.0/visualization/include/pcl/visualization/pcl_visualizer.h:2352,
                 from /home/redwall/catkin_ws/pcl-1.9.0/examples/segmentation/example_cpc_segmentation.cpp:49:
/usr/include/vtk-6.2/vtkMath.h:672:3: warning: multi-line comment [-Wcomment]
   // a & b \\
   ^
/usr/include/vtk-6.2/vtkMath.h:676:3: warning: multi-line comment [-Wcomment]
   // 1 & 0 \\
   ^
/usr/include/vtk-6.2/vtkMath.h:679:3: warning: multi-line comment [-Wcomment]
   // a & b \\
  等到编译完成之后,虽然有很多警告,只要能编译通过,就没关系,安装即可:  
$ sudo make install
  此时PCL1.9版本和之前ROS集成的PCL版本是可以共用的,通过在CMakeLists中即可指定特定版本。   期间出现警告可以不管,但出现error时会报错停止,下面是我遇到的一个错误:pcl_visualizer.cpp:3567:88  
visualization/src/pcl_visualizer.cpp:3567:88: error: invalid conversion from ‘int’ to ‘const char*’ [-fpermissive]
  通过github找到了解决办法:修改源码中pcl_visualizer.cpp,从3563行开始修改成如下代码  
#if VTK_MAJOR_VERSION >= 8 && VTK_MINOR_VERSION >= 2
 
mapper->MapDataArrayToMultiTextureAttribute(mesh.tex_materials[tex_id].tex_name.c_str(),
 
this_coordinates_name.c_str(),
 
vtkDataObject::FIELD_ASSOCIATION_POINTS);
 
#else
 
mapper->MapDataArrayToMultiTextureAttribute(tu,
 
this_coordinates_name.c_str (),
 
vtkDataObject::FIELD_ASSOCIATION_POINTS);
 
this_coordinates_name.c_str(),
 
vtkDataObject::FIELD_ASSOCIATION_POINTS);
 
#endif
 
polydata->GetPointData ()->AddArray (coordinates);
 
actor->GetProperty ()->SetTexture (tu, texture);
 
++tex_id;
 
}
 

2. 下载编译GPD库

  https://github.com/atenpas/gpd/tree/2.0.0,和上面的方法类似,下载文件之后解压到安装的目录下,但是由于这个包的依赖比较新,编译的时候提示cmake版本低,网上搜索发现,升级cmake需要先卸载老版本的cmake,但是ROS用户会由于这个操作把ROS给干崩,巨坑,所以我只修改了CMakeList.txt,将cmake_minimum_required(VERSION 3.5.1 FATAL_ERROR)中的版本换成自己的较低的版本,实测成功。可以看到后面库要求的依赖是 find_package(PCL 1.9 REQUIRED)是1.9的版本:老规矩先编译在安装  
$ cd gpd
$ mkdir build && cd build
$ cmake ..
$ make -j
 
$ sudo make install
 

3. 下载编译gpd_ros

  https://github.com/atenpas/gpd_ros.git ,编译过程中还是有问题:  
In file included from /home/redwall/catkin_ws/src/grasps/gpd_ros/src/gpd_ros/grasp_messages.cpp:1:0:
/home/redwall/catkin_ws/src/grasps/gpd_ros/include/gpd_ros/grasp_messages.h:44:12: error: ‘GraspConfigList’ in namespace ‘gpd_ros’ does not name a type
   gpd_ros::GraspConfigList createGraspListMsg(const std::vector<std::unique_ptr<gpd::candidate::Hand>>& hands, const std_msgs::Header& header);
            ^
/home/redwall/catkin_ws/src/grasps/gpd_ros/src/gpd_ros/grasp_messages.cpp:3:10: error: ‘GraspConfigList’ in namespace ‘gpd_ros’ does not name a type
 gpd_ros::GraspConfigList GraspMessages::createGraspListMsg(const std::vector<std::unique_ptr<gpd::candidate::Hand>>& hands, const std_msgs::Header& header)
  通过修改gpd_ros包中的CMakeLists.txt,将PCL依赖部分换成find_package(PCL 1.9.0 REQUIRED),指定成最新升级的版本而不是用老版本。   第二个错误:  
[ 95%] Built target gpd_ros_generate_messages
In file included from /usr/local/include/gpd/descriptor/image_generator.h:54:0,
                 from /usr/local/include/gpd/grasp_detector.h:51,
                 from /home/redwall/catkin_ws/src/grasps/gpd_ros/include/gpd_ros/grasp_detection_node.h:55,
                 from /home/redwall/catkin_ws/src/grasps/gpd_ros/src/gpd_ros/grasp_detection_node.cpp:1:
/usr/local/include/gpd/descriptor/image_strategy.h:40:31: fatal error: opencv2/imgproc.hpp: 没有那个文件或目录
compilation terminated.
grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps.dir/build.make:62: recipe for target 'grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps.dir/src/gpd_ros/grasp_detection_node.cpp.o' failed
make[2]: *** [grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps.dir/src/gpd_ros/grasp_detection_node.cpp.o] Error 1
CMakeFiles/Makefile2:1341: recipe for target 'grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps.dir/all' failed
make[1]: *** [grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps.dir/all] Error 2
make[1]: *** 正在等待未完成的任务....
In file included from /usr/local/include/gpd/descriptor/image_generator.h:54:0,
                 from /usr/local/include/gpd/grasp_detector.h:51,
                 from /home/redwall/catkin_ws/src/grasps/gpd_ros/include/gpd_ros/grasp_detection_server.h:46,
                 from /home/redwall/catkin_ws/src/grasps/gpd_ros/src/gpd_ros/grasp_detection_server.cpp:1:
/usr/local/include/gpd/descriptor/image_strategy.h:40:31: fatal error: opencv2/imgproc.hpp: 没有那个文件或目录
compilation terminated.
grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps_server.dir/build.make:62: recipe for target 'grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps_server.dir/src/gpd_ros/grasp_detection_server.cpp.o' failed
make[2]: *** [grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps_server.dir/src/gpd_ros/grasp_detection_server.cpp.o] Error 1
CMakeFiles/Makefile2:200: recipe for target 'grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps_server.dir/all' failed
make[1]: *** [grasps/gpd_ros/CMakeFiles/gpd_ros_detect_grasps_server.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2
Invoking "make -j4 -l4" failed
  看到了/usr/local/include/gpd/descriptor/image_strategy.h:40:31: fatal error: opencv2/imgproc.hpp: 没有那个文件或目录,应该是opencv库的问题,看了以下CMakeLists.txt,发现果然没有opencv的依赖,添加然后在编译即可:  
# Opencv
find_package(OpenCV REQUIRED)
include_directories(${OpenCV_INCLUDE_DIRS})
link_directories(${OpenCV_LIBRARY_DIRS})
add_definitions(${OpenCV_DEFINITIONS})
  今天又编译了一次,出现了新的问题:  
fatal error: gpd_ros/GraspConfig.h: 没有那个文件或目录
  可能是因为.msg文件没来得及生成,多尝试两次即可。  
fatal error: pcl_conversions/pcl_conversions.h: 没有那个文件或目录
  PCL17才有的头文件,可能安装新版本没有了,可以自己手动安装。 sudo apt-get install ros-kinetic-pcl-conversions   编译,完美通过  
分割线
  之前配置PCL1.9的时候,对于低版本的VTK,编译过程中会出现一些警告,刚好这段时间用的包需要依赖VTK8.2的版本,于是下载了源码包编译安装,一段时间之后,再次编译gpd_ros出现了一些错误,一番搜索之后,发现好像是更新VTK版本之后,之前编译的动态库就被替换了,换言之就是先基于VTK低版本编译安装了PCL1.9和gpd_ros包,然后升级了VTK8.2版本之后导致PCL找不到原本的动态库,导致gpd_ros编译报错。。。。坑!!!   由于是通过make源码编译安装的,没办法直接卸载VTK高版本,只能手动删除了,通过make编译的库一般都在/usr/local/lib目录下,具体再那个文件夹下,需要再/usr/local下的几个文件夹仔细找找,一般都是libvtkxxx.so.8.2这种格式的。   删除之后,再次编译gpd_ros之后,就没有上面的错误了,但是新的错误又来了,纠结了好久,千万记得将ROS工作空间里面的build文件夹和devel文件夹清理以下,否则都是之前的编译完成的东西,一旦链接的还是原来的动态库自然就会报错!   解决pip安装的绝大数问题:  
sudo pip install "numba==0.34" --default-timeout=1000 --no-cache-dir -i https://pypi.tuna.tsinghua.edu.cn/simple
 
pip install "rtree>=0.8,<0.9“ --default-timeout=1000 --no-cache-dir -i https://pypi.tuna.tsinghua.edu.cn/simple