如今语音识别在PC机和智能手机上炒的火热,ROS走在技术的最前沿当然也不会错过这么帅的技术。ROS中使用了CMU Sphinx和Festival开源项目中的代码,发布了独立的语音识别包,而且可以将识别出来的语音转换成文字,然后让机器人智能处理后说话。
一、语音识别包
1、安装
安装很简单,直接使用ubuntu命令即可,首先安装依赖库:
$ sudo apt-get install gstreamer0.10-pocketsphinx
$ sudo apt-get install ros-fuerte-audio-common
$ sudo apt-get install libasound2
然后来安装ROS包:
$ svn checkout http://albany-ros-pkg.googlecode.com/svn/trunk/rharmony
$ rosmake pocketsphinx
其中的核心文件就是nodes文件夹下的recognizer.py文件了。这个文件通过麦克风收集语音信息,然后调用语音识别库进行识别生成文本信息,通过/recognizer/output消息发布,其他节点就可以订阅该消息然后进行相应的处理了。
2、测试
安装完成后我们就可以运行测试了。
首先,插入你的麦克风设备,然后在系统设置里测试麦克风是否有语音输入。
然后,运行包中的测试程序:
首先,插入你的麦克风设备,然后在系统设置里测试麦克风是否有语音输入。
然后,运行包中的测试程序:
$ roslaunch pocketsphinx robocup.launch
此时,在终端中会看到一大段的信息。尝试说一些简单的语句,当然,必须是英语,例如:bring me the glass,come with me,看看能不能识别出来。
《ros by example》这本书中写得测试还是很准确的,但是我在测试中感觉识别相当不准确,可能是我的英语太差了吧。
《ros by example》这本书中写得测试还是很准确的,但是我在测试中感觉识别相当不准确,可能是我的英语太差了吧。

我们也可以直接看ROS最后发布的结果消息:
$ rostopic echo /recognizer/output

二、语音库
1、查看语音库
这个语音识别时一种离线识别的方法,将一些常用的词汇放到一个文件中,作为识别的文本库,然后分段识别语音信号,最后在库中搜索对应的文本信息。如果想看语音识别库中有哪些文本信息,可以通过下面的指令进行查询:
$ roscd pocketsphinx/demo
$ more robocup.corpus
我们可以自己向语音库中添加其他的文本识别信息,《ros by example》自带的例程中是带有语音识别的例程的,而且有添加语音库的例子。
首先看看例子中要添加的文本信息:
首先看看例子中要添加的文本信息:
$ roscd rbx1_speech/config
$ more nav_commands.txt

这就是需要添加的文本,我们也可以修改其中的某些文本,改成自己需要的。
然后我们要把这个文件在线生成语音信息和库文件,这一步需要登陆网站http://www.speech.cs.cmu.edu/tools/lmtool-new.html,根据网站的提示上传文件,然后在线编译生成库文件。
然后我们要把这个文件在线生成语音信息和库文件,这一步需要登陆网站http://www.speech.cs.cmu.edu/tools/lmtool-new.html,根据网站的提示上传文件,然后在线编译生成库文件。

把下载的文件都解压放在rbx1_speech包的config文件夹下。我们可以给这些文件改个名字:
$ roscd rbx1_speech/config
$ rename -f 's/3026/nav_commands/' *
在rbx1_speech/launch文件夹下看看voice_nav_commands.launch这个文件:
<launch>
<node name="recognizer" pkg="pocketsphinx" type="recognizer.py"
output="screen">
<param name="lm" value="$(find rbx1_speech)/config/nav_commands.lm"/>
<param name="dict" value="$(find rbx1_speech)/config/nav_commands.dic"/>
</node>
</launch>
可以看到,这个launch文件在运行recognizer.py节点的时候使用了我们生成的语音识别库和文件参数,这样就可以实用我们自己的语音库来进行语音识别了。
通过之前的命令来测试一下效果如何吧:
通过之前的命令来测试一下效果如何吧:
$ roslaunch rbx1_speech voice_nav_commands.launch
$ rostopic echo /recognizer/output
三、语音控制
有了语音识别,我们就可以来做很多犀利的应用了,首先先来尝试一下用语音来控制机器人动作。
1、机器人控制节点
前面说到的recognizer.py会将最后识别的文本信息通过消息发布,那么我们来编写一个机器人控制节点接收这个消息,进行相应的控制即可。
在pocketsphinx包中本身有一个语音控制发布Twist消息的例程voice_cmd_vel.py,rbx1_speech包对其进行了一些简化修改,在nodes文件夹里可以查看voice_nav.py文件:
在pocketsphinx包中本身有一个语音控制发布Twist消息的例程voice_cmd_vel.py,rbx1_speech包对其进行了一些简化修改,在nodes文件夹里可以查看voice_nav.py文件:
#!/usr/bin/env python
"""
voice_nav.py
Allows controlling a mobile base using simple speech commands.
Based on the voice_cmd_vel.py script by Michael Ferguson in
the pocketsphinx ROS package.
See http://www.ros.org/wiki/pocketsphinx
"""
import roslib; roslib.load_manifest('rbx1_speech')
import rospy
from geometry_msgs.msg import Twist
from std_msgs.msg import String
from math import copysign
class VoiceNav:
def __init__(self):
rospy.init_node('voice_nav')
rospy.on_shutdown(self.cleanup)
# Set a number of parameters affecting the robot's speed
self.max_speed = rospy.get_param("~max_speed", 0.4)
self.max_angular_speed = rospy.get_param("~max_angular_speed", 1.5)
self.speed = rospy.get_param("~start_speed", 0.1)
self.angular_speed = rospy.get_param("~start_angular_speed", 0.5)
self.linear_increment = rospy.get_param("~linear_increment", 0.05)
self.angular_increment = rospy.get_param("~angular_increment", 0.4)
# We don't have to run the script very fast
self.rate = rospy.get_param("~rate", 5)
r = rospy.Rate(self.rate)
# A flag to determine whether or not voice control is paused
self.paused = False
# Initialize the Twist message we will publish.
self.cmd_vel = Twist()
# Publish the Twist message to the cmd_vel topic
self.cmd_vel_pub = rospy.Publisher('cmd_vel', Twist)
# Subscribe to the /recognizer/output topic to receive voice commands.
rospy.Subscriber('/recognizer/output', String, self.speech_callback)
# A mapping from keywords or phrases to commands
self.keywords_to_command = {'stop': ['stop', 'halt', 'abort', 'kill', 'panic', 'off', 'freeze', 'shut down', 'turn off', 'help', 'help me'],
'slower': ['slow down', 'slower'],
'faster': ['speed up', 'faster'],
'forward': ['forward', 'ahead', 'straight'],
'backward': ['back', 'backward', 'back up'],
'rotate left': ['rotate left'],
'rotate right': ['rotate right'],
'turn left': ['turn left'],
'turn right': ['turn right'],
'quarter': ['quarter speed'],
'half': ['half speed'],
'full': ['full speed'],
'pause': ['pause speech'],
'continue': ['continue speech']}
rospy.loginfo("Ready to receive voice commands")
# We have to keep publishing the cmd_vel message if we want the robot to keep moving.
while not rospy.is_shutdown():
self.cmd_vel_pub.publish(self.cmd_vel)
r.sleep()
def get_command(self, data):
# Attempt to match the recognized word or phrase to the
# keywords_to_command dictionary and return the appropriate
# command
for (command, keywords) in self.keywords_to_command.iteritems():
for word in keywords:
if data.find(word) > -1:
return command
def speech_callback(self, msg):
# Get the motion command from the recognized phrase
command = self.get_command(msg.data)
# Log the command to the screen
rospy.loginfo("Command: " + str(command))
# If the user has asked to pause/continue voice control,
# set the flag accordingly
if command == 'pause':
self.paused = True
elif command == 'continue':
self.paused = False
# If voice control is paused, simply return without
# performing any action
if self.paused:
return
# The list of if-then statements should be fairly
# self-explanatory
if command == 'forward':
self.cmd_vel.linear.x = self.speed
self.cmd_vel.angular.z = 0
elif command == 'rotate left':
self.cmd_vel.linear.x = 0
self.cmd_vel.angular.z = self.angular_speed
elif command == 'rotate right':
self.cmd_vel.linear.x = 0
self.cmd_vel.angular.z = -self.angular_speed
elif command == 'turn left':
if self.cmd_vel.linear.x != 0:
self.cmd_vel.angular.z += self.angular_increment
else:
self.cmd_vel.angular.z = self.angular_speed
elif command == 'turn right':
if self.cmd_vel.linear.x != 0:
self.cmd_vel.angular.z -= self.angular_increment
else:
self.cmd_vel.angular.z = -self.angular_speed
elif command == 'backward':
self.cmd_vel.linear.x = -self.speed
self.cmd_vel.angular.z = 0
elif command == 'stop':
# Stop the robot! Publish a Twist message consisting of all zeros.
self.cmd_vel = Twist()
elif command == 'faster':
self.speed += self.linear_increment
self.angular_speed += self.angular_increment
if self.cmd_vel.linear.x != 0:
self.cmd_vel.linear.x += copysign(self.linear_increment, self.cmd_vel.linear.x)
if self.cmd_vel.angular.z != 0:
self.cmd_vel.angular.z += copysign(self.angular_increment, self.cmd_vel.angular.z)
elif command == 'slower':
self.speed -= self.linear_increment
self.angular_speed -= self.angular_increment
if self.cmd_vel.linear.x != 0:
self.cmd_vel.linear.x -= copysign(self.linear_increment, self.cmd_vel.linear.x)
if self.cmd_vel.angular.z != 0:
self.cmd_vel.angular.z -= copysign(self.angular_increment, self.cmd_vel.angular.z)
elif command in ['quarter', 'half', 'full']:
if command == 'quarter':
self.speed = copysign(self.max_speed / 4, self.speed)
elif command == 'half':
self.speed = copysign(self.max_speed / 2, self.speed)
elif command == 'full':
self.speed = copysign(self.max_speed, self.speed)
if self.cmd_vel.linear.x != 0:
self.cmd_vel.linear.x = copysign(self.speed, self.cmd_vel.linear.x)
if self.cmd_vel.angular.z != 0:
self.cmd_vel.angular.z = copysign(self.angular_speed, self.cmd_vel.angular.z)
else:
return
self.cmd_vel.linear.x = min(self.max_speed, max(-self.max_speed, self.cmd_vel.linear.x))
self.cmd_vel.angular.z = min(self.max_angular_speed, max(-self.max_angular_speed, self.cmd_vel.angular.z))
def cleanup(self):
# When shutting down be sure to stop the robot!
twist = Twist()
self.cmd_vel_pub.publish(twist)
rospy.sleep(1)
if __name__=="__main__":
try:
VoiceNav()
rospy.spin()
except rospy.ROSInterruptException:
rospy.loginfo("Voice navigation terminated.")
可以看到,代码中定义了接收到各种命令时的控制策略。
2、仿真测试
和前面一样,我们在rviz中进行仿真测试。
首先是运行一个机器人模型:
首先是运行一个机器人模型:
$ roslaunch rbx1_bringup fake_turtlebot.launch
然后打开rviz:
$ rosrun rviz rviz -d `rospack find rbx1_nav`/sim_fuerte.vcg
如果不喜欢在终端里看输出,可以打开gui界面:
$ rxconsole
再打开语音识别的节点:
$ roslaunch rbx1_speech voice_nav_commands.launch
最后就是机器人的控制节点了:
$ roslaunch rbx1_speech turtlebot_voice_nav.launch
OK,打开上面这一堆的节点之后,就可以开始了。可以使用的命令如下:

下图是我的测试结果,不过感觉准确度还是欠佳:

四、播放语音
现在机器人已经可以按照我们说的话行动了,要是机器人可以和我们对话就更好了。ROS中已经集成了这样的包,下面就来尝试一下。
运行下面的命令:
运行下面的命令:
$ rosrun sound_play soundplay_node.py
$ rosrun sound_play say.py "Greetings Humans. Take me to your leader."
有没有听见声音!ROS通过识别我们输入的文本,让机器人读了出来。发出这个声音的人叫做kal_diphone,如果不喜欢,我们也可以换一个人来读:
$ sudo apt-get install festvox-don
$ rosrun sound_play say.py "Welcome to the future" voice_don_diphone
哈哈,这回换了一个人吧,好吧,这不也是我们的重点。
在rbx1_speech/nodes文件夹中有一个让机器人说话的节点talkback.py:
在rbx1_speech/nodes文件夹中有一个让机器人说话的节点talkback.py:
#!/usr/bin/env python
"""
talkback.py - Version 0.1 2012-01-10
Use the sound_play client to say back what is heard by the pocketsphinx recognizer.
Created for the Pi Robot Project: http://www.pirobot.org
Copyright (c) 2012 Patrick Goebel. All rights reserved.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.5
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details at:
http://www.gnu.org/licenses/gpl.htmlPoint
"""
import roslib; roslib.load_manifest('rbx1_speech')
import rospy
from std_msgs.msg import String
from sound_play.libsoundplay import SoundClient
import sys
class TalkBack:
def __init__(self, script_path):
rospy.init_node('talkback')
rospy.on_shutdown(self.cleanup)
# Set the default TTS voice to use
self.voice = rospy.get_param("~voice", "voice_don_diphone")
# Set the wave file path if used
self.wavepath = rospy.get_param("~wavepath", script_path + "/../sounds")
# Create the sound client object
self.soundhandle = SoundClient()
# Wait a moment to let the client connect to the
# sound_play server
rospy.sleep(1)
# Make sure any lingering sound_play processes are stopped.
self.soundhandle.stopAll()
# Announce that we are ready for input
self.soundhandle.playWave(self.wavepath + "/R2D2a.wav")
rospy.sleep(1)
self.soundhandle.say("Ready", self.voice)
rospy.loginfo("Say one of the navigation commands...")
# Subscribe to the recognizer output and set the callback function
rospy.Subscriber('/recognizer/output', String, self.talkback)
def talkback(self, msg):
# Print the recognized words on the screen
rospy.loginfo(msg.data)
# Speak the recognized words in the selected voice
self.soundhandle.say(msg.data, self.voice)
# Uncomment to play one of the built-in sounds
#rospy.sleep(2)
#self.soundhandle.play(5)
# Uncomment to play a wave file
#rospy.sleep(2)
#self.soundhandle.playWave(self.wavepath + "/R2D2a.wav")
def cleanup(self):
self.soundhandle.stopAll()
rospy.loginfo("Shutting down talkback node...")
if __name__=="__main__":
try:
TalkBack(sys.path[0])
rospy.spin()
except rospy.ROSInterruptException:
rospy.loginfo("Talkback node terminated.")
我们来运行看一下效果:
$ roslaunch rbx1_speech talkback.launch
然后再说话,ROS不仅将文本信息识别出来了,而且还读了出来,厉害吧。当然了,现在还没有加入什么人工智能的算法,不能让机器人和我们聪明的说话,不过这算是基础了,以后有时间再研究一下人工智能就更犀利了。
评论列表(35条)
古月老师,为什么我下载了github里的源代码,去掉ROS2的内容后编译还是不通过,说找不到libmsc这个文件
[ 27%] Linking CXX executable /home/***/catkin_ws/devel/lib/robot_voice/tts_subscribe
/usr/bin/ld: cannot find -lmsc
collect2: error: ld returned 1 exit status
ros_exploring/robot_perception/robot_voice/CMakeFiles/tts_subscribe.dir/build.make:113: recipe for target ‘/home/***/catkin_ws/devel/lib/robot_voice/tts_subscribe’ failed
make[2]: *** [/home/***/catkin_ws/devel/lib/robot_voice/tts_subscribe] Error 1
CMakeFiles/Makefile2:1928: recipe for target ‘ros_exploring/robot_perception/robot_voice/CMakeFiles/tts_subscribe.dir/all’ failed
make[1]: *** [ros_exploring/robot_perception/robot_voice/CMakeFiles/tts_subscribe.dir/all] Error 2
Makefile:138: recipe for target ‘all’ failed
make: *** [all] Error 2
Invoking “make -j1 -l1” failed
解决了:https://blog.csdn.net/suer0101/article/details/7280240
请把robot_voice中的libmsc拷贝到/usr/lib中
古月,你好,ros中是否还存类似的包能够将音频文件输入,然后判断音频内容,因为目前我外接一个MIC 阵列,能够读取到音频采样后的pcm的值,然后想把音频文件直接作为输入,判断.
在你的书中,第188页,语音识别测试,如果遇到pocketsphinx功能包运行时错误,下载语音引擎,并且解压缩文件和拷贝,这里能讲一下具体操作吗?
同问,我探索了大半天,也遇到这个问题了
问题已经解决了,但是我把所有的东西都安装好后,包括修改recognizer.py文件,再robocup.launch中设置lm/dic/hmm参数的具体路径,然后运行robocup.launch启动语音识别,再执行rostopic echo /recognizer/output命令时,弹出警告
WARNING: topic [/recognizer/output] does not appear to be published yet
然后就没有反映了,请问老师这如何解决,才能成功识别语音
用rostopic list看下有没有output话题
古月老师,您好:
我目前正在学习ROS系统,用的教程是您的《ROS机器人开发实践》,第8章机器语音中,8.1.1 pocketsphinx功能包安装时,使用命令sudo dpkg -i libsphinxbase1_0.8-6_amd64.deb 时,报错如下:
dpkg: 处理归档 libsphinxbase1_0.8-6_amd64.deb (–install)时出错:
无法访问归档文件: 没有那个文件或目录
在处理时有错误发生:
libsphinxbase1_0.8-6_amd64.deb
上网差了半天也找不到原因
请老师指教,谢谢
执行这句命令的时候是在当前路径下找libsphinxbase1_0.8-6_amd64.deb的,确定路径没问题么?
我也遇到了相同的问题,请问如何解决的呢?
语句换成这个sudo apt-get install ros-kinetic-audio-common libasound2 gstreamer0.10-* gstreamer1.0-pocketsphinx
我按照您书上的方法,出现了以下的错误,ImportError No module named pygst
pygst没有安装
请问一下,这个你解决了吗?
请问你解决了吗
请问你解决了吗
见文https://blog.csdn.net/yikangj/article/details/82024809
请问一下我在安装依赖时为什么出现E: 无法定位软件包 gstreamer0.10-pocketsphinx,版本是kinetic,是因为需要翻墙吗?
问题解决了,源的问题
赞
请问一下,你是怎么解决的?
语句换成这个sudo apt-get install ros-kinetic-audio-common libasound2 gstreamer0.10-* gstreamer1.0-pocketsphinx
svn checkout http://albany-ros-pkg.googlecode.com/svn/trunk/rharmony
月哥当前我的虚拟机不能翻墙不能用这条命令获取到包,能麻烦发个包给我吗万分感谢
我的邮箱469249801@qq.com
参考http://www.guyuehome.com/514,在indigo上重新做的,现在应该不用单独编译rharmony了
我试了一下,bring me glass那边识别率还是挺高的,到了forward那边识别率就比较低了。另外我发现发音浑浊一些识别率会变高,可能还是跟英文的准确度有关。
是不是仅有博主可以回复。。。
@谢志浩
https://github.com/pirobot/rbx1
《ros by example》一书第五章有详细说明,下面是该书indigo版本的电子书
https://github.com/Durant35/ROS_Note/blob/master/books/ros_by_example_1_indigo.pdf
不好意思,所有人都可以回复,不知道为什么网站拦截了你的回复,我检查一下配置
https://github.com/pirobot/rbx1
《ros by example》一书第五章有详细说明,下面是该书indigo版本的电子书
https://github.com/Durant35/ROS_Note/blob/master/books/ros_by_example_1_indigo.pdf
请问文中提到的rbx1是什么文件,我在ros by example 中看到关于rbx1的文字,还是不知道从哪里下载以及安装。请大神指教
https://github.com/pirobot/rbx1
这个《ros by example》一书中第五章有详细说明
这里是indigo版本的电子书 https://github.com/Durant35/ROS_Note/blob/master/books/ros_by_example_1_indigo.pdf
https://github.com/pirobot/rbx1
《ros by example》一书第五章有详细说明,下面是该书indigo版本的电子书
https://github.com/Durant35/ROS_Note/blob/master/books/ros_by_example_1_indigo.pdf
rbx1是ros by example这本书源码的功能包,在书里有源码的下载地址
非常感谢,我下载了PDF版的并且安装好了,顺便想问一下哪里有英文的原版书卖
现在网上都有下载