Some of you may remember in my original review of the AIY voice kit, that I mentioned the fact that the ladybox needed to be activated by either a button press or by clapping. I commented at the time that someone was soon going to come up with a voice or hotword activation trigger fairly soon.
Well since then I came across a video on youtube by Sid’s E-classroom which does just this using Snowboy. However this wasn’t ideal since it required installing pyaudio and sox so uses a separate recording system from the AIY voice kit. And you may remember that sox (though brilliant in a lot of ways) is very difficult to get working satisfactorily. Also due to the rather complex code of the kit it seemed a bit convoluted to set up.
Since then version two of the AIY voice kit api has completely rewritten the original code. Buried in one of the python files (I forget which) there is a hotword option. But this works on the basis that the hotword along with the everything else is sent to google to be transcribed. If the hotword isn’t found then the transcript isn’t returned. Ok – but this means all your private conversations get sent to google – do you really want that. Snowboy on the other hand works on simple one or two hotwords which are recognized locally and don’t go to the internet.
So I’ve taken the Snowboy approach with version 2 of the api.
The task was to get Snowboy working on the raspberry pi. The site gives the option to compile with swig. Now I don’t know about you but I’ve never heard of it before. But downloaded and tried to compile the code from github and it turns out you need version 3.0.10 or higher. Unfortunately the latest version of swig for the pi is below that. Next option download the precompiled version which can be found on the same page. But guess what this is set up to work with Python 2 not 3 as is used by the AIY voice kit. Luckily the solution I found was on the forum where one of the Snowboy authors posted a compiled version for Raspberry Pi – Python3 in response to someone’s request. Strangely a few days later the files were removed. I’m requesting the author on this at the moment, but in the meantime I will make the file available on my site. But bare this in mind.
After these initial hurdles everything run smoothly. I cut down the Snowboy python module to one function as it had many references to pyaudio which aren’t needed for use with the aiy voice kit. To integrate it with the google supplied software what I did was create a processor type object which is passed to the recording object and has callback functionality which goes into a queue. This is then passed in a loop to Snowboy. Snowboy returns a different status if voice is detected or the hotword is detected.
Creating a hotword is fairly simple. You need to open an account with Snowboy and send three sample wavs. You can then download a pmdl file which Snowboy will use to detect the hotword. It’s as simple as that. I have created a small python program which will record the wavs for you, send the files and retrieve the file all in one go.
There are three ways to use the functionality. Initially the ladybox waits for the hotword. She can then wait for the next human interaction. As soon as voice is heard this get’s sent to google. Thirdly there can be timed voice recognition. She waits for a certain amount of time for voice before going back to sleep (or waiting for the hotword again). I also include a context list which keeps track of the conversation context.
Adding this functionality has made the kit more naturally conversationally. I hope this will be useful for some of you. Let me know if you get it working or have any issues.
Code is available at bitbucket. Since posting this there is a newer version of the aiy code which uses python 3.6 which doesn’t work with the snowboy bindings I have. The version of the code that works can be found here https://github.com/google/aiyprojects-raspbian/releases/tag/voicekit. If anyone finds a way to get the later version working let me know.
January 2, 2018 at 4:39 pm
This is excellent! I was pretty mortified by the unnecessarily convoluted approach taken in https://youtu.be/mETxgn0vH2I?t=323 I started researching to do an approach similar to yours. https://www.raspberrypi.org/forums/viewtopic.php?f=114&t=200926#p1253264 I’m glad you saved me the trouble. I’ll give you more feedback once I put this into practice.
LikeLike
January 2, 2018 at 8:53 pm
Yes – the previous approach is convoluted mainly due to the convoluted nature of the aiy software which wasn’t written to be easily used by developers. For example – why did you need to hack the action.py file rather than call whatever functions you need. Version 2 is a lot better though it still required quite a bit of patience to trawl through the innards of the code to find how it was doing things. Snowboy seems to work really well for me after cutting out the superfluous stuff. Looking forward to hearing any feedback from yourself in due course.
LikeLiked by 1 person
January 6, 2018 at 11:15 pm
Is there no way to create a new hotword within the .py files? I’ve currently got a basic assistant running in my_assistant.py, and would love it if I could simply add some code or something to create a new hotword.
LikeLike
January 6, 2018 at 11:24 pm
Yes absolutely. You have to create an account with snowboy and then put the token and a few other settings into the recordKeyword.py at https://bitbucket.org/dani_thomas/aiyhotworddetector. Then run that and it will record three the keyword three times and send you back the file you need for your keyword recognition. Let me know if you have any problems.
LikeLike
January 13, 2018 at 2:28 pm
Hey. I setup the assistant on my Pi using the SDK and the instructions provided by Google. Could you post a how-to guide explaining how you can integrate snowboy on top of that to replace the hotword ? Thanks!
LikeLike
January 13, 2018 at 4:47 pm
When I get a chance I’ll write an example specifically for the assistant rather than the cloud services. But basically you would set it up the same way (see the instructions https://bitbucket.org/dani_thomas/aiyhotworddetector/overview). If you then look at the assistant_grpc_demo.py where it has the button.wait_for_press() put miaHot.waitForHotword() code (see hotwordDemo.py). Hope that is enough to get you going in the meantime.
Update: have done a google assistant example assistant_grpc_hotworddemo.py and updated the instructions on bitbucket give a bit more clarity.
LikeLike
January 10, 2018 at 1:25 pm
I’m hawe a trouble with import module: ImportError: libpython3.4m.so.1.0: cannot open shared object file: No such file or directory. Please, help me.
LikeLiked by 1 person
January 10, 2018 at 2:49 pm
I think that is due to you running python 2 not 3. Easiest way to fix that is to run the shell script that is on the desktop “Start Dev Terminal”. Or if you are running from the command line it is /home/pi/bin/AIY-voice-kit-shell.sh. That should sort you out – let me know if you still have problems.
LikeLiked by 1 person
January 13, 2018 at 7:15 pm
Hello, I am having the same error even though I am running it from the Dev Terminal and all the files are in the src folder.
LikeLike
January 13, 2018 at 7:56 pm
I believe it’s because I’m running Python 3.5 and not 3.4 like the one it tries to import.
LikeLike
January 14, 2018 at 11:13 am
I think you are probably right. I used the aiy voice kit image which uses 3.4.2. If you run “which python” it should be “/home/pi/AIY-voice-kit-python/env/bin/python” because that has been set up for the environment and should be 3.4.
Unfortunately I don’t think there is a way to get Snowboy bindings for 3.5 – it was difficult enough finding 3.4. You may be able to get it working by setting a path to the 3.4 lib with LD_LIBRARY_PATH=. see https://stackoverflow.com/questions/7880454/python-executable-not-finding-libpython-shared-library. Good luck.
LikeLiked by 1 person
January 18, 2018 at 9:22 pm
I managed to get it working, thank you!
I really like this approach to get a hot-word recogniser working.
In your example you created a chat box or custom assistant, which is really useful, but is there any possibility to integrate this hotword recognized to trigger the google assistant as well? I was trying to do that.
LikeLike
January 18, 2018 at 10:27 pm
I haven’t done so myself but if you check the assistant demo assistant_grpc_demo.py which uses the button and replace the wait for button with wait for hot word it should work exactly the same. I will try and get an example done over the weekend as someone else has also asked the same.
LikeLike
January 20, 2018 at 4:52 pm
As promised I’ve done an example using the google assistant based on the assistant_grpc_demo.py. The other demo is doable as well but requires adding a callback routine in place of the button one.
LikeLiked by 1 person
January 23, 2018 at 2:38 am
Thank you for for the work!
Is there any way to shut up the google assistant conversation for certain functions using the grpc assistant?
My assistant has among others, a function that tells the temperature of the CPU. Before I could just use “assistant.stop_conversation()” so only the aiy.audio.say will play out; but this function doesn’t work anymore, so after it says the text, the google assistant says something like “I didnt understand” and I wish to eliminate that.
LikeLike
January 23, 2018 at 8:33 am
I’m not sure what is happening. Could you send me a snippet of the code. In theory the code should work like this – first wait for hotword – then run assistant recognize while recording – stop listening and recording – play audio. Then back to waiting for hotword. I’m wondering whether you are running the temperature in a different process so the code is looping round to the start again.
LikeLike
January 23, 2018 at 10:08 am
I believe that’s what’s happening. But not so sure how to fix that without including all the code in the main() loop. Here is the code:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
My_assistant_hotword_temp.py
hosted with ❤ by GitHub
LikeLike
January 23, 2018 at 12:45 pm
I like what you are trying to do. I think the problem is that you are “saying” the reply from pi_temperature and also “playing” the response from google. What I think you should do is — if text == “whats the temperature” else if audio is not none: play audio. Or alternatively return a flag from pi_temperature and on that basis don’t play the audio. Also just thought of another way (probably the simplest) straight under the call to pi_temperature set audio to None.
LikeLiked by 1 person
January 23, 2018 at 6:30 pm
Thank you very much. Didn’t occurred me to set the audio to None. It works flawlessly now!
LikeLike
January 27, 2018 at 11:59 am
As a further thought to this. You may find it a bit irritating that the voice changes between the cloud services and the assistant. You might consider using the google translate text to speech rather than the pico2wave which is a bit robotic. I haven’t done a proper comparison but I think it is similar. Though on the downside there might be more lag possibly.
LikeLike
January 21, 2018 at 9:10 pm
Any chance you can share how you managed to get it to work?
I am struggling to get python 3.4.2 on the raspberry pi 3 (original AIY kit with latest image) and change the LD_LIBRARY_PATH.
Cheers
LikeLike
January 22, 2018 at 1:22 pm
Hi mpolo,
I received this below which may be of use. You should be able to install more than one version of Python and as long as snowboy has access to the 3.4.2 library then it should work.
Hi Dani,
I wish I could tell you exactly the steps I’ve performed to fix the issue, but I’m not so sure myself.
But I can tell you that most likely was due to me trying to install the python package anaconda (miniconda for the pi).
Because after I’ve succeeded doing that I’ve noticed that it used python 3.4 as well, so I tried to install it in the virtual environment (env) that it’s used by the AIY script.
Perhaps this caused it to symlink or use the libraries from previously available python packages, and install python 3.4.
For doing so I’ve followed the steps here: https://stackoverflow.com/questions/39371772/how-to-install-anaconda-on-raspberry-pi-3-model-b
Can’t tell you for sure that was what fixed it because I’ve also tinkered around with other things but that seems to be one of the causing effects.
LikeLiked by 1 person
June 14, 2018 at 10:48 pm
How did you get it to work? I’m using the AIY Voice Kit as well, but it’s running python 3.5.
LikeLike
January 27, 2018 at 9:48 pm
Hi Dani,
This is excellent. I got it working.
Regarding : assistant_grpc_hotworddemo.py
When I run this python program, it is working fine, but the python process is consuming more and more memory over time which makes my raspberry pi crash after a couple of hours.
I see that the memory of the process (checked with the top) command is linearly increasing over time (also in case we are away from home). So there is a kind of memory leak.
Have you any idea how I can get rid of this memory leak ?
kr
Jan
LikeLike
January 27, 2018 at 10:06 pm
Thanks Jan for your comments. Sorry, I don’t know off hand where the memory leak is, but I will investigate. Does it happen even if the hotword isn’t said or does it increase each time it loops. Possibly I need to remove the audio processor after each hotword detection. There is also a snowboy reset command which may clean up. It is difficult to know since snowboy is such a closed black box.
LikeLiked by 1 person
January 28, 2018 at 12:09 am
Hi Dani,
The memory of the python process keeps increasing (linearly) also when no one is saying something. I have tested this with the top command : the VIRT, RES and %MEM of the python process is steadily increasing.
I also tested the other python process (=HotwordDemo.py) and it has the same issue.
kr
Jan
LikeLike
January 28, 2018 at 7:44 pm
Hi Jan,
I didn’t seem to have the same issue as you, but I have done a fix that might help. I have updated the code but it is to use recorder.remove_processor(audio) at the end of the WaitForHotword routine in the class miaHotword (outside the while loop). Before it was adding the audio to the processor each time and therefore slowing down over time. The memory usage still fluctuates a bit but on my system goes from 3.0 to 4.5 but not higher. I didn’t have any issues if the hotword isn’t said and the system remains in the WaitForHotword so you may be having a different issue. Let me know.
Dani
LikeLiked by 1 person
January 28, 2018 at 11:07 pm
Hi Dani,
The problem seems to be fixed now.
Note that besides installing your latest version.
I installed the aiyprojects branch (before that I was using the voicekit branch) and
in order to be able to use the aiyprojects branch I also had to upgrade python from 3.4 to 3.5 (for this I needed to compile python).
But now it all looks fine: %MEM of the python3 process stays around 4.4%. So that is great.
Many thanks
Jan.
LikeLike
February 2, 2018 at 2:03 pm
Anyone tried replacing speaker with jack to connect to external audio system. Would like to connect to robot system
LikeLike
February 2, 2018 at 10:22 pm
Hi Ken. On the face of it you might have problems. If you run aplay -l it looks like the voice hat has hijacked/disabled the default output through the jack. There may be a way of re-enabling it as a secondary device. Your other option is to get two wires from the voice hat and connect to a jack (positive is usually the tip). The other possibility depending on your set up is to send the audio file via the wifi and deal with it on the receiving device.
LikeLike
February 3, 2018 at 7:54 pm
If the Hotword is created via snowboy can the button be removed?
LikeLike
February 3, 2018 at 9:12 pm
I can’t see why not. The button consists of an led and a switch. I use the led part in my demo code as does some of the google demos, but that shouldn’t be an issue. Take that bit out or leave as you wish.
LikeLike
February 4, 2018 at 7:19 pm
Well I’ve started:-(
I followed the “MadPi AIY projects” instructions which seemed to go ok until step 13 “Start the voice unit” typed src/assistant_library_demo.py
message returned “no such file or directory”.
Can you help please?
LikeLike
February 4, 2018 at 7:30 pm
Make sure you are running the command from the dev terminal. See icon on desktop.
LikeLike
February 4, 2018 at 7:41 pm
yes running from desktop terminal icon as per instructions in guide
LikeLike
February 4, 2018 at 8:04 pm
If you type cd src can you get into the directory and then type ls to see if the file is there.
LikeLike
February 4, 2018 at 8:37 pm
command prompt pi@raspberrypi: running ls shows no directory src
LikeLike
February 4, 2018 at 8:41 pm
You are almost certainly not running from the dev terminal. You would have a different prompt starting with (env)
LikeLike
February 4, 2018 at 8:41 pm
CORRECTION prompt pi@raspberrypi:~/AIY-projects-python/src
running ls
aiy examples readme,md readme.txt setup.py
LikeLike
February 4, 2018 at 8:47 pm
You might need to go into examples. Slightly different from my version. But my previous comment re terminal still holds
LikeLike
February 4, 2018 at 9:19 pm
cd examples
ls
vision voice
regards dev terminal on the desktop the icon states “Start dev terminal”
LikeLike
February 4, 2018 at 9:57 pm
changed to directory pi@raspberrypi:~/AIY-projects-python/src/examples/voice
run ls
assistant_grpc_demo.py
assistant_library_demo.py
assistant_library_with_button_demo.py
assistant_library_with_local_commands_demo.py
cloudspeech_demo.py
What now?
LikeLike
February 4, 2018 at 11:00 pm
Type python followed by demo you want to run
LikeLike
February 5, 2018 at 2:04 pm
Dani, that worked but get error:
ALSA lib pcm.c:8403:(snd pcm set params) rate doesn’t match (request 1600hz get 4800hz)
Error:audio input processor.cc 755
Input error. Sorry no underscore on kbd.
LikeLike
February 5, 2018 at 2:34 pm
It looks like some problem with the mic/voice hat. Check arecord -l that voice hat is there. Also have you tried running the 3 tests provided by google
LikeLike
February 5, 2018 at 3:58 pm
Checked audio via desktop icon. Both speaker and mic recording worked fine. Did a bit of checking on line seems I’m not the only one to have the exact same problem. Suggested sound config problem but where is the file and what should the parameters be.
LikeLike
February 5, 2018 at 4:12 pm
I can’t check at the moment but look in drivers alsa.py for line arecord. Or possibly record.py.
LikeLike
February 5, 2018 at 5:35 pm
Can’t find files. When you get a chance please point me in right direction.
Thanks
LikeLike
February 5, 2018 at 7:16 pm
/home/pi/AIY-voice-kit-python/src/aiy/_drivers/_recorder.py on my machine.
LikeLike
February 5, 2018 at 10:45 pm
For info using image aiyprojects-2018-01-03.img.xz
LikeLike
February 5, 2018 at 10:54 pm
Found recorder.py in same tree as your machine. Sent copy but seems to be moderating. As recorder.py not directory how do I investigate or modify?
LikeLike
February 6, 2018 at 8:42 am
To edit python files you have various options. There is Idle3 – I think it is on the desktop or possibly in the menu under programming. From the terminal you can call nano followed by the filename. I personally use mobaxterm and its inbuilt editor as I remote on to the raspberry pi from my pc. If you just want to view the contents use the command cat. However you should probably follow the advice of someone who has sorted out this issue as to what needs changing. I wouldn’t change things unless you are sure what you are doing.
LikeLike
February 6, 2018 at 5:35 pm
On Tue, 6 Feb 2018 08:43:01 +0000
LikeLike
February 6, 2018 at 9:21 pm
Dani SOLVED. You were right I did the cat command and it showed up.
I then used IDLE from the desktop and following some reading on line I
backed up the original file and then edited/replaced with solution found
in the Raspberry Pi forum.
The LADYBOX as you call her performed perfectly.
That’s first base completed😊
THANK YOU for all your help.
Next step is to use Snowboy and hopefully change the Hotword to “HAL”
I would also like to change the voice to male if possible.
LikeLike
February 8, 2018 at 9:59 pm
Hi,
I downloaded the files and place them in the src folder. Next I opened and Start dev terminal and tried to run assistant_grpc_demo.py but it showed: bash: src/assistant_grpc_demo.py: Permission denied
How can I resolve this?
Thanks!
LikeLike
February 8, 2018 at 10:35 pm
It could be several things. Did you run it with python. Have you set up your Google account. If it is still a problem you can always run as administrator by using sudo python followed by the file name. But you shouldn’t have to do that.
LikeLike
February 10, 2018 at 2:04 pm
Hi Dani,
I did setup my google account and everything as instructed. Then I tried using google’s demo (assistant_library_demo.py) and it worked fine.
Next I download the files from your link and cut paste them to the src folder. I changed the model_file in miaHotword.py to ‘/resources/snowboy.umdl’
Then I launched the start dev terminal and wrote the following commands:
cd src
python assistant_grpc_hotworddemo.py
I took a screen shot of the error I get:
https://drive.google.com/open?id=0B6AuFY98-9TnUzNKODN6QlMxMWRjOF9VZTdvS0l3VUxwdUFr
LikeLike
February 10, 2018 at 3:23 pm
This is an issue others have. Look at the comments above. You need to have version 3.4 of python running or at least have that library in the path.
LikeLike
February 10, 2018 at 5:04 pm
Thanks for the help and the code Dani! It’s finally working. Also I would just like to ask if these is any way I can start this program (assistant_grpc_hotwordDemo) automatically when the Pi boots up.
LikeLike
February 10, 2018 at 10:23 pm
There are a number of ways see https://www.dexterindustries.com/howto/run-a-program-on-your-raspberry-pi-at-startup/. I would recommend Method 3 using init.d as you can then start and stop like a service. You will need to run the script that sets the dev terminal environment and also remember to use full paths. Thirdly you need to run as a background process by using & after the command. I have done a script for another system here https://bitbucket.org/dani_thomas/multimotiondetect/src/75b981aa3e806ee80c5cf7f759bd24a6b0ec5d60/motionDetect?at=master&fileviewer=file-view-default which should help. Look at the instructions for this system on how to install.
LikeLike
February 12, 2018 at 10:49 pm
Hi Dani,
I was trying out your code for the voice_only and seconds part. For voice only = True and seconds = 3 it did not work out as I expected. It stays on forever if I set the “voice only” as true. I want to make the conversations more natural such that I do not need to use the hotword for next 3 sec. I tried editing the maiHotword as well but couldn’t figure out how to. Would really appreciate if you could have a look into the issue.
Cheers!
LikeLike
February 13, 2018 at 8:21 am
Hi Prakhar. The way the code should work is — if voice only is false then the box waits for the hotword. If you want voice activation (VAD) then set voice_only to true and seconds to 0. That way you will get the more natural conversations you want. If you set the seconds to any value other than 0 if there is no voice for that amount of seconds it will revert to needing the hotword. So basically only use the seconds when you want the box to go back to sleep after that amount of time. Hope that helps.
LikeLike
February 9, 2018 at 12:58 pm
You need to use the command
sudo chmod +x ‘filename’
to be able to execute it.
LikeLike
February 10, 2018 at 2:20 pm
Hi Juan,
I tried using this command as well but nothing happened. It just went to the next line.
LikeLike
February 10, 2018 at 8:20 pm
What is the “snowboy token”? I have created account and downloaded. I have the ###.pmdl etc
LikeLike
February 10, 2018 at 10:26 pm
The snowboy token is given when you create the account. If you have downloaded the pmdl file you won’t need it anymore. It is only used in order to send the recordings to snowboy and retrieve the pmdl file.
LikeLike
February 11, 2018 at 12:53 pm
Hi Dani thanks for the update I have the assistant_grpc_demo.py working with the button and and new hotname with a male voice. Tried going for voice only response but get error
env) pi@raspberrypi:~/AIY-projects-python/src/examples/voice $ python assistant_grpc_hotworddemo.py
Traceback (most recent call last):
File “assistant_grpc_hotworddemo.py”, line 4, in
import miaHotword
ImportError: No module named ‘miaHotword’
(env) pi@raspberrypi:~/AIY-projects-python/src/examples/voice $
mia hotword modified as below.
class miaHotword:
def __init__(self):
############# MODIFY THE FOLLOWING #############
model_file=’./resources/Hal.pmdl’ # put your hotword file here. if you want to just try out use ./resources/snowboy.umdl
sensitivity = 0.5
############### END OF MODIFY ##################
self.detection = miasnowboydecoder.HotwordDetector(model_file, sensitivity=sensitivity)
What am I missing?
LikeLike
February 11, 2018 at 1:57 pm
The assistant_grpc_hotworddemo.py can’t find the miaHotword.py file. Ideally they should be in the same directory with the resources a subdirectory off that. The src/examples/voice directory should be ok as a place – but is the miaHotword and other files there as well?
LikeLike
February 11, 2018 at 2:43 pm
I made sure all the files were in the voice directory BUT see below :-
(env) pi@raspberrypi:~/AIY-projects-python/src/examples/voice $ python assistant_grpc_hotworddemo.py
Traceback (most recent call last):
File “/home/pi/AIY-projects-python/src/examples/voice/snowboydetect.py”, line 18, in swig_import_helper
return importlib.import_module(mname)
File “/home/pi/AIY-projects-python/env/lib/python3.5/importlib/__init__.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 986, in _gcd_import
File “”, line 969, in _find_and_load
File “”, line 958, in _find_and_load_unlocked
File “”, line 666, in _load_unlocked
File “”, line 577, in module_from_spec
File “”, line 914, in create_module
File “”, line 222, in _call_with_frames_removed
ImportError: libpython3.4m.so.1.0: cannot open shared object file: No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “assistant_grpc_hotworddemo.py”, line 4, in
import miaHotword
File “/home/pi/AIY-projects-python/src/examples/voice/miaHotword.py”, line 3, in
import miasnowboydecoder
File “/home/pi/AIY-projects-python/src/examples/voice/miasnowboydecoder.py”, line 4, in
import snowboydetect
File “/home/pi/AIY-projects-python/src/examples/voice/snowboydetect.py”, line 21, in
_snowboydetect = swig_import_helper()
File “/home/pi/AIY-projects-python/src/examples/voice/snowboydetect.py”, line 20, in swig_import_helper
return importlib.import_module(‘_snowboydetect’)
File “/home/pi/AIY-projects-python/env/lib/python3.5/importlib/__init__.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: libpython3.4m.so.1.0: cannot open shared object file: No such file or directory
(env) pi@raspberrypi:~/AIY-projects-python/src/examples/voice $
LikeLiked by 1 person
February 11, 2018 at 2:49 pm
Hi ken. A few other people have had this issue. It is snowboy using python 3.4 and you probably using python 3.5. Look at the other comments and check with them
LikeLike
February 12, 2018 at 4:22 pm
Despite several attempts to get 3.4 loaded so far no success. Strange I can get the button demo to work with the new hotword and male voice but not the hotword demo. I’ve downloaded python3.4 per instructions but when I check version still running 3.5.
Perhaps I should start again and load an earlier image if I can find one. Any suggestions?
(env) pi@raspberrypi:~/AIY-projects-python/src/examples/voice $ LD_LIBRARY_PATH=/python3.4
(env) pi@raspberrypi:~/AIY-projects-python/src/examples/voice $ python –versionPython 3.5.3
(env) pi@raspberrypi:~/AIY-projects-python/src/examples/voice $
LikeLike
February 12, 2018 at 6:31 pm
I will try and get some instructions together to get this working as a few people have solved it. But its a bit difficult for me as I didn’t have any issues. If you have installed 3.4 you should be able to call that by using python34 followed by the file, though you may have issues if the voice kit has 3.5 specific code. But also as long as snowboy can access the library file everything should be ok.
LikeLike
February 17, 2018 at 9:06 pm
Dani been trying to run assistant_grpc_hotworddemo.py at start using systemd. I thought it might be the easiest to get started. Wrong!
sudo nano /lib/systemd/system/sample.service
[Unit]
Description=My Sample Service
After=multi-user.target
[Service]
Type=idle
ExecStart=/usr/bin/python /home/pi/AIY-voice-kit- python/src/assistant_grpc_hotworddemo.py
[Install]
WantedBy=multi-user.target
sudo chmod 644 /lib/systemd/system/sample.service
udo systemctl daemon-reload
sudo systemctl enable sample.service
Help appreciated.
LikeLike
February 17, 2018 at 10:21 pm
I’m not familiar with that method. But you will need to run the shell script that sets up the environment (dev terminal).
LikeLike
February 18, 2018 at 2:00 pm
Dani, I’ve sorted out the headless self start problem.
However I’ve noticed that on occasion it will detect “HOTWORD” nice blue light and report “not sure how to help with that” even though no hotword spoken. I can continue using hotword and it will give correct answer. Sit quiet for a while and again a phantom hotword detected and “not sure how to help with that” reported. Any ideas what could cause this.
LikeLike
February 18, 2018 at 2:19 pm
HOTWORD update it appears to be extremely sensitive to my hotword “hal”. With the TV on at the other side of the room any “H” sound triggers the hotword e.g. ha ha, or Hi etc. etc.
Other than change hotword any other ideas?
LikeLike
February 19, 2018 at 8:04 pm
There is a sensitivity setting for snowboy you could try setting. Also worth checking the recordings you did in case they missed the ending of the hot word.
LikeLike
February 20, 2018 at 5:25 pm
Hi Dani thanks for the feedback. I get false hotwords from background television noise or even a clatter of dishes can trigger the hotword followed by “I’ don’t understand”. I’ve listed my miasnowboydecoder.py showing the adjustments I’ve tried.
#!/usr/bin/env python
import collections
import snowboydetect
import time
import wave
import os
import logging
logging.basicConfig()
logger = logging.getLogger(“snowboy”)
logger.setLevel(logging.INFO)
TOP_DIR = os.path.dirname(os.path.abspath(__file__))
RESOURCE_FILE = os.path.join(TOP_DIR, “resources/common.res”)
class HotwordDetector(object):
“””
Snowboy decoder to detect whether a keyword specified by `decoder_model`
exists in a microphone input stream.
:param decoder_model: decoder model file path, a string or a list of strings
:param resource: resource file path.
:param sensitivity: decoder sensitivity, a float of a list of floats.
The bigger the value, the more senstive the
decoder. If an empty list is provided, then the
default sensitivity in the model will be used.
:param audio_gain: multiply input volume by this factor.
“””
def __init__(self, decoder_model,
resource=RESOURCE_FILE,
sensitivity=[], (Tried changing parameter from 0.1 to 1)
audio_gain=1):
tm = type(decoder_model)
ts = type(sensitivity)
if tm is not list:
decoder_model = [decoder_model]
if ts is not list:
sensitivity = [sensitivity]
model_str = “,”.join(decoder_model)
self.detector = snowboydetect.SnowboyDetect(
resource_filename=resource.encode(), model_str=model_str.encode())
self.detector.SetAudioGain(audio_gain)
self.num_hotwords = self.detector.NumHotwords()
if len(decoder_model) > 1 and len(sensitivity) == 1:
sensitivity = sensitivity*self.num_hotwords
if len(sensitivity) != 0:
assert self.num_hotwords == len(sensitivity), \
“number of hotwords in decoder_model (%d) and sensitivity ” \
“(%d) does not match” % (self.num_hotwords, len(sensitivity))
sensitivity_str = “,”.join([str(t) for t in sensitivity])
if len(sensitivity) != 0:
self.detector.SetSensitivity(sensitivity_str.encode())
LikeLike
February 24, 2018 at 9:55 pm
Dani I’ve manged to reduce the sensitivity to eliminate false ‘hotwords’.
Also to enable a headless shutdown.
The only other change that I would like to do is change the voice to GB, English, Welsh, Geordie anything.
Have you attempted any change to the US voice. I’m running the male US version at the moment.
LikeLike
February 24, 2018 at 10:06 pm
Glad you have made progress. I recall changing the setting to recognize gb english. I can’t remember off hand where I changed it. As for the voice returned by google I’m not sure. I will try and look for both these things and get back to you.
LikeLike
March 7, 2018 at 8:49 pm
I’m using Google Voice AIY kit. Do you know how to make a kit response by voice (and light up the button) when we say a wake word? example: when we say ‘OK, Google’, the kit will response with voice ‘OK’ or ‘Yes’ and light up the button as well.
I want to copy from Alexa with Pi project, the kit response with voice ‘Yes’ when we say a wake word ‘Alexa’. Thanks
LikeLike
March 7, 2018 at 9:57 pm
Yes, I do that in the assistant_grpc_hotworddemo.py by setting the status_ui.status(‘ready’) or ‘listening’ etc. The various options turn the light on or flash it slowly or rapidly. I forget exactly what each one does, but you should be able to work it out fairly quickly. And the wake word – if you check this post and the associated code you should be able to do it.
LikeLike
March 16, 2018 at 5:46 am
I’m having a heck of a time trying to get this to work. Trying to run the grpc hotword program on the dev terminal gives me
(env) pi@raspberrypi:~/AIY-voice-kit-python/src $ python assistant_grpc_hotworddemo.py
Traceback (most recent call last):
File “/home/pi/AIY-voice-kit-python/src/snowboydetect.py”, line 18, in swig_import_helper
return importlib.import_module(mname)
File “/home/pi/AIY-voice-kit-python/env/lib/python3.4/importlib/__init__.py”, line 109, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 2254, in _gcd_import
File “”, line 2237, in _find_and_load
File “”, line 2226, in _find_and_load_unlocked
File “”, line 1191, in _load_unlocked
File “”, line 1161, in _load_backward_compatible
File “”, line 539, in _check_name_wrapper
File “”, line 1715, in load_module
File “”, line 321, in _call_with_frames_removed
ImportError: libf77blas.so.3: cannot open shared object file: No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “assistant_grpc_hotworddemo.py”, line 18, in
import miaHotword
File “/home/pi/AIY-voice-kit-python/src/miaHotword.py”, line 18, in
import miasnowboydecoder
File “/home/pi/AIY-voice-kit-python/src/miasnowboydecoder.py”, line 17, in
import snowboydetect
File “/home/pi/AIY-voice-kit-python/src/snowboydetect.py”, line 21, in
_snowboydetect = swig_import_helper()
File “/home/pi/AIY-voice-kit-python/src/snowboydetect.py”, line 20, in swig_import_helper
return importlib.import_module(‘_snowboydetect’)
File “/home/pi/AIY-voice-kit-python/env/lib/python3.4/importlib/__init__.py”, line 109, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
ImportError: libf77blas.so.3: cannot open shared object file: No such file or directory
(env) pi@raspberrypi:~/AIY-voice-kit-python/src $
What is happening?
LikeLike
March 16, 2018 at 5:58 am
Never mind. I got it to work, I simply forgot to download libatlas.
Also, I finally found the image version that lets the current version on your website work. You have to use their build dated on 9-11, which still uses Python 2.4. I’ll link it here:
https://github.com/google/aiyprojects-raspbian/releases/tag/voicekit
LikeLike
March 16, 2018 at 12:46 pm
Glad you got it working and thanks for the link. To clarify – aiy kit runs on python 3 – that version uses python 3.4 which works with the snowboy bindings. Unfortunately later versions of the kit use python 3.5 which isn’t compatible with snowboy.
LikeLike
April 17, 2018 at 10:31 pm
Hi Dani, i still trying use the image of Google AIY ( Feb 2018), with Python 2.7 and work correctly, in italian language. I downloaded ur files on src folder, modified the miaHotword.py file, but when run the assistant_grpc_hotworddemo.py file, i get error:
File “assistant_grpc_hotworddemo.py”, line 18, in
import miaHotword
File “/opt/aiy/projects-python/src/miaHotword.py”, line 17, in
import queue
ImportError: No module named queue
Can u help me?
Ty very much for ur work
LikeLike
April 17, 2018 at 9:48 pm
For python 2.7 use import Queue with capital Q. I think you may have problems with the voice kit and python 2.7.
LikeLike
April 18, 2018 at 2:22 pm
I have python 3.5.3 too in my Raspberri Pi 3 B. Google AIY Voice kit work without problems. If i use Queue, i get other errors 😦 :
File “assistant_grpc_hotworddemo.py”, line 18, in
import miaHotword
File “/opt/aiy/projects-python/src/miaHotword.py”, line 18, in
import miasnowboydecoder
File “/opt/aiy/projects-python/src/miasnowboydecoder.py”, line 17, in
import snowboydetect
File “/opt/aiy/projects-python/src/snowboydetect.py”, line 21, in
_snowboydetect = swig_import_helper()
File “/opt/aiy/projects-python/src/snowboydetect.py”, line 20, in swig_import_helper
return importlib.import_module(‘_snowboydetect’)
File “/usr/lib/python2.7/importlib/__init__.py”, line 37, in import_module
__import__(name)
ImportError: libpython3.4m.so.1.0: cannot open shared object file: No such file or directory
LikeLike
April 18, 2018 at 6:51 pm
Hi Luca. A few people are having problems with this. The simplest way to get it working is to use the image from here https://github.com/google/aiyprojects-raspbian/releases/tag/voicekit which uses python3.4. I’m sure there is a way of using later versions as long as the correct libpython is put in the path but until someone gets it working we don’t know.
LikeLike
April 19, 2018 at 10:49 pm
Hey Dani!
Thanks for the great piece of work.
After I execute assistant_grpc_hotworddemo.py, I receive the following error after the hotword is detected :
Traceback (most recent call last):
File “assistant_grpc_hotworddemo.py”, line 58, in
main()
File “assistant_grpc_hotworddemo.py”, line 42, in main
text, audio = assistant.recognize()
File “/home/pi/AIY-voice-kit-python/src/aiy/assistant/grpc.py”, line 49, in recognize
response = self._request.do_request()
File “/home/pi/AIY-voice-kit-python/src/aiy/_apis/_speech.py”, line 255, in do_request
service = self._make_service(self._channel_factory.make_channel())
File “/home/pi/AIY-voice-kit-python/src/aiy/_apis/_speech.py”, line 367, in _make_service
return embedded_assistant_pb2.EmbeddedAssistantStub(channel)
AttributeError: ‘module’ object has no attribute ‘EmbeddedAssistantStub’
Any suggestions on the same?
LikeLike
April 20, 2018 at 7:17 am
Hi Shivam
I don’t really know, though it looks like something with the aiy kit. Can you run the normal demos that come with the kit? Are you running the script to set up the environment?
Dani
LikeLike
April 20, 2018 at 9:16 am
The demos seem to run normally. I’ve tested them for a headless start as well.
Even after executing HotwordDemo.py, I receive an error.
Following is the error (if you’ve any clue of what might be the issue) :
(env) pi@raspberrypi:~/AIY-voice-kit-python/src $ python HotwordDemo.py
/home/pi/AIY-voice-kit-python/src/aiy/_drivers/_led.py:51: RuntimeWarning: This channel is already in use, continuing anyway. Use GPIO.setwarnings(False) to disable warnings.
GPIO.setup(channel, GPIO.OUT)
Traceback (most recent call last):
File “HotwordDemo.py”, line 90, in
main()
File “HotwordDemo.py”, line 24, in main
recognizer = aiy.cloudspeech.get_recognizer()
File “/home/pi/AIY-voice-kit-python/src/aiy/cloudspeech.py”, line 83, in get_recognizer
_cloudspeech_recognizer = _CloudSpeechRecognizer(CLOUDSPEECH_CREDENTIALS_FILE)
File “/home/pi/AIY-voice-kit-python/src/aiy/cloudspeech.py”, line 35, in __init__
self._request = aiy._apis._speech.CloudSpeechRequest(credentials_file)
File “/home/pi/AIY-voice-kit-python/src/aiy/_apis/_speech.py”, line 283, in __init__
credentials, _ = google.auth.default(scopes=[self.SCOPE])
File “/home/pi/AIY-voice-kit-python/env/lib/python3.4/site-packages/google/auth/_default.py”, line 277, in default
credentials, project_id = checker()
File “/home/pi/AIY-voice-kit-python/env/lib/python3.4/site-packages/google/auth/_default.py”, line 138, in _get_explicit_environ_credentials
os.environ[environment_vars.CREDENTIALS])
File “/home/pi/AIY-voice-kit-python/env/lib/python3.4/site-packages/google/auth/_default.py”, line 63, in _load_credentials_from_file
with io.open(filename, ‘r’) as file_obj:
FileNotFoundError: [Errno 2] No such file or directory: ‘/home/pi/cloud_speech.json’
LikeLike
April 20, 2018 at 10:42 am
This looks like you haven’t set up the aiyvoicekit completely. Check here https://aiyprojects.withgoogle.com/voice – the section on Create a service account and credentials. You should have a cloud_speech.json in your home directory.
LikeLike
April 20, 2018 at 1:12 pm
Thanks a lot!
I’m able to execute HotwordDemo.py properly but the assistant_grpc_demo.py still returns the same error.
LikeLike
April 20, 2018 at 7:01 pm
It will be something with the voice kit setup. I’d go back through that.
LikeLike
May 7, 2018 at 1:28 pm
Thanks, Dani. This is a great post!
As many have already reported here, I got the error “ImportError: libpython3.4m.so.1.0: cannot open shared object file: No such file or directory” I spent a day trying to fix it, but eventually gave up and got the image version you posted about: https://github.com/google/aiyprojects-raspbian/releases/tag/voicekit
I had one more issue after that. Even after activating the virtual env by starting the dev terminal, the AIY library could not be loaded. To solve it, I changed the path for the AIY library.
echo “/home/pi/AIY-voice-kit-python/src” > /home/pi/AIY-voice-kit-python/env/lib/python3.5/site-packages/aiy.pth
And with that, all is working now.
LikeLike
May 7, 2018 at 9:42 pm
Thanks for the info Valentin.
LikeLike
May 11, 2018 at 2:42 am
Will it work in raspberry pi zero w
LikeLike
May 11, 2018 at 5:41 am
I’m not sure the voice kit will because it uses a voice hat. My stuff and snowboy should work.
LikeLike
May 13, 2018 at 6:33 pm
I want to install google assistant and snow boy for different hotword can you please tell me what can I do in raspberry pi zero w
LikeLike
May 13, 2018 at 9:41 pm
Well the voice kit software assumes you have the full voice kit including the voice hat and button. The voice hat sits on top of the raspberry pi’s pins. So if you are going to do this you would need to solder the pins on the Pizero. First thing you need to check is whether the voice hat is compatible with Pizero. Otherwise maybe you could install a different microphone. As long as you can get it to work with arecord the google assistant software would probably work, but will be a bit fiddly. After that you just need to follow the instructions in this post and with the code. Please remember to use the version that works with snowboy. Hope that helps.
LikeLike
June 12, 2018 at 4:09 am
Thanks for your tutorial. After making some change, I am able to create custom hotword trigger for AIY voice kit version2.0, with aiyprojects-2018-04-13.img.
LikeLike
June 12, 2018 at 7:46 am
Thanks James. Any chance you could send what the change was so that other people can do the same.
LikeLike
June 16, 2018 at 2:18 pm
Hi Dani,
I’m not novice, but had been struggling to integrate voice hat/google assistant/snowboy, then I can across this post. I followed your method and… it works, first time. Thank you so much for this – I’d almost given up with the unfathomable complexity!
Cheers, Jonathan.
LikeLike
July 19, 2018 at 7:52 pm
Reagrding the “ImportError: libpython3.4m.so.1.0: cannot open shared object file: No such file or directory” error,
swig 3.0.10 is now available in raspbian stretch (9.0) repo, so it is possible to compile python3 wrapper.
LikeLike
July 20, 2018 at 7:16 am
Thanks for this info. Will have to look into this.
LikeLike
February 23, 2019 at 2:52 pm
just replaced wifi router and now have to input wifi key every time raspberry rebooted.
LikeLike
February 23, 2019 at 3:48 pm
You shouldn’t have to. The wifi details are normally stored in the wpa_supplicant file though this can vary. Have a look at this guide which should help you https://www.raspberrypi.org/documentation/configuration/wireless/wireless-cli.md
LikeLike
March 7, 2019 at 4:35 pm
I’ve manually setup the supplicant.conf file as per instructions. All looks ok I then ran wpa_cli -I wlan0 reconfigure. That works ok. I’ve then checked the supplicant file again and both old router and new router details are shown. However when I switch off the Pi and then reload the Raspberry Pi won’t load onto the new router. I then recheck the supplicant.conf file and the new router details have disappeared. The supplicant.conf file looks exactly as it was before modification. Yes I’ve saved the file at exit and gone back in to check it has saved but when unit rebooted or switched on/off the file returns to type. Any ideas???
LikeLike
March 7, 2019 at 4:41 pm
I’m not sure. Check the file timestamp in case it didn’t save properly. Make sure you edit using sudo so you have the correct perms. I would also remove the old router details from the file. Though keep a copy just in case.
LikeLike
March 7, 2019 at 5:06 pm
Yes definitely used sudo. Also tried removing old router and saving only with new router details. Provided I don’t switch off I can work on the web no problem. Switch off and on and the supplicant.conf has removed new router and reinstalled old router even though it’s switched off.
LikeLike
March 8, 2019 at 11:39 am
This is a bit strange. Check the timestamps before and after. I can only assume that during the boot the original file is copied over. Check that there isn’t another file with the same name (or similar) somewhere else. If that fails you might be able to set the permissions on your updated file to read only which may then write something to the error logs.
LikeLike
March 7, 2019 at 4:47 pm
Another thought. Perhaps there is some problem with the file so the pi has reverted to previous version.
LikeLike
April 8, 2019 at 4:45 am
Is it still compatible with the latest AIY image? Thanks!
LikeLike
April 8, 2019 at 7:28 am
Unfortunately the snowboy version I have isn’t compatible with later versions of the aiyvoicekit. I think it is only a matter of setting paths in the environment to reference the libpython version. I will have a look to see if there is anything changed on this recently.
LikeLike
April 10, 2019 at 2:21 pm
Thanks so much! Really appreciate your reply!
LikeLike
April 4, 2020 at 1:38 pm
Anyone know why the google voice on the raspberry Pi no longer knows the time, day of the week or date.
All I get is “sorry can’t help with that yet”. All other questions answer ok including an explanation of the Coronavirus.
Stay well
LikeLike
May 25, 2020 at 8:31 pm
great issues altogether, you just gained a new reader.
What could you suggest in regards to your
put up that you just made a few days ago? Any certain?
LikeLike