Monty, a Raspberry Pi personal assistant
Liz: James Gallagher is 13 years old. He sent me a little bit of information about a voice-activated personal assistant project he’s been working on: I thought it was so neat that I asked him to write about it here. Over to James!
I envisioned an idea where you could rather than controlling your computer with a mouse, you can do it with your voice. This got me thinking, Siri is available for Apple devices and Cortana is available for Windows devices but what about Linux? So I embarked on a project to attempt to bring this ability to the Raspberry Pi, I know that it would take a long time and might not happen however I stuck at it and finally created a product. Meet “Monty”! Named The “Monty” Project, I made a piece of software where you could interact with your Raspberry Pi via Voice. So far, you can execute some basic commands that would normally be used in terminal and you can exchange pleasantries.
Some basic commands include asking it about the news and weather, some mathematical questions, launching programs and executing commands normally done through terminal. Also adding items to Crontab and lots lots more! This program takes advantage of the Web API’s available from Wolfram Alpha and RSS Feeds to send data to the program where it then speaks what you asked. Below you will see a video demo of the software in action.
https://www.youtube.com/watch?v=bPopD74xbc0
I created this to provide an experience I couldn’t find anywhere else and to help others use computers or just have some fun. I showcased “Monty” at my school and allowing others to attempt to speak to my Raspberry Pi and proved effective. I hope to attend some Raspberry Pi conferences to show it off to the public and give others the chance to use it before release. When the software is released there will be an API with full documentation so that it will be easy for you to configure the software and are it your own.
Thanks for reading and have a nice day!
Note: Right now this software is still in Beta and not publicly available, the release window is July-December 2016. There are more features I hope to implement and still quite a few bugs to iron out. Please follow me on Twitter to keep up to date with the software. @James1Gallagher
36 comments
solar3000
Waaa! Reminds me of windows 95 and clippy! Then office assistant. And now we have suri, cortana. I forgot what google’s version is.
Well good luck lad. Another young brilliant engineer.
Seems like a lot of engineers coming from those islands.
dobra-dobra
Great job! Keep it up, but consider putting your code on GitHub to grab some more attention and possibility to cooperation.
John-Paul
Great job. I could never have imagined accomplishing something so complex when I was 13, pretty sure I still can’t do it. Though you might want to investigate the Jasper project as well as the Alexa Voice Service from Amazon before you get to much further. Could save you some time in implementing your additional features.
@Solar3000, Googles version is just “Ok Google” I believe, no real name for it.
Ian Minara
Google Now
James Gallagher
Thanks you guys for all the positive feedback, the code is going on github soon and I will look into Jasper. Thanks again, have a nice day!
James Gallagher
Just noticed, I made a mistake at the end. It should say: “When the software is released there will be an API with full documentation so that it will be easy for you to configure the software and make it your own.”. I just noticed.
Connor
Hey, I’m 13 years old as well! What you have made here is awesome, and could change things forever. Carry on with this, I think this’ll go places!
Bruce Berryhill
Hello MR. Gallagher,
Will your program be able to execute commands for a .MP3 player? I have an older brother who has mental retardation. He doesn’t speak clearly, kind of muddles his words. Although he is physically 52 years of age his mind is roughly equivalent to 4 to 10 years of age. He has been going blind but he dearly loves music. I have engaged the services of an online friend to come up with a .MP3 player for my brother using Raspberry Pi. When your software is done can it be used as the primary interface when the computer is turned on without having to do anything else?
Cheers,
Bruce Berryhill
Game Designer, Writer, Fractal Artist
James Gallagher
I am working on .mp3 integration with it however I am having a few troubles. It is a shame what happened, that just makes me want to work harder to get this out.
AndrewS
I’m not sure how much work it’d be, but I wonder if you could integrate with http://www.freedesktop.org/wiki/Software/DbusProjects/ in order to allow Monty to control other programs?
AndrewS
You should be able to adapt the audiobook player featured here into a music player?
https://www.raspberrypi.org/blog/sunday-grab-bag/
Joel
It was great! Looking forward to seeing more great things coming from you, James.
Shannon Spurling
This was awesome! It would be nice if some of the cleaner text to speech systems were available for Linux/Pi, so the voice could be less robotic. I would like to know if this uses local voice processing, or if it uses the cloud based voice to text systems? For portability sake, it’s nice to have local voice recognition, but understand that the cloud systems can bring in so much more power and accuracy to the problem. Either way, this is so awesome! Looking forward to hearing more of this in the future!
James Gallagher
Well, I am running a Ubuntu Server on a Raspberry Pi but only as a backup of all files. I thought that cloud implementation would just take up too much time I don’t currently have. I am going to look into it just after I have worked on the API. It wouldn’t be fun to download the tool and download lots of huge files just to make it work. Do you have any cloud service recommendations I could use for files? As for the voice, I couldn’t program something that didn’t sound robotic so I stuck with the current voice. Thanks for the feedback, I will take cloud file and voice hosting into consideration. Have a nice day!
James Gallagher
I must mention, I did not know there was voice recognition programs available already so this was independent until you told me about Jasper and espeak. I shall look into them just to see what they are all about. Have a nice day.
AndrewS
James, you might also find some useful pointers in these blog posts:
https://www.raspberrypi.org/blog/books-the-digitising-and-text-to-speechifying-thereof/
https://www.raspberrypi.org/blog/build-a-universal-translator/
https://www.raspberrypi.org/blog/raspberry-pi-smart-glasses/
Keep up the good work!
Shannon Spurling
So, are you using the pocket Sphinx package? I found that really cool, but it had issues with noise on my older model B’s and the USB mics that I had. The cloud based voice recognition is usually done with the google engine. There’s an API. There is also an engine through Apple, I think. You just send the WAV file up to the system, and it sends back the text file to be parsed. There are tutorials on that. It just means the Pi has to be tied to the network to work. I work on the Internet, I rather my things not be dependent on it. (Tw/oI) :-)
J
Excellent job! Another great example of the vast functionality of the Pi.
DGA110P
looks (Sounds) good – is this an off shoot of Jasper / Jarvis ?
using espeak ?
will look forward to seeing progress on this
Excellent Job
Samer Najia
Outstanding work. Looking forward to seeing it in full glory and then playing with it myself.
Fester Bestertester
Lovely! What extra hardware is necessary? Getting live audio *into* a pi ain’t the easiest task without.
G10DRAS
Another Jasper ?
https://www.youtube.com/watch?v=UzaqNF6NlBA
Take a Look at Jasper Forum, if you want to build a similar project….
https://groups.google.com/forum/#!forum/jasper-support-forum
Jasper Source Code
https://github.com/jasperproject/jasper-client
Rory Jackson
Fantastic work James. I will show this to our coderdojo club and know it will inspire them.
We are using rasberry p’s in the club for advanced coding.
Looking forward to seeing your progress with the voice project
Rory
MalMan35
Wow. I am 14 and got into coding a little over a year ago. To see someone younger than me that has done something that I have no clue how to do is very humbling. I wish you the best in your endeavors and I hope that some day I will be able to do something just as amazing.
P.S. You can click on my name above to see what I would consider my biggest project.
Pete Stevens
You have no idea how those of us three times his age feel!
t3chnico
Absolutely fantastic! When I was 14 years old, I played video games.
Great job!
drew
James or other interested parties, you may want to take a look at Mycroft.
https://mycroft.ai
It is a fairly new open source project that does language analysis etc. It has been touted as the next generation of speech to text and actions for Linux. e.g. a competitor to Siri/ Cortana/ Alexa etc.
I think Mycroft’s analysis is done in the cloud when used on the Pi, but that may change.
I have no idea how you have done this so far, good work & good luck :)
William AC0mputerGuru
Absolutely fantastic job, James!! It doesn’t matter if your project parallels the Jarvis project or others…it spoke to you (literally) and you made it happen! It would be lovely to see you continue with this work, either independently or in conjunction with others like Jarvis. As each programmer has their unique style and way of interpreting a problem, you have the ability to analyze, theorize, and capitalize on Monty. Again, very impressive, ESPECIALLY at your age! Good work and best wishes in everything you pursue :-)
vsilent
Very nice, good job !
I started a similar project https://github.com/vsilent/smarty-bot 2 years ago, and I see people interested in this :) . That’s motivates me to continue !
PBM
There is also the similar Gladys project (http://gladysproject.com/)
Johan Furuskog
Good work!
Please continue with the good work. It would be nice to see your work on github, because I want to learn about it. Dont worry about it not beeing finished or unpolished we are all here to learn! Even a 36 year old guy like me. Best wishes James!
James Gallagher
If I create something and wouldn’t use it myself then I would not release it. I do use “Monty” but at the moment everything is done in the least productive way possible. I am at the cleanup stage right now, to this point i just wanted it to work but now whilst cleaning I am finding more obstacles to ruin me. That is the way I have always been but I am considering a private beta for a limited amount of people to join and test then I can release it. Follow me at @James1Gallagher if you don’t already to get more info as it happens. Thanks for the advice and I hope you have a nice day.
MARKSE
Sounds nice! Are you using an API to reach an external speech to text service or doing it all local to the Pi? I’m building a Jasper system at the moment (hooking up a high quality mic to it this weekend) to play with. I want to use the Julius STT engine so all the processing is local but it’s a pain to install. Best of luck with your project.
dunnoor
heeyy greaattt project!! can i have the step by step process along with the codes???if u dnt mind
Jeff Proehl
Very nice work, congratulations.
I have an interest that might use a stripped down version of your code. I have a remote activated trolling motor on my fishing boat that uses a small hand-held radio control with 5 buttons to Stop, Slow down, Speed up, Turn right and Turn left. I would like to implement a voice control so I do not have to stop fishing to push buttons. The simplest thing would be to get another hand control (which the trolling motor has to recognize) and disassemble it and wire the button circuit switches to an RPi that would activate the buttons based on 5 voice commands (Halt, fast, slow, right, left or H,F,S,R and L if that’s easier). Would it be difficult to modify your code to do such a thing? Thanks!
Dimitris Nasoufis
I want the program code :P
Comments are closed