Raspberry Pi smart glasses translate in real time
Will Powell read about Google’s Project Glass, and decided he’d have a go at a DIY attempt to try to achieve something similar. And what he’s come up with is downright remarkable. Here’s his automatic translation system, which uses a couple of Raspberry Pis, a 3d headset, some microphones, a TV and an iPhone to display real-time subtitles in your glasses as you have a conversation.
Fantastic! That would be of great use to deaf people too for real-time language to subtitle provision.
Deaf people usually do just fine reading lips. At least, the three deaf people in my family are. ;-)
I am going deaf slowly, at the age of 51. This could be amazing for me in crowded spaces where all the audio turns to mush!
Brilliant stuff. The word fantastic is rarely used appropriately but in this case it certainly seems ‘out of this world’!
Finally, It actually feels like I’m living in the future. This is the kind of thing I was promised when I was a kid. Next stop hover-cars!
I was thinking about something like this the other day, but in my mind I could see someone looking at a menu in a foreign language and it’d change the words on the menu to English.
It exits as a iPhone app, so all we need is someone to incorporate it into some glasses and were there!
If it is possible for an iPhone to output video than it could be paired with an iTheatre type Glasses… Make sure you get the ones with better resolution.(800×600 or more) Some may come with an iPod/iPhone adapter.
Yes, I’ve got a pair of my own but just the 320×240 rez… :/
And even more cool would be to read a news paper and the pictures of the news paper would “come-alive”, like shown in this TED talk
This would be extremely cool, until the advertising companies would enter the game.
Wow! At present, if the police detain a non-English-speaking foreigner, they have to figure out what language they speak, then arrange an interpreter. Maybe in the future all they’ll have to do is put on their glasses…
While I don’t doubt it’s genuine, it’d obviously be very easy to ‘spoof’ a video like that ;)
The RaspberryPi(s) are “only” being used as video output, IMHO it’s the reliable speech recognition (the article doesn’t mention how/where that part’s being done?) that is the impressive part. But it’s a really nice demo of “plugging together” different bits of technology to create something impressive :)
I think you’re using the term “real-time” a little loosely here. But it’s definitely a step in the right direction.
*this post has been edited by Raspberry Pi to remove the personal data of another user at the request of that user.
I am gonna give this idea a Thumbs Up! Universal Translators are not Sci-Fi anymore? Great Idea!
Next up is to shrink this setup down to the size of a combadge and the future is here.
Is this for real? Has rt voice recognition and translation really become this good?
How is the speech-to-text processing being handled? Thanks!
For anyone who wants to know the “Magic” that is behind this idea!
Via: YouTube Channel…
“Published on 22 Jul 2012 by Will Powell
Inspired by google’s project glass I put together an application that can provide translated subtitles in real time. It allows me and my sister Elizabeth to have a conversation when I speak English and she speaks Spanish.
It uses two Raspberry Pi boards, Vuzix Star 1200, Jawbone mic. Audio is picked up and streamed from mobile device to a server that recognises, translates (uses microsoft translate API and caching layer) and pushes back. One pi drives the glasses and the other the TV that shows both halves of the conversation. Elizabeth uses a mic but would use the same Vuzix, Jawbone, Pi setup if I had another set of kit.
Find out more at http://www.willpowell.co.uk or follow me @WillPowellUK”
All it says is “a server that recognises (sp)” the speech. But what is doing that recognition?
would you be willing to write something for the magpi to tell us how you did the project?
If this device can understand (let alone translate) the Spanish this girl is talking is better than me and I am native from Spain .
Being deaf myself, that would be marvellous just to be able to read in my spectacles what people are saying!!
Awesome project, but just for the record: Is well known that in Spain we have bullfighting and it is usually to see flags with the “spanish bull” on in. I have to say (as a spanish guy) that most people living in spain (90%+) hates bullfighting and see as an old tradition that should be removed. I don’t want to start a debate pro/con bullfighting but I just want to be clear that a lot of spanish people hate this “bull” on our flag. Congratulations for the projects and keep doing things like that ;)!
Whats the song name in that YouTube clip?
Thanks Liz, I thought this might not make it here, your the best.
‘Only’ $5000 for the glasses? Seems a bit pricy… :-/
The girl’s Spanish is terrible but….
ARE YOU FREAKIN’ KIDDING ME?
Great, honestly great project! It’d be interested to know what the TV and iPhone (actually it was an iPad I believe) do… Is the translation done by the Pis?
What is the small display the other Pi is connected to? Has it got a touchscreen?
Absolutely amazing. I too am somewhat deaf and constantly asking my kids to speak up drives them nuts. It would be nice to have these built into my prescription glasses so I could see subtitles when people talk. The time it would save me in confusion would be incredible! Kind of in that limbo of not truely 100% deaf but just to the point its annoying. I can’t wait. Hope its in my lifetime.
Comments are closed