The view from Gordon’s helmet
Liz: if you haven’t entered our contest to win a pre-production camera board, have a look at the post explaining what you’ll need to do. And if you’re looking for inspiration, here’s a guest post from Gordon, our Head of Software, about a mini-HD camera project he worked on at home using the prototype boards we showed the BBC back in 2011.
I may have mentioned that Gordon does a lot of cycling. He bodged up a 3D helmet cam a couple of years ago: here’s how he did it. (He has also made me include some 2D video because he likes showing off.)
Careful with the last video, which is in 3D – if you’re using bi-coloured 3D glasses to view it, as I did, you are liable to feel VERY motion sick if you’re susceptible to that sort of thing. Over to Gordon!
A few years ago I really wanted to play around with a helmet-mounted camera for my mountain biking. There were quite a few out in the market, but they were quite expensive, and it’s always difficult getting toys past my wife! Because I was working at Broadcom, I was able to get my hands on what we called the MicroDB (the thing David and Eben first showed to the BBC as the Raspberry Pi), and since I had all the software and a bit of competence, I decided to try doing a bit of HD helmet recording.
The hardware I used was based on the same BCM2835 chip that we all know and love. The hardware also had a PMU chip (power supply), which meant you could power it directly from a lithium ion battery and record 720p HD video for about an hour.
So I rigged up some properly engineered mounting. I used a rubber from my daughter’s pencil case (Americans, breathe easy – this is the UK word for what you call an eraser), a couple of cable ties, and a USB socket! I set out on a voyage of discovery…apologies in advance for the lycra clad arses, but It’s something you’ll just have to put up with!
Liz interjects: that’s not the half of it. Eben and Gordon have a regular date on Wednesdays where they take an hour and a half over lunch to go cycling and have a software meeting at the same time. This means a certain amount of strutting sweatily around the office dressed in lycra at the end of the ride. This week, Jack turned up, tutted and said: “You two do realise there are showers downstairs, don’t you.” The rest of us cheered.
This is an example of the helmet cam being used in a chain gang, which is a fast-moving (we’re doing around 26mph average for the whole of the clip) club ride, where you continuously rotate who’s cycling at the front, making it a very efficient way of travelling at speed!
This is another clip from the helmet cam, at the start of a mountain bike race held by a good friend of mine who’s an elite rider.
When I took these videos, I expected to experience the same feeling of speed as when you’re riding for real, but it doesn’t quite make it. The main issue is that the feeling of speed you get is a product of the full 3D stereoscopic experience that the 2D camera throws away. It’s there and it’s fun, but it doesn’t actually feel real; you don’t quite get the full-force feeling of what it’s like to tear down that trail!
I was missing a dimension, so I had to go find it again! OK, now you ask, surely it’s going to cost me a lot of money to buy a proper 3D camera, and you’d be right if you didn’t have a whole bunch of little camera boards kicking around in the office. I realised that all I needed was two of them, and a spot of work to synchronise the pictures: then Bob’s your uncle!
I took two MicroDB’s and connected them together (actually I used a USB -> USB connector which I then cable-tied to my bike helmet with a rubber/eraser to give it something soft to sink into). So what you get out is two videos (each 720p30). To get the images working together, you need to do some processing, which presents a number of problems:
1) The two cameras are not aligned and therefore you have to rotate and translate the images.
2) You also need to invert one of the images.
3) You need to hand-synchronise the two videos (and keep them synchronised during the video).
So I wrote a bit of software based on FFMPEG and SDL, and lots of handcrafted fun code to take the two videos and output them as one in a number of formats, including interleaved line (odd lines are left image, even lines right), horizontal half-resolution and vertical half-resolution (because we had a number of different 3D televisions to play with!) Application of Bresenham’s algorithm is so much fun!
I then went and did a 24-hour mountain bike race in a team of five (we came third that year) and recorded the first half of one of the laps in glorious 3D. You are going to either need a proper 3D television to watch this or use some red/green (actually cyan is closer) glasses (the kind you get in breakfast cereals!) – otherwise you can just hold two bits of suitably coloured filters against your face.
Liz again: editing this post, I have realised that the next video gives me motion sickness even without Gordon’s 3D glasses. Proceed with caution. Gordon, I can’t believe you kept this stuff up without sleeping for 24 hours.
Why am I showing you this? Well mostly because I had so much fun doing it, and it really shows how the real 3D helmet cameras can make the experience of home video just so much better if you’re doing something fast and aggressive. I hope you agree!
Finally, of course, the Raspberry Pi camera (now in production and being released next month) is very closely related to this one – although it’s actually higher quality; the images we’ve been seeing in test are looking fantastic. This project gives you an impression of the kind of thing you’ll be able to do with it with a bit of extra coding – and of the sort of extra legwork we’re looking for from people entering the competition to win a pre-production camera board.
I’ve been looking for where I put the video manipulation code; if I can find it, I’ll put it into GitHub somewhere so you can have a play yourself (if anyone is remotely interested)!
Finally – really finally – you have to think about the fact that the Raspberry Pi has two CSI interfaces, meaning there’s a potential to add two camera boards. Does that mean it would be possible to do all this completely on a single Raspberry Pi? We haven’t experimented with the idea yet – only the future can tell…
45 comments
Mike Kopack
Wait, how/where are there 2 CSI interfaces on the Pi ???? I only remember there being the 1 CSI connector….
Or is the DSI one also able to be used for CSI if we so choose? Or does the single CSI connector allow for 2 cameras to be attached??
Gert van Loo
There are two CSI interfaces on the BCM2835 but only one is brought out to a connector on the Pi. The other interface is hidden between a plethora of BGA balls underneath the chip.
Andrew Scheller
http://elinux.org/RPi_BCM2835_Pinout if you’re curious… ;-)
The RasPi’s CSI connector connects to the CAM1_ pins, so the CAM0_ pins are the unused CSI interface.
Gordon
Someone asked about how difficult it was to synchronise the two cameras… Actually each frame is time stamped with a micro second accurate time therefore you just use the left frame as the master and choose the closest (in time) right frame to go with it…
The more interesting code came next… You know I said I connected the two camera’s through a USB socket to socket connector to attach to the helmet, you wonder why I didn’t bother sync the two camera’s using the USB… That’s exactly what I did next, so I set one of the USBs and a host and one as a device and sent a short message from host -> device every frame, depending on when it arrived I either reduced or increased the blanking interval on the device’s camera sync signals so that over a couple of seconds the two cameras would sync within about 1ms (which is easily enough to get rid of the ‘wobble’ between left and right frames.
The main issue with using the new camera’s to do this is that they are using a rolling shutter, this means that when you have fast moving scenes you need as much of the image to be exposed at the same time otherwise you get ‘wibble’ (you’ll know what I mean when you see it!) This means we need to very carefully choose the frame rate and resolution to make it work… Shame they don’t make the same camera anymore it was lovely!
Gordon
Andy
“…so I set one of the USBs and a host and one as a device…”
Interesting
Jim Manley
Oh, really? There have been rumors that the USB controller could be set to client/slave mode, but I don’t recall it being documented. How do we do it?
Gordon
This was done with Jungo drivers on the GPU not the full Linux driver… But I believe the gadget driver should work on the model A
JBeale
“…time stamped with a micro second accurate time…” Do the two CPU clocks stay synchronized to the microsecond over the length of a bike race? Or the continuing USB packets are what maintain sync? Even with rolling shutter, if every frame start time is synchronized, then each sub-part of the frame should be as well, I would think?
Gordon
I’m just saying that the timestamps are accurate to the ppm of the oscillator (50ppm or thereabouts) so they aren’t going to drift very far over an hour or two.
Gordon
Gordon
Mike,
Sorry, none of the above, the 2835 chip has two CSI interfaces but we only have access to one… To do any more would require a different board… :-<
Gordon
meltwater
I think you might get shot for mentioning two CSI interfaces, some poor sole (Gert?) is going to have to route the 2nd one out next time.
guru
I imagine those connectors are relatively expensive tho. You don’t get *everything* for $25/$35, you just get a *lot* :-)
toxibunny
Yeah, I had a bit of a barney with Jamesh in the camera thread. I was all like “Pardon me, my fine sir. In my humble opinion, it’s not impossible that exposing the second CSI port would be regarded as a very advantageous feature, and hence attract more customers despite a modest price raise.” and he was all like “Oi you muriatic bard-twizzler! We had a price-point to keep to, and there was no way we could keep the second floppin’ CSI port. Anyway, it’s all done and dusted and far too late now, alright!??”*
So yeah I’d excise that last musing from the article if I were you…
*quotes altered for literary effect.
Simon D
Good bit of chain ganging.
How did you solve the environmental protection? Fine on a dry day but on a gnarly weather/course you want something a bit more “soldier proof”. If you wipeout you don’t want to have to go hunting for fragments of SD/PCB in the dirt.
Add GPS? Then you can correlate with your Garmin logs.
Gordon
You’re right, on a wet day I didn’t use it!!!
There was no weather proofing (it way just a demonstrator after all!)
Gordon
Andrew Scheller
Watching the first video, I was just thinking that if you’re doing this on Raspi ModelB, you could plug in a USB GPS dongle and a USB 3G dongle, and then overlay a live-update GoogleMaps position onto one of the corners of the video… :-)
Gordon
One of the things I was thinking of doing was to integrate the information from my power meter on the bike (road bike) which would then give you real time power and speed information…
Andrew Scheller
And if you’re doing multiple laps of the same circuit, you could use the GPS on the first lap to record the shape of the circuit, and then on subsequent laps you could draw the shape of the circuit, and your current position on the circuit, like you get in the corner of the screen on Mario Kart :-)
And if you had some way of telling it the position of the finish line (presumably the same as the start point?), you could even record approximate lap times automatically…
All subject to the precision of the GPS readings of course!
Henrik Laursen
This is very much the application I’ve been tinkering with. I have a small 3D-stereo usb-webcam that registers fine with the RPi. Using some commercial Windows (!) sw, I’ve managed to stream anaglyph or side-by-side 3D-stereo vdo. However, using the RPi to do the vdo-processing locally before streaming, is what I wan’t. *So, yes please, do make the code available!*
BTW, youtube’s way of displaying 3D vdo can be changed by hitting the “3D” icon at bottom/right. If you select “other options”, and “no glasses”, you usually get a perfect side-by-side video, which you can watch cross-eyed without glasses (otherwise, select “swap (left-right)”). Possibly, you also could watch directly on a 3D tv, with a proper option.
Ben
What’s going on with the colours on the road? YouTube compression?
Hove
That’s fantastic – the CUCC lycra and the clip you’re all riding at I mean!
Oh and the video is cool too – looking forward to the camera availability for my Drone Pi!
omenie
Holy crap, cross-eye 3D camera cam is NAUSEATING. And my eyes hurt. I may need to go barf right now.
ridgelift
For those comfortable with the Arduino style I/O programming looking for a welcome layer of abstraction between their code and the BCM2835, Gordons WiringPi makes porting Arduino projects to the Pi a breeze.
https://projects.drogon.net/raspberry-pi/wiringpi/
Gordon
Different Gordon… Sorry
ridgelift
Oh, alright then.
Since Raspbian is the recommended Linux distribution for the Pi, is there a recommended GPIO abstraction layer the Foundation would like to see used?
Gordon
Not yet, but there will be. I’m hoping to get there but it’ll be using device tree blobs for the board specification. In terms of API, I’m not sure if there is a suitable one out there yet (as I said I haven’t looked yet!)
Gordon
Homer hazel
The magazine Maximum PC who generally does articles about fast over clocked computers appears to be a fan of Raspberry Pi. They have done two articles about Raspberry Pi since its inception. They did an article about building a stereo camera a while back. It only took photos and not movies, but I found it very interesting.
Jim Manley
Yes Gordon, please find that code so I won’t need to reinvent it when I attempt to do 3-D scene object extraction for the Pi camera beta test in which I hope to participate. Gordi-Wan, you’re our only hope! :D
BTW, as a fellow cyclist (a 100+ miles-per-day Centurion … preferably on the flats, though ;)), if you ever get to carry Eben’s and Liz’s bags on one of their trips out here to SillyCon Valley, perhaps we can ride along the hundreds of miles of local trails (we’ll provide a nice bike if needed – what flavor of carbon-fiber do you prefer for breakfast?) and if there’s time, perhaps get up to The Wine Country in the Napa and Sonoma Valleys. If you’re really looking to commit suicide, there’s always the the opportunity to do 70+ mph / 110+ kph coming down from the Sierra Nevada. There could be some REALLY scary 3-D video generated that way … at least for a limited number of I-frames! 8O
This has to be the only blog post with the keyword “thighs” associated with it … unless Liz decides to post a chicken dish recipe here … :D
Gordon
Yep…
I’ll remember that, never pass up the chance to cycle when I go on a business trip!
Gordon
George Witherspoon
You really should rethink that headline…..
liz
Or it *could* be deliberate. You never know.
Andrew Scheller
Haha, I love Liz’s attitude ;-) (including previous posts too, of course)
Andrew Scheller
And on the subject of helmets… https://twitter.com/Raspberry_Pi/status/309316669870862337/photo/1
Andrew Scheller
Even more helmet-cam news!
http://www.bbc.co.uk/news/uk-wales-21745250
Gordon
For whatever reason I can’t reply to the link below… But that’s great, I love the fact that having dropped 100ft down over rocks and stuff, the first thing he does when he stops is check the headcam is still working!
Gordon
David
Nice and flat there :) In-race footage is fun but banned in Scotland. (Under UCI technical regs – unless your camera is built into the speedo it counts as ancillary equipment not needed for racing and is verboten).
Cool project and I can’t wait to try out a camera on my Pi. Currently wrangling a DSLR off it (not as a helmet cam) but I might try to get my cheap veho clone to run as a webcam.
Andrew Scheller
“Under UCI technical regs – unless your camera is built into the speedo it counts as ancillary equipment not needed for racing and is verboten”
Gordon mentioned above “integrate the information from my power meter on the bike” – presumably using the RPi’s GPIO and an LCD screen it shouldn’t be too hard to rig up a RasPi as a speedo which /just happens/ to also record HD video footage? :-)
jp
You could have just taped a Nintendo 3ds to your helmet. The fashion statement comes at no extra cost.
Paul
my eyes hurt now lol, but yes i could see the difference using a pair of 3d glasses that i got free from sainsburys ages ago for something on channel 4.
Chris Evans
Does ‘microDB’ stand for micro Development Board?
Chris Evans
Sorry meant to say great project. I could do with one on my cycle commute. Though even a Model A is a bit bulky. I wonder how long before a range of Mini Raspberry Pi’s will appear with only a varying subset of connectors. For this project the HDMI, Composite, Audio, DSI and most GPIO could be dropped from a model A. i.e. just have SD, power, USB & CSI !
Gordon Hollingworth
Yes, MicroDB stands for Micro Development Board, it was what we created at Broadcom to test our chips… (and show off what it could do to the rest of the world)
The hardware is still working (I found out today) which is literally exactly as you mention although we keep the hdmi since it fits fine…
The hardware has, 2835 + PMU + audio codec + USB + HDMI. There are two buttons and two leds (and a bunch of test points that get the odd GPIO off the chip!) And of course a battery, we used the iPod shuffle battery which would allow us to record video at 720P for 1 hour…
Gordon
Ed
Hi,
Did the code ever get put on GitHub?
liz
He hasn’t been able to find it yet! When I asked last week, he said it *might* be on an old PC in his loft somewhere.
Greg
Any thoughts on how to make an enclosure for the production camera board that would allow mounting on a pair of eyeglasses?