A Raspberry Pi, hosting…Raspberry Pi
Liz: Here’s a guest post from our old friend Pete Stevens, who runs Mythic Beasts, the hosting company that keeps this website on its legs even though more than 100,000 of you visit every day. He’s in charge of a Raspberry Pi which has some of the best connections in the world. And we don’t mean social connections: read down to learn more. (This post was first published at http://blog.mythic-beasts.com/.) Thanks Pete!
Raspberry Pi hosts itself
We’re now running a mirror for the Raspberry Pi download server and the Raspbian apt repository on a Raspberry Pi.
The first problem was obtaining a Raspberry Pi as buying one was tricky: firstly the online shops were down, then the queue for a Pi was rather long. As any banker can tell you, occasionally crime does pay, so I abducted Mooncake, a cat which owns Liz and Eben. I then ransomed the feline back in exchange for a Raspberry Pi.
[Liz interjects: and Mooncake hasn’t got over it yet. She now hides every time someone with a beard visits the house.]
The hardware setup starts with a power supply with an IEC14 connector connected to the masterswitch for remote power cycling. This PSU connects to a powered USB hub, with a USB lead connecting to the power connector on the Raspberry Pi. The Pi is then connected back to the hub on the data cable with a 1TB USB external hard disk attached to that. There’s a 100Mbit ethernet cable which connects up to the core Mythic Beasts network and out to the Internet. Technically the switch port on the other end is 1Gbit but the Raspberry Pi isn’t fast enough to use that.
The yellow fibre in the background is a very large Internet exchange with over a terabit of bandwidth. The Pi isn’t connected directly (that’d be too stupid even for us) and the packets travel from the Pi to the exchange a couple of feet away via another building. However with LONAP and LINX within 2ms and AMSIX a mere 8ms away it’s still rather well connected.
The software setup on the Pi is fairly straightforward. We started with the Debian squeeze image, ssh/apache enabled, munin enabled (graphs here). We’ve changed the password (obviously), the ssh keys (shipping the same ssh key on every OS image isn’t optimal), and moved /var/www and /var/log to the USB disk so as not to fill the SSD card. rpi-update was needed to make the USB/network setup stable under load as the initial image kept crashing. We also set the RAM split to 224MB for Linux as we really aren’t using the GPU.
It’s up and running both IPv4 (18.104.22.168) and IPv6 (2a00:1098:0:80:1000:13:0:3), as the core Raspbian server is also running IPv6 and the main Raspberry Pi server is also IPv6 at present; this machine has seen more than 50% of its traffic over IPv6. I suspect this will change as people download images from it though.
At present the Raspberry Pi is devoting nearly half its CPU to drawing munin graphs so I need to benchmark the new Raspbian distribution to see if the hard float debian build improves the anti-aliasing, as presently all the calculations are done without using the FPU on the arm core in the Pi. Benchmarking suggests that we can deliver 35-50Mbps of file downloads reasonably comfortably at present.
Is this sensible? We’ve had a few customers ask us if the Raspberry Pi would be a sensible device for hosting on as it’s very cheap and very low power. Unfortunately it’s also very slow for this kind of application and the supporting hardware is very bulky. The i7 quad core Mac Mini occupies less space than the Pi + hub + disk + PSU, uses about fives times as much electricity, costs about five times as much once you include the supporting hardware but is hundreds of times faster. So revolutionising the hosting industry isn’t going to happen with the Raspberry Pi, at least not until they build a PoE one with gigabit ethernet and more RAM.
[Liz: you may be waiting some time for that, Pete. I’d stick to the Mac Minis.]
It’s not currently sensible to do this with shelves full of Raspberry Pis because the performance per Watt isn’t good enough. But we’re working on it.
Thanks go to Liam Fraser and Mike Thompson for adding us to the official mirror lists for the Pi and Raspbian. Additional thanks go to Eben and Liz for paying the ransom fee of one Raspberry Pi in exchange for the safe return of Mooncake.
Sod all the tech, that is one über cute kitty. How could anyone ever kidnap that! You evil, evil man Pete!
I agree, Lovely looking cat.
If Mythic Beasts is interested in small form factor with low cost and low power then they should probably be looking at the Odroid instead:
Like the Rasberry Pi on steroids, it has 8x the processing (4×1.4GHz), 4x the RAM (1GB), still only 100Mbit ethernet but he might bond some USB ethernet adaptors for good measure. All in 5W of power!
This is not an advert, I just think this is a cool product.
That said I would probably do the munin graphing on a different machine and just use munin-node on the R-Pi. It would save a significant amount of processing.
For the serving of static pages the processing requirements are pretty small. If it doesn’t already exist I’m sure someone could build a hardware implementation of the http protocol that could serve up static pages at a blistering speed. It is the dynamic content that is the big overhead with things like databases.
I wonder if in future we might begin to see small low powered systems that are optimised for a particular task. For example a http server, database server etc that have specialist hardware for the task. Much like we have specialist hardware for video decoding included on some graphics cards.
It’s pretty easy to saturate a 1GBit link with a virtual machine with a 1/40th share of a quad core machine (that’s the other Raspi mirror server we have up). I’m pretty certain the bottleneck in the Raspi is the USB-ethernet and USB-disk, were it a proper network card (and Broadcom do make some nice network cards) I suspect it’d be trivial to fill the cable.
I should probably benchmark squeeze vs raspbian serving out of RAM to see what the raw network performance is.
I suppose the R-Pi might make a nice little Memcached box? 150MB of edge cache? an ngnix configuration?
Yes I think you are right, the way the network is connected via USB will add significant overhead, especially the way USB drivers use a lot of processor cycles. The Pi would make a good lights out manager though.
Darren, for lights out I would prefer something like the mbed processor, even if it is almost the same price as the Rasberry Pi:
Unfortunatlely everything you’d use for hosting stuff ends up stuck behind a single USB connection like on the Pi :(
I’m so glad Mooncake is back with her servants!
Oh, I never thought of it before (perhaps you all have), we need rack-mount options for the Pi.
There are quite a few rack builds on the cases forum. See this post, and this for example.
You don’t really need a hub and you don’t really need a PSU per computer either. One decent 5V supply like should easilly be able to run 20 Pis with hard drives.
The tricky bit of course is you would need custom cabling which may be a bit of a pain to fabricate.
With a centralised power supply and short custom cables I reckon you could get a Pi with hard drive into less space than a mac mini without too much trouble.
Those terminals aren’t independently switchable. So that way if one Pi crashes and needs restarting you have to power cycle all 20 in one go. That’s going to make the other 19 customers jump for joy with happiness.
mmm, still making a board that took 5V in at high current, switched it, fused it and distributed it to the Pis and HDDs shouldn’t be too difficult.
I would’ve kept the cat. she’s cute :)
Nice job Pete. I’m surprised to hear that the ssh keys are stored in the OS image. Wouldn’t auto generation of the keys on first boot be a fairly simple thing to do?
I think this is a grat idea. I thoughd this would have been done anyway.
Can you post a description of how to do this manualy when the system is already in use? I’m just a beginner, so a step by step description would be very cool. If you know a website already containing this information, a link would be ok. I would highly appreciate that. Thank you very much.
Looks like the seedling for a bramble is planted, considering 20 raspberries won’t consume more than 80 watts it would still make quite a frugal system.
Sure one raspberry alone wont do the trick but the potential for tailored solutions is overwhelming!
I hate to agree with a cat but – never trust a man with a beard (Black Adder I think)
I also have a web server on my pi running for abt 10-12 days
that is really incredible! I saw a picture of several raspberrypi’s “rack” mounted in a lego case .. which made me wonder if you could network them together to get more power for your server app???
Just a heads up, you may want to take a look at the link bellow. They are selling r pi(‘s) under the name of “Raspberry Pi Foundation”
Thanks Ricky. I’m dealing with trademark infringements and I’ve just taken a look at this. I don’t think they *are* impersonating the Foundation; Amazon product pages are just set out like that, with the brand/manufacturer below the product name, regardless of who the seller is. Heck of a mark-up, though!
Thanks Helen! Everybody, meet Helen: Helen is our resident trademark valkyrie, and she is *brilliant*.
The price there is £71! O_o
Why is this guy using macs when he could buy much cheaper machines that are the same but with less pretty housing?
Pete actually has a long and very considered answer to your question – I asked him the same when Mythic started doing it. I’ll precis it for him here (although I’m sure he’ll answer you himself): it turns out to be less expensive *and* more reliable. Housing be damned.
I think they were a bit harsh on the rpi by judging its web server suitability while as a download site for a large file. Rpi would probably be great for static pages or web apps. Why not aim to host part of this site on an RPi or RPi array…that would be cool.
You have 10U of rackspace. Please specify 24 quad core servers each with 8GB of RAM, 2x500GB disks, individually masterswitched and switched in that space. Your total power budget is 1kW. I’ll await your answer by e-mail to firstname.lastname@example.org. Bonus marks will be available if you can guarantee delivery of quantities of up to 100 in 48 hours and if you can guarantee identical hardware for 6 months+ to minimise the integration work with netboot recovery.
Peter, I was wondering the same thing as Elliot, and I appreciate the detail in your response, but why the condescending attitude?
I think Apache was a somewhat curious choice here given it’s thirst for resources. I’d venture that if you have to use a Raspberry Pi as a server then Node.js is probably going to be a wise choice due to small footprint and thrifty use of threads. Admittedly the asynchronous programming style gets a bit of getting used to !
If you use netcat across the local lan,
time cat debian6-17-02-2012.zip | nc raspberry.mythic-beasts.com 7777
you discover that usb -> network bottlenecks at roughly 40Mbps with roughly 25% cpu in system, roughly the same as apache manages. I really don’t think the performance issue is the choice of webserver, it’s the disk & network both being attached over USB. I may of course be wrong.
This is an educational project and we like science. Your hypothesis is that node.js is faster than apache for large static file serving. I suggest your experimental method should be to benchmark the two of them with ab or siege and report back with a full method conclusion and results. Then we’d know the answer and you’ve had contributed something useful to the field instead of merely speculating wildly in public as you’re doing at present. We await your results. For bonus marks I suggest adding in ngnix and lighttpd and soak testing all four options for a few days to test stability.
Jools – I’ve just deleted your response because it was patronising, unpleasant and doesn’t contribute. Reel it in, please.
Is that with the file stored on a USB disk (i.e. also ‘fighting’ the ethernet for bandwidth) ?
I wonder if the figures would be any different with the file being read from SD card? (obviously you’d want to use a fast SD card and a recent kernel)
Raspbian is nearly 100GB in size so it wouldn’t fit on the SD card.
I connected my kill-a-watt to the raspberry pi. It draws between 2.7-3 watts. My home server with an atom processor draws around 40 watts. I am considering replacing it with a raspberry pi. That being said, I have an ivy bridge Macbook Pro (with SSD) that only draws around 9-10 watts at idle but will ramp up to around 25 watts. The MBP has way more power per watt than the Raspberry pi but also costs a lot more.
Is that Apple TV’s I see above the MacMini’s?
What are the Apple TVs for? I’m running Crystalbuntu on mine. What do those do, and what OS are they running?
IIRC mythic were offering the apple TV as their cheapest dedicated server option at one point. They don’t seem to be offering it to new customers now though (I think apple may have redesigned the apple TV to make this no longer possible).
They did redesign the Apple TV. The older one runs a modified version of OSX 10.4, and has a Intel Pentium M processor, you can run full versions of OSX 10.4, and OSX 10.5, as well as Ubuntu, and other Linux distributions. I have OSX 10.5 running on mine. The newer one(s) run IOS 4, IOS 5, IOS 6, and had an arm A4 processor, now an A5. What OS are they running? OSX, Ubuntu, Linux? And how big are the hard drives?
Theoretically, I could emulate a pi on my pi to download from this mirror hosted on a pi..
Pi downloaded by Pi running on Pi from Pi…
Ozgur D. Cyric
woot, accept the challenge and make an RPi Model C with more RAM and GBi Eth. Dad wood spice fings alil beat up!
Great work getting the RPi running as a webserver!
If 100MBaud is not enough: What holds you from using a USB->GBaud adapter?
Should give 300-400MBaud bandwidth.
Not if the purpose is to have remote access from power-up since one can’t generate SSH keys without access :)
Having a known SSH key is very nice as it allows using the RPi even if all one has available is the end of an Ethernet cable connected to some other accessible machine. The important part is to change it as soon as appropriate just like any other default password.
Please note: I am only 1bit and my warranty long expired ;_;
I protest against feline abuse! :)
PS, For this kind of static file hosting, lightweight http servers like nginx or lighttpd might work better than Apache.
Im reading the thread and wondering. Can or cant it do as a server. The real question is what level do you want?
Looking over in a small corner of my work area. There’s this little board. I swear it smiling. Running Apache+PHP+FTP ( using notes found in the forum here ) being a great little basic web page server. And at the same time time running icecast + ezstream as a small internet based radio station. Running off of a 5v 1Amp size of an Apple phone charger clone with a chunk of 16GB C10 SDHC memory.
It not doing a lot but it could use it to teach server tech and even giving students a how to do streaming net audio aka their own micro radio station. It may max out with a few users but like its main task it was developed for. To offer a way to allow students to get basic net skills for a reasonable price.
Need a 1gbit link more? i have a 1gbit connection at home, 20gb space, and 512mb ram to spare in my vmware server? :)
I have been running 2x Raspberry Pi web servers with active and constant access. You can have a very fast system if done “right”. The problem is Apache2 was not designed for low memory systems. It hogs memory/threads and bogs down on resource starvation.
On the flip side NginX “was” designed as a high performance web server for minimal systems. Compared to Apache It runs like a speeding freight train and leaves plenty of memory left over. I run it with PHP (no wrapper running: php-cgi -b 127.0.0.1:9000 using port 9000 on localhost).
The Pi is not a PC. If you run inefficient bloat the limitations will hurt. If you run optimized software for the platform the sky is the limit on application.
Plus as far as cost compared to a Macmini. The cost of a powered USB hub is excessive as most don’t power that way. I have 2 Pi’s running off a single 5v supply ($8 ebay) though in a Raq Mount system I would tuck it away somewhere with a USB cable connected to a free Mac usb port or 5v power connector and Ethernet cable into the network. Unless you have a lot of disk writing going on the SD card would be fine for hosting up to it’s read throughput.