Website hosting on Raspberry Pi 4 with Mythic Beasts
Here’s Mythic Beast’s Pete Stevens to talk about how we run the Raspberry Pi website on Raspberry Pis, and how Mythic Beasts can run your site on Raspberry Pis too!
Rent a Raspberry Pi
In late 2016, Mythic Beasts launched a Raspberry Pi cloud, allowing you to rent a Raspberry Pi 3 as a service.
Raspberry Pi 4 is a much more capable computer, with more than twice the performance and, crucially, four times the memory. We were so excited by it, we bet Eben Upton a beer that we could host the launch site for Raspberry Pi 4 on Raspberry Pi 4. We’d demonstrated that it was just about possible to run a normal day on a cluster of eight Raspberry Pi 3s, but launch day is a bit more exciting — tens of millions rather than a million visitors.
Eben, being a fool supremely confident in the work that his team had done, took the bet and let us. On Thursday 20 June 2019, he dropped off eighteen 4GB RAM Raspberry Pi 4 computers that had previously been used in testing. We set about configuring them to replace all the web servers that run the Raspberry Pi blog.
- 14× Dynamic Web server (PHP/Apache)
- 2× Static webserver (Apache, flat files)
- 2× Memcache (in memory store to accelerate web serving)
We started the build on Friday 21 June. We immediately ran into our first ‘chicken and egg’ problem. The Raspberry Pi web servers are built from Puppet, based (at the time) on Debian Jessie. Raspberry Pi 4’s release OS was a not-yet-released version of Debian Buster, which at the time wasn’t supported by Puppet. In conjunction with Greg Annandale at the Raspberry Pi Foundation, we created a Puppet build that would run on Raspberry Pi 4, updated the configuration from Jessie to Buster (newer Apache/PHP), and did some testing.
We have pre-built enclosures from our Raspberry Pi 3 cloud. We followed the same approach using Power over Ethernet to provide power and data to each Raspberry Pi 4. This dramatically reduces the cabling and complexity of the setup. Late on Friday 21, just over 24 hours after we started, we moved the hastily constructed Raspberry Pi 4 setup to Sovereign House, a key Mythic Beasts data centre and one of the best-connected buildings in Europe.
Over the course of a few hours, we gradually moved the entire production load from the existing virtual servers to the Raspberry Pi 4 cloud and every page from the blog was being served directly off Raspberry Pi 4. We left it for two days to bed in before the real test: launch day.
The launch was almost perfectly smooth. The Raspberry Pi cluster coped fine with the tens of millions of users. However, the Raspberry Pi cluster and website is fronted by Cloudflare, which provides acceleration for static resources and protects the site from denial of service. Unfortunately, they had a two-hour outage in the middle of the launch thanks to a misconfigured internet optimiser run by a customer of Verizon. So the Raspberry Pi 4 cluster had a long lunch break wondering where all the users had gone.
We ran the website on the Raspberry Pi 4 cluster for over a month before reverting back to the usual virtual server-based environment. We’d proved that RaspberryPi 4 would make an awesome hosting platform.
Commercialising Raspberry Pi 4 as a service
We were already running Raspberry Pi 3 as a service for many customers (e.g. PiWheels, which builds Python packages for Raspberry Pi), and being able spin up Raspberry Pi 3 on demand is incredibly useful.
At launch, Raspberry Pi 4 wasn’t suitable. We rely on network boot in order to be able to remotely re-image Raspberry Pi. SD cards just aren’t very reliable; visiting the data centre for manual intervention on every SD card failure is not only expensive in time, but also means we’d have to maintain physical access to every Raspberry Pi 4 in our cloud. Netboot means that we just build large enclosures of 108 Raspberry Pis and seal them in, as they will never require physical attention. If one fails — and we’ve not yet seen one fail — we can shut it down and take it out of our database.
For Raspberry Pi 4 we had to wait for network booting to be a reality. We had access to beta firmware in November 2019 and built a sample Raspberry Pi 4 network boot setup. We then had to integrate it into our management code, build Raspberry Pi 4–compatible operating system images, and enhance our billing to cope with multiple models and by-the-hour billing. Then we had to do a file server and network upgrade: serving lots of machines with true gigabit needs more ‘oomph’ than the 100Mbps of Raspberry Pi 3. This also all needed to be backward-compatible so as not to break the existing Raspberry Pi 3 users. On 17 June 2020 we launched, and Raspberry Pi 4 is now ready to order in our cloud.
Is it any good?
Yes. Raspberry Pi is twice as fast as the same-sized instances in AWS, for a quarter of the price. Just see for yourself:
| Raspberry Pi 4 | a1.large | mg6.medium | |
| Spec | 4 cores @ 1.5GHz 4GB RAM | 2 cores 4GB RAM | 1 core 4GB RAM |
| Monthly price | £8.63 | $45.35 (~£36.09) | $34.69 (~ £27.61) |
| Requests per second | 107 | 52 | 57 |
| Mean request time | 457ms | 978ms | 868ms |
| 99th percentile request time | 791ms | 1247ms | 1056ms |
But what about 8GB and 64-bit Raspberry Pi OS?
That sounds like a jolly nice idea. Keep watching the Mythic Beasts blog for updates.
25 comments
Giles
Interesting how you’ve done this.
I have found that “high endurance” (micro)SD(xc) cards are useful in demanding environments. Where a regular card might last just three months of intensive use, the high endurance cards will run without error for much, much longer.
High endurance cards are mostly sold for use in repetitive write applications such as car dashcams, security cameras and the like, so they’re often optimised for long block writes. I understand the actual storage mechanism is more resilient than the ‘standard’ cards, which is the source of their longevity. High endurance cards are a little more expensive (as of July 2020 I see 128GB cards at about UK £21 retail, delivered) but compared to the hassle of a site visit to recover and replace a damaged card it’s a worthwhile investment in my opinion.
Peter Stevens
For the production Pi 4 cloud all the storage is over the network on a high performance filesystem with mirrored disks. No SD cards to fail, no access required to the Pis. In the production cloud, access to one Pi 4 requires switching off the 53 other customers in the unit – that’s not acceptable to us which is why it’s netboot only.
Anders
There I was all happy and stable with my site on Lightsail and this comes along. Does anyone know if Mythical Beasts use the standard Raspberry Pi OS image, or do they tweak it?
I’d like to run identical environments for dev/test on my own Pis, which would use nginx for reverse proxy with gunicorn.
stu dent
good day ( .. drizzley tbh )
a link is mort !
:0
‘http://(https//www.mythic-beasts.com/order/rpi/’
should be
‘https//www.mythic-beasts.com/order/rpi/’
save the () for lisp ;)
Alex Bate
Doh, I can’t even blame anyone but myself.
Thanks for the heads-up. I’ve edited the link.
Gerardo Galvan
Hi, greetings from Mexico!! So, does the Raspberry Pi 4 support PoE? I had not seen that in is specs.
Eben Upton
Yes, using the optional PoE HAT accessory.
Rohit
Very nice work Alex and team Mythic Beasts! Great proof point. Something that I have believed since early tests with RPi3 and then used it to base our telecom product on that runs 24×7 and one of our largest installs runs with a 512GB microSD for past year over, processing and recording at peak times about 1500 calls in a day with 10-15 calls being simultaneously recorded and stored on the same microSD at any given time. Like yourself, we also were concerned about the cost of support due to sd failure, but some of the tweaks that we did has ensured that we do not have any sd or Pi3 failures in the past 3+ years. I am remiss in jotting a blog on how to ensure even a TLC card performs without corruption and failures. Will do shortly and share.
Keep up the good work with the Pi!
Bennet
Any chance of a rent to own option? Say after a year of service, I would own the Pi outright. Upon termination of the service at any time afterwards, the Pi would be shipped to me.
Peter Stevens
This doesn’t really make any sense as an idea – it’d be much easier to ship you a fresh Pi than dismantle the cluster and cause a 50+ server outage to get to ‘your’ Pi. If you need a Pi, buy a Pi. If you want a Pi with network attached storage and 1Gbps symmetric ethernet for significantly under a tenner per month, rent one in the cloud.
Richard Paul
I shut down my big old ML350P for a couple of RPi4s. I’m not at the point of needing it yet for my personal projects, although after months I have still to sort out getting my personal sites back up and running yet. The Pis are more than man enough for hosting and I’d be surprised if the tweaks to the puppet config was anything to really note.
ProDigit
You should have used those $35 AMLogic tv box boards.
Much better io, just no gpio pins.
Or, the $30 Atomic pi.
Peter Stevens
No PoE and network boot – doubles the wiring and we need to maintain physical access to each to be able to replace SD cards each time one is reallocated. Please start a competing service based on it, you’ll need an army of staff members to service them an 1/3rd the rack density we get.
Paul
I was wondering if there is an Office 365 type service available from Mythic Beasts, based on Raspberry Pi’s as cloud storage and productivity suite. Now that the Pi 4 is out there as a desktop replacement could I just switch from Microsoft to Mythic Beasts ?
Anders
Looking at the services offered on the website, Mythic Beast do offer hosted NextCloud, though it doesn’t say this can be on Rapsberry Pi.
Nev Young
I’ve been running several charity websites from my single Pi since 2012. Initially an original Pi model B and, since they came out, a Pi2. Works fine.
TIMOTHY HUNOLD
I hosted from home on RPI 4b. Great tool. Put all my data on external USB to avoid the read and write fatigue.
Anil
Hi ,
Raspberry Pi based servers cool ,can you make a blog how to make a server on pi so that I can try hosting a website ?
Peter Stevens
Order a Pi4 in the cloud from.
https://www.mythic-beasts.com/order/rpi
Log in via ssh and install apache2. Put your files in /var/www/html
Congratulations, you’ve finished.
SnapShooter MySQL Backups
Amazing work again!
Its really interesting to see what kind of DC you could built on a PI cluster!
Eric Olson
I’ve been working on a Pi-based cloud. Two questions have come up:
1. Is it possible to physically disable the WiFi radio to prevent problems when the Pi is housed in a data center?
2. Is it possible to physically disable writing to the EEPROM to prevent hackers from placing weird code there and the clueless from disabling network boot?
Have you done anything to address either of these questions, or are you finding them not to be an issue at present?
Peter Stevens
(1) Wifi/Bluetooth. We’ve not worried – you almost certainly can’t get anything useful out of it and they’re in a big steel box.
(2) That is a concern although not one we currently have a solution for. We may see if we can get a config.txt option implemented that disables eeprom update.
sk maltare
All well done.
My concern remains regarding heat generated. even when i used pi4 a lot of heat is seen on cpu, even for small application like motioneyeos or others.
do you also face heat and how you cope up ?
Peter Stevens
The PoE HAT has a fan and we ventilate the whole shelf. Currently our entire Pi4 cloud is reporting CPU temperatures below 60C, although someone can probably exceed that if they try hard enough.
Hitesh Jethva
Very nice work again!
You are doing interesting stuff. Keep it up.