Adieu to the database errors!

Because Pete Stevens at Mythic Beasts is brilliant, the period the site was down for while he moved our database and web servers lasted for under 5 minutes (and in the time between his bringing everything back up, dialling my phone number and my picking up, we had 30k hits, which graphically illustrates our need for the move…)

We think everything is working smoothly now, and all the functionality with commenting and posting on the forums should be there. If you see anything peculiar, please leave a comment here or email us using the contact page (link at top of page).

Liz again, 15:34h. Because so many of you were asking for details here and on Twitter about what our server setup is, Pete’s just mailed me this:

Thanks for the nice comments. Since they’re all going to ask, feel free to post on my behalf,

The new box is dedicated to There’s a fairly quick six disk RAID10 array with battery backed cache and enough RAM to cache everything on disk without really running short. Some server porn from /proc

# cat /proc/meminfo  | head -1
MemTotal:       99202196 kB

# pvscan
PV /dev/sda5   VG mainvg   lvm2 [4.05 TiB / 3.95 TiB free]

# cat /proc/cpuinfo | grep processor | wc
8      24     112

# w
15:12:37 up 17:08, 10 users,  load average: 1.39, 1.09, 0.86

Whilst the box doesn’t have that much CPU, everything should be served direct from memory so we should be able to keep those CPUs nice and busy. I have a plan to add lots of quad core web processing boxes on the front if it turns out to be necessary.

Now being the generous lot you are you’re probably about to hang me for incorrectly configuring the kitten morphing plugin for wordpress that I didn’t know about, and the few minutes where we probably threw comments away while I did the db dump/restore. Flame on…


It’s almost as if he’s read the comments section here before…but as far as we are aware, all kittens are being morphed correctly. 


MarshallBanana avatar

hooraaayyyyyy :-)

liz avatar

That’s very similar to the noise I’ve been making for the last few minutes.

deaid avatar

i noticed that site is loading faster

Rick avatar

Great :) Next on the list is to ditch the crummy forum software and get a design without 100 pixels padding on the content area :p

Top work guys – Mythical Beast are awesome…might have to get a Mac Mini collocated there :D

mobeyduck avatar

[quote]might have to get a Mac Mini collocated there [/quote]
Have you ever heard Sheldon about the guys @mac stores?

andyfrommk avatar

Make sure you tell mythic beasts that you came to them from so the foundation can get three months free hosting

dave_s avatar

Speed increase is fantaaaaaastic!

XAPBob avatar


Only comment is that when browsing mobile I don’t see the password field here, so I can’t post…

itsonlyme avatar

Well done Pete, and Liz

Mario avatar

Thank you! it is suuper fast now :D

liz avatar

From my end doing admin stuff, it feels like upgrading from a tricycle to an F19. I’m almost as excited about this as I was about launch…

Mario avatar

Wow, you actually replied one of my posts? Wow, that’s flattering! and yes, I dont even doubt it, you passed from a LowEnd VPS to a Dedicated, isnt it? it feels like you had the 15 dlls/yr VPS and now a dedi.

btw, your password no-spam plugin is broken… sometimes it doesnt accept the pwd… or it is my cache?

Clinton avatar

I had trouble with that password too yesterday.

liz avatar

Hmm – might be our cache, actually. Let me just whitelist you, and tell me if you have any more problems.

DataSpree avatar

testing password acceptance…

Roger Wolff avatar

LIz, for me, firefox under Linux it’s quite structural that it almost never works. I have to reload the page and then immediately submit the quote and password for it to work….

Roger Wolff avatar

Hmmm. That post was more or less the first that worked…..
And this one is back to the old: “hit back and enter the password” (which would clear the comment already typed, and not help getting the comment posted.)

XAPBob avatar

Hey – some tricycles are rather nice thank you very much ;)

I do 40mph+ every day on my way home from work…

Forums are somewhat more browseable now though :D

Michael avatar

I misread that last line as “I’m almost as excited about this as I was about lunch…” and I thought: typical foodie :D

liz avatar

Lunch was, in fact, some most excellent home-made gazpacho poured over a bit of shredded white crab meat. I was pretty excited about that too.

psergiu avatar

It’s faster ! It’s doesn’t crashes anymore !
What on earth we will going to do without those insightful posts asking if the forum runs on a RPi or a BBC Micro ?

May Raspberry Pi become so popular and widespread that this will be the first upgrade of many to follow.

Jan Vachun avatar

Link to Shop still reports:

Down for Maintenance
This store is currently unavailable. Please visit our main website at

liz avatar

That’s because it’s closed until mid month, when we’re getting someone in to run it. Nothing to do with this server move.

psergiu avatar

Can you please add “Will be closed until end of March” to that notice on – it will save you a lot of twitter & forum replies :)

redndahead avatar

Make that closed until the end of June. Then people will get a surprise when it opens earlier.

Harrkev avatar

Oh No! How am I supposed to get my full-color sticker to go on my Pi? Sorry, the white silk-screen does not do much for me ;)

PS: I keep on getting “Error 1: Click back and type in the password.” Twice as much work to post. Also, the comment window does NOT auto-scroll down very well when you type. I have to keep hitting the down arrow to see what I am typing. Firefox 10 on Red Hat Enterprise 5.

João Reis avatar

I always have this error while reading, but if I reload the page and immediately post a comment it works. (maybe the password has a timeout I don’t know).

… coping this text to the clipboard, just in case it fails again :) …

LLiinnuuxx avatar

Comments doesnt work … get an error

thats why wrote an mail to you

St. Nolden avatar

Wow, that’s fast now.

May I ask on what kind of machine the site now runs? And what was used before?
Oh, and can we get some statistics please?

Darren avatar

How can the server have 99GB RAM ?

psergiu avatar

96 GB of RAM (most likely 12 sticks of 8 GB) – minus about 1.4 GB used by the kernel (and video card ?) * 1024 * 1024 = 99202196 KB

Abishur avatar

Well, to be fair it’s only 94 (and a half-ish), but seeing as how I can easily buy 8 GB sticks of RAM, and most servers have about 8 ram slots, then I can easily get 64 GB of RAM. And it’s been a while since I looked for server RAM so it’s not unreasonable for there to be 12 GB sticks of RAM (12*8=96 GB with a little loss bringing it down to ~94.6)

Mike avatar

There were 99GB of RAM in the box, but then they took some down and passed them around. Now there are 94GB of RAM in the box.


Peter Stevens avatar

The server still has 6 free RAM slots, currently it has 6x16GB although we’d need to install another processor to use them.

zag avatar

SSD drive on a server would help even more. We saw about a 500% increase in db IO just upgrading to a simple Intel SSD.

psergiu avatar

RAM is even better.
cp my_ginormous.cnf my.cnf

Alex Langer avatar

If you don’t let the DB server hit the disk, you have no speed to gain at all from faster disks.

Andy avatar

Mythic Beasts are brilliant; they host the O2 Joggler forums where I spend a fair bit of time, and they never miss a beat.

In fact, while you’re waiting on your Raspberry Pi in the mail, you could grab a Joggler as a friend for it. 1080p on a picture frame format device? Yes! :)

Alex Langer avatar

I thought you were serving this site off of some thousand Pis. Ain’t that the one true reason we have to wait for ours?

cnxsoft avatar

I can’t really feel any speed difference (maybe because of my location), but I’m glad I won’t get those nasty database errors anymore.

Bluemerlin avatar

That’s some good server porn right there.

Pete Rocks!

nikomo avatar

Mmmmmm, server porn.

Ben avatar

Porn is just going to make the load problem much worse.

Alex Buell avatar

I’d have liked a full and complete /proc/cpuinfo though…

SonicBroom avatar

Hi! Sorry if this is a known issue. I’ve been lurking for a while and decided to register for the furums, the only issue (however slight) i’ve seen so far is that avatar selection is not working.

It says:
“Sorry, uploads disabled! Storage location does not exist or is not writable. Please contact a forum Admin.”

Other than that congrats on the release and new server update. I quite enjoyed the 6AM rush on that eventful morning, looking forward to getting my pre order :)

psergiu avatar

Avatar uploading was never enabled – use gravatar.

Vanden Circkel avatar

psergiu, odd because after I registered a few days ago from the forum page I clicked “profile” (next to logout) and then select your avatar. I’ve never visited gravatar.

Vanden Circkel avatar

eek, just noticed the avatar I upload that way only appears in the forum, not over here.

Matthew Risi avatar

I really love that somehow the phrase “kitten morphing” is important to this website, and propose that this delightful lingo is somehow included in RasPi hardware/software. Perhaps an animated .gif Easter egg?

Joe Blow avatar

This is a test of the Emergency Broadcast System. If this had been an actual emergency, you would have been instructed to tune to your local Emergency Broadcast Station. This is only a test.

Server porn! As George Takei of Mr. Sulu fame would say, “Oh, MY!!!” OK, _now_ you folks are starting to really “get” us! :)

Forum responses are lightning fast, now, even out here in a remote area in California at the bitter end of a wimpy cell phone link. Thank you, thank you, thank you!!!

Well, let’s see if this makes it on-the-air! Fingers crossed … and Post Comment …

Awwww, crap! The dreaded “Error 1: Click back and type in the password.” error. Well, at least the error comes back lightning fast now. I only take back some of my plaudits … for now. And Post Comment again … and again … and …

Vanden Circkel avatar

“Error 1: Click back and type in the password.”

One workaround is to log into the forum page first. If you do the reply box will show your login and you get to skip password etc.

Liz, I can log out from the reply box. It would be brilliant if we could log in from the reply box or register there.

Ben avatar

You mean Mr. Sulu, of George Takei fame? The man has over 1.25 million followers on Facebook, most of whom are too young to remember Mr. Sulu.

Tom avatar

You know that these commands accept file as argument and reading them with cat is a waste (forcing the kernel to fread them twice (second time in ram, sure) and forking a process)?

head -1 /proc/meminfo
grep processor /proc/cpuinfo| wc

If it was my sysadmin, I’d be worried about his skills.

liz avatar

And as predicted, here’s this comment thread’s first troll. Thanks for your not-at-all-helpful input, Tom. I hope getting it off your chest made you feel better.

Vanden Circkel avatar

But Pete might have been running # cat /proc/meminfo | head -1 thousands of times per second for weeks even.;) #TrollSocialSkills4Hire

jbk avatar

But Liz! It’s terribly inefficient! Clearly that should have been handcoded in assembly for optimal performance and minimal memory footprint…

Think of the kittens!

Shish avatar

I generally do things the multiple-command way too; I find command lines easier to edit when each command does a single thing separated by pipes – I use multiple “grep” processes with simple matches rather than one complicated regex too. That also has the beneift that it scales across multiple cores better :-)

Weuxel avatar


Peter Stevens avatar

The previous command I’d typed was ‘cat /proc/meminfo’, by my count ‘up arrow’ ‘| head -1, is 10 additional keypresses whereas your command requires 21 key presses. Since this is an educational project I’ll expect 2000 words in my email by Monday with a detailed calculation of the number of cpu cycles consumed in each case taking into account both the command run and the cycles consumed to process the shell into which it was typed. For additional marks you can include the cycles taken by my laptop and the two ssh processes in order to communicate the extra key presses over the server. Further marks will be available for a reasonable estimate of the additional load on network devices for the additional internet traffic.

I particularly look forward to your discussion regarding the relative costs of cycles on (a) the server cpu, (b) my laptop cpu, (c) embedded network processors, and the cost utility function you suggest to weigh off usage between them.

XAPBob avatar

That’s a proper response!

Of course the cost of your time also needs to be factored in, and the fact that both (system cost and human cost) require an opportunity analysis to determine whether the cost in terms of time/CPU was actually “prepaid” (i.e. didn’t stop anything else happening because it used idle time)

Peter Stevens avatar

I thought about that but the MBA course is next week.

Rek avatar

As somebody who has has spent far too many hours of their life trying to translate ‘geek’ into ‘bean counter’ and vice versa my opinion would have to be that the ‘Cost of Tom’ far outweighs any possible benefits.

Jason Ozolins avatar

Masterful serve. :-)

Jason Ozolins avatar

For me at least, as a poor simple sysadmin these past 19 years, treating everything as a pipeline, with “cat” as the simplest way to start a pipeline from a file, frees me from a small cognitive load that allows me to get on with caring about stuff that actually matters. If there was a gigabyte of stuff squirting through that pipe, or that pipeline was going to be run in an inner loop of some long-running job, that’s when the caring would start.

Instead, I’ll concentrate on getting 1300TB of data migrated from our old storage system to the new one, at 500MB/sec… that is a task worth tuning.

Guilherme de Sousa avatar

Tks Pete :)

jbk avatar

Finally Pete unveils the specs for the secret Model C! ;)

Absolutely great work! – everything is super fast again just like in ye olde days

WASD avatar

I thought your servers were running on a Raspberry Pi.

PisRUS avatar

Excellent work – thanks ! :D

bretmac avatar

Smokin’ !

oninoshiko avatar

The site is definitely faster, no DB errors.

Be sure to remember RAID is not a backup, seen a number of people bit by confusing the two.

JamesH avatar

Well, I’m sure the professional webhosting company looking after the server probably know what they are doing….

Kambiz avatar

Well, you were also sure that the web-site would do just fine on the product release time, and also you were sure that your designated distributor’s site could handle the traffic … :)

Austin avatar

They aren’t in charge of the distributor’s websites, and they knew that their site would be swamped.

liz avatar

Not to mention the fact that our site *did* do fine at release – we had realised we wouldn’t cope with the traffic with the regular site, so we swapped out to a static page for the day.

Jason avatar

I’d recommend trying as a storefront.

DataSpree avatar

Thank you for upgrading the server!

PerseP avatar

Maybe a picture of the server?

CheechWizard avatar

Since Pi-day is next week, I’m thinking there should be a contest for most creative kitten morphing submission, with the winner announced at Pi-time (3/14 1:59:26). The prize being up to Liz and the gang, whilst a Model A or B would be obvious, perhaps some of Liz’s favorite chocolates? :)

NAB avatar

Pi-time if you’re in the US and you write dates month before day, perhaps. Putting my picky hat on, I’d also say that if you’re going to use the 24-hour clock (given that you haven’t specified AM or PM, you must be), then it should be written as 01:59:26 (note leading zero).
However, if you’re here you must be, at least in part, a bit of a geek. Being so, you would know that there is an international standard for date notation. Using this, the time you suggest should actually be written as:
So, nothing special there then!


zmerch avatar

To get pickier: Wouldn’t you *have* to use US dates for “Pi-time?” Day before month can’t work — there aren’t 14 months in a year. Also, moving the slash — 31/4 can’t happen – only 30 days in April. And… pickier yet: shouldn’t pi time really be 3/14 1:59:27? <– rounded up; as the next digit is a '5'. At least that's what I learned way back in school… (and back then I had pi memorized to almost 100 places; I can only remember about 25-30 or so now – still more than good enough for the math I can't/don't do any more! ;-) )

BTW, great job on the server upgrade!

Vanden Circkel avatar

Off topic, but the RPi Fedora Remix download instructions are *buried* in the forums:

No links on the download page.

OdieTurbo avatar


Rek avatar

Feel free to mis-morph all the kittens you want. I’m a dog person.

Thomas avatar

cat /proc/cpuinfo | grep processor | wc
grep -c processor /proc/cpuinfo

ayeowch avatar

Great! Firebug reports I’m getting consistent

ayeowch avatar

Hmm, that comment looks wrong. I posted “..consistent 900ms loading time..”

Lee avatar

I want a morphed kitten! Wait… what do the kittens morph into?

JamesH avatar

Evil kittens.

Rek avatar

Is the kitten morphing native in Puppy Linux?

gz avatar

Does this morphing kitten involve a box, a radioactive source and small flask of poison?

ps makes my 32gb server seem woefully inadequate…

Robert_M avatar

Thanks, Pete! Forum works like brand new.

…oh, wait… I guess it is! :-D

Mads Nielsen avatar

Sounds great with faster servers, but have you considered a reverse proxy as Vernish or something similar?

Peter Stevens avatar

If you read the source to this page you’ll notice that Wp-Supercache reports that it takes 0.5s to generate the page, and most of the rest of the time we’re serving statically cached copies. Some rough benchmarking on the server reports that we can serve something in the region of 10,000 hits/second of cached pages and about 15/s of the uncached pages. Varnish would allow us to serve the cached pages more quickly at the expense of a more complicated setup, and to be fair, serving the cached pages quickly hasn’t historically been the issue…

liz avatar

Plus, of course, we can’t cache the forum (which is a big database user) because users need to be able to see new replies on it!

Tim White avatar

As another commenter said, changing from Apache to something like Nginx will improve performance significantly. (I was recently looking at some benchmarks between different webservers, you really do need to think about something other than Apache for high load sites)
Also, using Microcaching for things like forums also help.
Most users won’t know that they are viewing “old” cached information, so you can serve “old” (say upto a minute out of date) cached information to most users, and only server newer information to users who post. (Simple check if a user has posted as normally you create a cookie when they post)
Also, if you look at Pagespeed reports, there is heaps more you can be doing to decrease server load. Like allowing the browser to cache some items such as pictures and some of the javascript files (just need to set a Expiration as none is being sent currently).

Other than Pete, I’m sure there are some keen enthusiasts here who are also system admins that would be willing to help a charity…. You know where to contact me.

Kevin Mulder avatar

100 GB Ram, that means, that they put 400 board B in cloud … :)

Morgaine avatar

Which also tots up to $10,000 (about half the likely cost of such a pimped out server) and a cumulative 280 GHz which is likely to be about 10 times that of the server.

Of course this is a silly comparison in a thousand other respects, but still, perhaps there is a viable role for Rpis in web clusters of some kind, at least static ones.

What about comparative figures for power consumption?

Tim avatar

What is the purpose of 4TB of hard drive if all the DB fit in less than 100GB?

JamesH avatar

Futureproofing. Disk space is pretty cheap, might as well put it in there whilst you are upgrading the server. Then you don’t need to go offline to add them later.

Jason Ozolins avatar

It also happens that often if you buy enough disks to get a certain number of IOs per second from them in a RAID setup, you end up with a lot of capacity on that RAID array without even trying. Disk capacity has grown like crazy but IOPS per disk has grown at a much lower rate… a cluster filesystem that I help run has more space for metadata storage that we’ll ever use, just because of the number of disks required to get the IOPS numbers we wanted.

steviewevie avatar

Sooo much faster now, thank you so much, well done Pete !

Jay avatar

It doesn’t help the server when you use WordPress…

D.Q. avatar

I really hope you are not paying for that!

JamesH avatar

For what?

D.Q. avatar

For that server. Some of mythic beast servers cost 6K pounds a month! We are paying 0.3K to a local company for dedicated hosting with half the ram and reasonable disk space.

JamesH avatar

Well, we have to pay something….not the sort of thing you get for free.

Ryan avatar

Mate, you need to fix your software and site. I have Nginx on a 1.6Ghz Atom with 1GB RAM that pushes 1500+ requests per second for static HTML.

Is there really any reason why you need 96GB RAM or even a database to serve what essentially should be a few static pages? I absolutely guarantee you I could manage your traffic on a sub-£100 VPS no problem whatsoever.

GluffiS avatar

Switching from Apache to Nginx will give you a speed boost. Or putting some Varnish servers in front of the Apache.

JamesH avatar

Please get facts straight before telling other people how to do their jobs. It’s not just a static blog site – it’s has a very active forum and a commenting system. Please refrain from insulting people, it WILL get you banned.

deejay31 avatar

Wow, that sure is alot of RAM!
Hopefully the servers’ usage will get nowhere near that amount!
Can’t wait for the RPi’s to come back in stock and me get my hands on one.
As for @Ryan, has he not seen the comments system, or the Forum, or the Powered by WordPress banner?

Comments are closed