News

All news

Quake III bounty: we have a winner!

At the end of February, Broadcom announced the release of full documentation for the VideoCore IV graphics core, and a complete source release of the graphics stack for the BCM21553 cellphone chip. To celebrate, we offered a $10k prize to the first person to port this codebase to the BCM2835 application processor that sits at the heart of the Raspberry Pi, and to get Quake 3 (which already runs on the Pi) running on the newly open ARM driver, rather on the closed-source VPU driver. Our hope was that the ported driver would be a helpful reference for anyone working on a Mesa/Gallium3D driver for VideoCore IV.

Hands up if you spent far too long playing this when you were young.

I’m delighted to say that we have a winner. Simon Hall is a longtime Pi hacker, who also produced the first ARMv6-accelerated copies-and-fills library back in 2012 and wrote the DMA kernel module that we integrate in our Raspbian releases. The prize couldn’t have gone to a more fitting recipient.

So, without further ado, here are Simon’s instructions for getting the driver up and running.

SETTING UP THE DEVICE

You will need:

a Raspberry Pi, preferably a 512MB version, with the latest Raspbian
a network connection
a monitor capable of displaying 1080p
an SD card, at least 8GB (10GB is recommended)

We need plenty of space to build the kernel. Compiling will take around 12 hours, so it is helpful to overclock the Pi for this task. We also require the latest firmware, and the necessary packages we’re going to use to build the code.

Note: We’re going to use gcc 4.7, as the code generated is 10% faster than with 4.6. 4.8 is 10% faster still, but this is not available on Raspbian. If you cross-compile you can get better frame times.

Enter the raspi-config utility with:

sudo raspi-config

Expand the filesystem, set the overclock to at least medium (900 MHz), and reboot. Now perform an update with:

sudo rpi-update

and reboot again. We need to install several packages. Enter the following command to do this:

sudo apt-get -y install gcc make bc screen ncurses-dev g++-4.7 libsdl1.2-dev

FETCHING THE SOFTWARE

Enter the following commands to retrieve the necessary software from GitHub:

git clone --depth=1 https://github.com/raspberrypi/linux.git
git clone https://github.com/simonjhall/challenge
git clone https://github.com/simonjhall/dma
git clone https://github.com/raspberrypi/quake3.git

BUILDING THE KERNEL

This will take around 10 hours with all kernel modules. Pruning the modules to the bare minimum can improve compile times if you wish. Enter the following commands:

cd linux
git apply ~/challenge/kernel/patch.diff
zcat /proc/config.gz > .config
make oldconfig
make menuconfig

Enter “general setup”, select “local version”, enter the string “simon” and then exit to the main menu and save. Now build the kernel with:

make ARCH=arm

INSTALLING THE KERNEL

Enter the following commands to install the new kernel:

sudo make ARCH=arm modules_install
sudo cp arch/arm/boot/Image /boot/kernel_simon.img
sudo su
echo kernel=kernel_simon.img >> /boot/config.txt
echo gpu_mem=128 >> /boot/config.txt
reboot

When the devices comes back up, verify that the new kernel is loaded with the following command:

uname -a

You should see something similar to this:

Linux raspberrypi 3.10.33simon+ #1 PREEMPT Sat Mar 22 09:49:59 UTC 2014 armv6l x

BUILDING THE REST

Enter the following commands to build the rest of the software:

cd ~/quake3
git apply ~/challenge/quake/patch.diff
./build.sh
cd ~/dma
cp ~/challenge/kernel/module/dmaer.c .
make
./install.sh

Next, verify that the module has installed with this command:

tail /var/log/kern.log

You should see something similar to this:

Mar 23 15:22:45 raspberrypi kernel: [ 20.814750] smsc95xx 1-1.1:1.0 eth0:lin1
Mar 23 15:22:45 raspberrypi kernel: [ 21.376702] bcm2835-cpufreq: switching td
Mar 23 15:22:45 raspberrypi kernel: [ 21.376710] bcm2835-cpufreq: switching td
Mar 23 15:22:46 raspberrypi kernel: [ 24.472575] Adding 102396k swap on /var/S
Mar 23 15:25:02 raspberrypi kernel: [ 143.984835] 20c00000 f2c00000 deadbeef
Mar 23 15:25:02 raspberrypi kernel: [ 143.984866] major device number 248
Mar 23 15:25:02 raspberrypi kernel: [ 143.984890] vma list size 12, page list 6
Mar 23 15:25:02 raspberrypi kernel: [ 143.984909] allocated dma channel 4(f208
Mar 23 15:25:02 raspberrypi kernel: [ 143.985242] qpu ENABLED
Mar 23 15:25:02 raspberrypi kernel: [ 143.985264] V3D identify test: V3D versi2

Now enter the following commands:

cd ~/challenge/source
make
sudo make install

SETTING UP THE GAME

First of all you must ensure that you have the Quake 3 Arena data files on your Pi. You require the ‘point release’ pak files installed. There are various ways to do this but you could either transfer them from another machine with SCP, or copy them across on a USB stick. Copy the files into a folder called ‘baseq3’. This should now contain pak files numbered from 0 to 8 (eg pak1.pk3).

Next, enter the following commands:

sudo mkdir /root/.q3a
sudo mv baseq3/ /root/.q3a/
cd ~/quake3/build/release-linux-arm/
sudo mknod char_dev c 100 0
sudo cp ~/challenge/quake/demo.cfg /root/.q3a/baseq3/

RUNNING THE GAME

Enter the game folder with the following command:

cd ~/quake3/build/release-linux-arm/

Run the game using this command:

sudo ./ioquake3.arm +exec demo

If you wish to play the game after a reboot, you must run the following commands to re-load the necessary files:

cd ~/dma
./install.sh

If you see multi-second pauses of the game, this is because the system is paging to swap! You can see this by running top at the same time, and watch the swap usage jump during a spike. Close some running programs to alleviate this problem. Running the game without gdb and loading minimal kernel modules will prevent swapping.

85 comments

Congratulations Simon!

Awesome Job! Congratulations and Thanks a lot for all your hard work! Well deserved!

I think the RPi will make a better streaming device than the Apple TV … with the new driver even more optimised in a few months the RaspBMC build will be even more snappier .. (not that it already isnt :P )

Well…. no.

In video/audio quality, power, etc. The Pi (as it is right now) has nothing to do vs an Apple TV.

I’d love to see that, but it’s not gonna happen any time soon, unfortunately.

On the other hand, the Pi is already better in some aspects (content, flexibility).

One question – why build the kernel on Pi itself and not use cross compiling?

I’d imagine as not everybody who would want to compile the new kernel wants to set up a cross-compiler although I’d imagine that it would be a lot faster to cross-compile. And the instruction for the pi would be the same for everyone whereas cross-compiling would have lots of different instructions.

Because the instructions for doing so are much easier than for cross compiling. If you really want to then go ahead…

Gordon

Kernel cross-compiling infrastructure is actually really quite good. And once you have the toolchain set up it should be a breeze.

Sorry, my mind glitched there for a second.

I imagined a world in which there were no OS X or Windows users to worry about, this is not the case.

I have a cross toolchain set up for ARMv5, in fact, I have it by accident(I’ve never used it). Just now I confirmed that it works perfectly out of the box though.
If only it could be this way on OS X and Windows.

Care to provide instructions? Because I’m sure myself and others have no idea where to start with that.

Yay Simon!

Well done Simon. I wondered if this might be what JamesH was hinting about last week.

Nope.

Congrats!!!!
Well done

Congrats! LOL 10 h compile time. I love the “because you can attitude” :-)

When we could expect this to be integrated in Rasbian and other distributions?

same question! what’s it’s ETA for normal use?

This is a really exciting first step for a fully FOSS graphics stack on the Raspberry Pi (and a great achievement for Simon Hall), but there’s a way to go before we’d want to ship this as the default. If the kernel patch can be made minimally invasive and toggleable at runtime, I for one would be interested in providing it as a switchable option for people wanting to experiment. Of course a bigger question is what direction should FOSS graphics on Raspberry Pi (VideoCore IV) take. Is the best way to continue development of the BCM21553 code generously open sourced by Broadcom or would it be better to use it as a reference for integration of VideoCoreIV support for Mesa/Gallium?

What would be cool to have is a gui dev-kit to show each of the open ‘cores’ working in unison and independently if at all possable, mainly for educational use.

First of all congratscSimon!!!
I do have a few questions though:
1 when will it be integrated into raspbian (i hope soon)?

2 will it only affect the game or will it generally affect the gui performance?

3 can you pl. Provide a youtube link cuz i cant wait 10hrs !!!

See my reply above regarding ‘integration into Raspbian’ (I speak about the Foundation Raspbian image you get from the downloads page). This graphics driver will only be used by applications using OpenGL ES, just like the current VideoCore IV graphics driver.

so whenever it is released with a new raspbian image, will there be any possibility of upgrading an old one or will we have to reflash our sdcards?

I always try to provide an upgrade path for people with older images. If it needs anything more than an apt-get update/upgrade it will be documented on the blog post announcing the image (possibly by me dropping by the comments).

Great news and took a lot less time than I expected

Not really understanding the finer points of video drivers I’ve a few questions.

Is this implementation only good for Quake3 or can it accelerate other applications?

Will this affect the Desktop environment?

Permit improved acceleration of the Desktop?

If yes to above can this happen with X or is still something needed for the Wayland work?

How about Pygame/SDL applications?

Wondering how useful this is and what work would need to be done to make it into a ‘normal release.

Again, congratulations.

Bloody good work!

Are there any performance increases to be found by using the newer code game wise?

Perhaps, perhaps not. Note that the graphics stack has moved from the Videocore vector/scaler processor which runs at 250Mhz, to the Arm at 700Mhz. But the ARM is also running all the GUI, apps etc. So the likelihood is that this stuff might be slower once the workload is taken in to account – you are now not taking advantage of the videocore CPU to run stuff in parallel.

This is an important point. This is an important milestone for FOSS graphics on the Raspberry Pi, but for the reason James states you’re likely to see some performance degradation. Very likely stability regressions as well. This isn’t something that’s going to directly benefit the end user right now, but is obviously super helpful for anybody wanting to further develop the driver or indeed a Mesa/Gallium VC4 driver.

James – When you say “the graphics stack has moved from the Videocore vector/scaler processor which runs at 250Mhz, to the Arm at 700Mhz”, do you mean the local graphics-data structures/variables (including those declared static) that are used to make calls into OGLES, etc., or something else? I haven’t waded through all of the GPU docs yet, so it’s not clear to me yet what’s where in a typical Python program that’s making OGLES calls (e.g., Pi3D) and if that differs from the C OGLES examples in /opt/vc/src/hello_pi. This could have some very interesting ramifications, although it also suggests a 256 MB Pi might need a bit more care and feeding if the GPU has 128+ MB allocated and there’s some fat, lazy stuff sitting in ARM CPU RAM, especially if swapping is disabled. Thanks! The Other James ; )

Does this mean hardware support is now possible for software like handbrake, making it possible to trancode videos on the pi using the GPU? That would be really awsome!

No, the codecs and their HW blocks have not been opened due to licences implications.

First let me say AWESOME!!
I have a question, if this game works will other Quake Engine based games also work?

congratulations on the win but can I be lazy and just ask someone to create an image of the Sd card instead of having to compile it myself :)

Great news!

Now let’s all buy even more RaspberryPis to convince Broadcom that opening the specs was a smart move.

Now let’s all buy even more RaspberryPis to convince Broadcom that opening the specs was a smart move.
On a related note, as nobody seems to have mentioned it yet, if you do have multiple Pis you should be able to use distcc to build the kernel in some fraction of the 10h time that it would take on a single Pi. While I haven’t built the Pi’s kernel like this, I have built other kernels with a mix of distcc and cross-compilers on a bunch of machines each with different CPU architectures.
The instructions for using distcc are actually quite simple and it shouldn’t take more than 10 to 20 minutes to read up on it and set it up properly. A quick search turned up instructions for doing this on Gentoo, but it should be the same for all Linux platforms:
http://www.gentoo-wiki.info/HOWTO_Compile_Kernel_With_Distcc_and_CCache
That page also includes setting up ccache, which is a method of caching compiled object files so that they don’t have to be recompiled the next time you run a make and the original source file wasn’t modified. It’s useful if you’re changing the source, but most readers here won’t be doing that. It could still be useful if/when the upstream git tree is updated, though, as it will save you from having to do a lot of recompilation (though you may want to have more than the recommended 10Gb storage space if you’re caching compiled files with ccache).

Why not crosscompile kernel and modules? There’s are tool Crostool-NG (http://crosstool-ng.org/) which allow it very easily:
https://github.com/paczor/raspberrypi/blob/master/README.md

Great effort and congratulations Simon!

On a side note: Does the recent code drop changes the situation concerning DSI? Will the community be eventually able to use any DSI panel (with panel-specific quirks) or is this out of scope?

Thanks.

Oot of scope – the code release was for the graphics side of the chip, not the display or camera.

fantastic news !! Does it mean there is a chance there will be a version of android for the pi ?

Yes, it does. We’re not developing a version of Android here (it doesn’t fit with our educational mission), but this enables the community that wants Android to develop for the Pi themselves, where previously the closed drivers prevented them doing that work.

somebody make a youtube vid of you playing q3 with this kernel, thank you.

This is brilliant. You’ve just published something that sparked an interest in the RPi from my fourteen year old. I, simply, asked him “Would you like your RPi to run Quake?” and got a positive answer.

With a list of tasks I can take checkpoints to see how he’s going.

When it comes to the 10 hour task we’ll leave that running over night.

The Pi has always been able to run Quake3, using the closed source Videocore drivers. It was one of the first things I did with my 256mb first release version Pi. It runs quite admirably at 1080p (and that was with soft-float, before Raspbian, I’m sure it runs a bit better now)

I wonder what the speed is with the closed driver. Just for comparison.

I think the closed driver is about twice as fast.

But that has obviously had a lot of work done on optimising it.

I’m sure the open driver could be optimised much further (and as it’s open, anyone is free to improve it).

Welcome free software –
Toward free hardware

Freedom to Raspberry Pi !

Indeed

Glory to Arstotzka!

Congratulations Simon!!! ;)
-Grex

The new style sheet rocks.

Thanks! We worked really hard on it.

Okay, here I am thinking Rpi go hacked. IE it’s intentional…

TBH, it’s for more readable IMHO.

Argh! While I generally like immersive designs, this new design for the site is painful, unprofessional, and very off-putting. If I may suggest, perhaps have the site use the old theme, and make this theme an option or an Easter Egg. When designing a home page, keep in mind that people will be visiting who aren’t interested in the Pi, but just want to know what its about. Assume your audience is stupid and you’ll have a very successful design. Assume your audience is educated and knowledgeable and everything falls to bits. I’m in college for this sort of thing, and this is an excelent example of what NOT to do.

I think it meant to be an April fools joke

“Assume your audience is educated and knowledgeable and everything falls to bits”

I think you’re right there :)

We have all gone to college smart guy.

Might be worth it to look at a nearby Calendar old boy ;)

Ummmm, I don’t want to be the one who said, “The Emperor isn’t wearing any clothes!”, but I think that in the process of “upgrading” the kernel, Simon blew out all of our GPUs!!! Mine refuses to show anything but Christmas-colored Courier in 12-point anymore! For this we need parallel pipelines???

GREAT FUN, FOLKS!!! First, they release the Pi on the world, then they get Broadcom to allow open-sourcing of the GPU driver, and now this. What’s next, Zork … in 3-D lettering??? :lol:

Oh, rats, not only are the text codes for emoticons suppressed, so are the images. Time for the v2.0000001 bug fix! : D

I fail to see the bug you’re referring to?

If you insert any of the standard emoticons such as :lol: ( : lol : without spaces ), it appears that the text of the emoticon is suppressed (by WordPress?) and, of course, there is no graphic icon displayed. Now, watch and see that the :lol: above is displayed, just to mess with me! Here’s a test of all of the emoticons, in case some of them aren’t suppressed: :D :) ;) :( :o :shock: :? 8-) :lol: :x :P :oops: :cry: :evil: :twisted: :roll: :!: :?: :idea: :arrow: :| :mrgreen: :geek: :ugeek:

See? Only :geek: and :ugeek: made it through unscathed. Release the emoticons!!! : D

Competition rules were for GLESv1, not GLESv2. Is Simon going to finish the job and make this work with 24bit and hWnd != 11 ?

April fools? time zones…? i hope not, i have 2 raspberry pi

If i compile for 10 hours and see april fools… i’m gona puke.

@Xeon: The news is from yesterday.

Congratulation! It would nice to hear if there exists any other programmers or teams who try to win the competition.

Fantastic! Congratulations Simon!

Congratulations Simon

Why are you compiling the kernel on the raspberry itself?
When you cross-compile it with multi-core option enabled you get your new kernel binaries in less than a half hour.

I did this for several other ARM Platforms but never for a raspberry, but the shouldn’t be a difference.

Did anyone get this to work with kernel version 3.10.36? I followed these instructions and when quake3 runs, it’s just a garbled screen…

I’m having the same problem. I’d love to try with 3.10.33, if only I hadn’t just spent 12 hours compiling…

Yep… just did this… same boat. Turned off OC and same result still… Lame. Guess I’ll reimage the card again and go find something else to think is cool.

I was wondering why the name Simon Hall was so familiar, and then I realised he was the guy that ported Quake and Quake II to the DS way back! Absolute legend in the DS homebrew community, and seems to have a bit of a thing for porting Quake :D

Can we use PC Minecraft now then? With Mesa3D OpenGL?

All the cool software for the Pi need hours to setup. My wishlist is Weston/Wayland with your Opensource driver, and Epiphany browser with youtube support. Hazzle free as upgrade package.

Does this mean that the Pi will be able to boot without a blob?

Yes. If you use this driver. I am not aware of it being included by default in any repos or with Raspbian, correct me if I’m wrong anyone. I would really like to see a more official way to do this for future use :)

In the mean time the next chance I get I will be compiling this and loading it on my Pi. Thanks Broadcom, Raspberry Pi Foundation and Simon for liberating the Pi! :D

Can someone Upload an Compiled SD Image (Ready2Run) ?
Would be awesome.

Mailto:mactep@live.de

What is the rate of frames?

Hi,
first of all congratz! :)

I could use some of advice,
when I am building quake following error appears:

/usr/bin/ld: error: lib/libSDL.so uses VFP register arguments, output does not
/opt/vc/lib/libvchiq_arm.so: error: undefined reference to ‘sem_getvalue’
/opt/vc/lib/libvchiq_arm.so: error: undefined reference to ‘sem_init’
/opt/vc/lib/libvchiq_arm.so: error: undefined reference to ‘sem_destroy’
/opt/vc/lib/libvchiq_arm.so: error: undefined reference to ‘sem_post’
/opt/vc/lib/libvchiq_arm.so: error: undefined reference to ‘sem_wait’

am I missing something important?
are the others getting something similar?
or did I just simply messed something up?

Thanks,
Luke

So I followed everything and built the kernel just fine. When I compiled the dmaer.c it built fine but I’m getting an error running the install.sh in ~/DMA. I’m not in front of my pi right now. But the v3d (I think is what it is) module is not correct or won’t start. And checking the Kern.log I’m definitely not getting the same as the example output. Any ideas?

Retried the install.sh in the DMA and got it to work(?). but when I try to fire up Q3 I get this:

% sudo ./ioquake3.arm +exec demo

ioq3 1.35 linux-arm May 8 2014
—– FS_Startup —–
Current search path:
/root/.q3a/baseq3/pak8.pk3 (9 files)
/root/.q3a/baseq3/pak7.pk3 (4 files)
/root/.q3a/baseq3/pak6.pk3 (64 files)
/root/.q3a/baseq3/pak5.pk3 (7 files)
/root/.q3a/baseq3/pak4.pk3 (272 files)
/root/.q3a/baseq3/pak3.pk3 (4 files)
/root/.q3a/baseq3/pak2.pk3 (148 files)
/root/.q3a/baseq3/pak1.pk3 (26 files)
/root/.q3a/baseq3/pak0.pk3 (3539 files)
/root/.q3a/baseq3
./baseq3/pak8.pk3 (9 files)
./baseq3/pak7.pk3 (4 files)
./baseq3/pak6.pk3 (64 files)
./baseq3/pak5.pk3 (7 files)
./baseq3/pak4.pk3 (272 files)
./baseq3/pak3.pk3 (4 files)
./baseq3/pak2.pk3 (148 files)
./baseq3/pak1.pk3 (26 files)
./baseq3/pak0.pk3 (3539 files)
./baseq3

———————-
8146 files in pk3 files
execing default.cfg
couldn’t exec q3config.cfg
couldn’t exec autoexec.cfg
Hunk_Clear: reset the hunk ok
—– Client Initialization —–
Couldn’t read q3history.
Your network rate is too slow for VoIP.
Set ‘Data Rate’ to ‘LAN/Cable/xDSL’ in ‘Setup/System/Network’ and restart.
Until then, VoIP is disabled.
—– Initializing Renderer —-
——————————-
QKEY found.
—– Client Initialization Complete —–
—– R_Init —–
SDL using driver “x11”
Open display[0]…
Display is 1280 x 720
init dispmanx type 1 dims 1280×720
ioquake3.arm: source/brcm_usrlib/dag/vmcsx/middleware/khronos/common/2708/dispman_connection.c:80: get_resource: Assertion `msg.m_tag.m_recvDataSize == 0x80000008′ failed.
Received signal 6, exiting…
—– CL_Shutdown —–
RE_Shutdown( 1 )
———————–

Not sure what I’m missing here.

By the way: Here’s what my kern.log looks like:

May 8 19:04:00 PiBang kernel: [24138.304969] failed to turn on and identify V3D
May 8 19:19:41 PiBang kernel: [25079.498867] closing dmaer device, cache stats: 0 hits 0 misses
May 8 19:19:41 PiBang kernel: [25079.528761] qpu DISABLED
May 8 19:19:41 PiBang kernel: [25079.693770] 20c00000 f2c00000 deadbeef
May 8 19:19:41 PiBang kernel: [25079.693797] major device number 248
May 8 19:19:41 PiBang kernel: [25079.693819] vma list size 12, page list size 2008, page size 4096
May 8 19:19:41 PiBang kernel: [25079.693838] allocated dma channel 4 (f2007400), initial state 00000008
May 8 19:19:41 PiBang kernel: [25079.693953] qpu ENABLED
May 8 19:19:41 PiBang kernel: [25079.693973] V3D identify test: �� version 222
May 8 19:19:41 PiBang kernel: [25079.698480] failed to turn on and identify V3D

I’m not using Raspbian, I’m using PiBang, which honestly is isn’t that different. It’s built from the raspbian repo.

I also had this bad V3D identify result and found that it went away after updating the /boot/ files to the latest versions from the raspberrypi/firmware github repo

Nice job. Though it does not meet the requirements set out in the original request.

The original challenge required a fully open source implementation. Using the Linux kernel and the ID Software engine both restrict the source do to the limits of the GPL and thus this is not a FULLY OPEN SOURCE implementation. This is a well done restricted Open Source implementation though.

I do look forward to someone completing the challenge as originally outlined.

Do not get me wrong that is nice work.

This means that after I build the kernel I’ll have hw acceleration on my Pi? I’m not interested on running Quake, but interested in the better performance of the Pi.
Thanks.

Comments are closed

News

Latest posts

This massive nose sniffs things then prints a description of the smell

Paragraphica creates "photos" based on your location

Dragonfly Spectral Line Mapper

Refurbished vending machine

Next Post

New website design is here!

Previous Post

Home automation. For your Galaxy-class starship.

Share this post

85 comments

Chinmay Thosar

Palantir

brox

Piers

Gordon

Aaron Hamilton

Aaron Hamilton

Stanto

tap5

JamesH

Werewolfc

Gregor

Marius

tzj

ric96

ric96

winkleink

EdwinJ85

JamesH

Jim Manley

Alfred

JamesH

ChicagoBob123

Rich Steed

Gerry

Declan Malone

paczor

entropy

JamesH

Nikki81

nakamoto

mm0zct

Christopher

dom

nonino

Grexit

Bill Smith

STrRedWolf

entropy

Dio

Jessie

Davespice

Jim Manley

Jim Manley

Jim Manley

Jim Manley

hexameron

Lolren

Xeon

Yggdrasil

Fabiano Furtado

Stefan Spack

Erik S. Bailey

Tobi R.

Yoriku

Ryan

Benjamin Lambert

Friedemann Wolpert

John

Mactep

Pete

Asuraku

Asuraku

Asuraku

Bernhard M.

DavidS

David Piedra