When we released our first Raspberry Pi OS image based on Debian Bullseye last week, we pointed to a change that is hugely important to people who have written code to use cameras with Raspberry Pi: the driver that Raspberry Pi uses to access camera modules has been replaced with libcamera.
These very significant changes mean less closed-source code, and they make it easier for people outside of Raspberry Pi to develop new camera hardware and software; but they also mean that new Raspberry Pi OS releases will no longer support the familiar raspicam apps and Picamera Python library.
In the place of this older camera system is the new and almost entirely open source camera stack based on standard Linux frameworks such as V4L2 (Video for Linux) and libcamera. Our kernel drivers have been moving in this direction for some time too, and have just recently taken further large strides towards the preferred new Media Controller architecture.
But the principal difference that users will notice is that OS releases from Bullseye onwards will no longer support the older camera system and applications, and Raspberry Pi’s libcamera-apps will be built and pre-installed instead. Before we go further, note that Raspberry Pi OS Buster is still available to download if you’re not ready to use Bullseye. If you are using camera applications with your Raspberry Pi, we recommend you take some time to weigh up whether to move to Bullseye at this point. This blog post considers why, and why not, you might want to do so.
What are libcamera-apps?
Libcamera-apps are designed to copy most of the functionality that users will know from raspistill, raspivid and raspiyuv. There are some unavoidable differences, which are examined in greater detail here. The new applications include:
- libcamera-hello – a simple “hello world” application which starts a camera preview stream and displays it on the screen.
- libcamera-jpeg – a simple application to run a preview window and then capture high-resolution still images.
- libcamera-still – a more complex still image capture application which emulates more of the features of raspistill.
- libcamera-vid – a video capture application.
- libcamera-raw – a basic application for capturing raw (unprocessed Bayer) frames directly from the sensor.
- libcamera-detect – this application is not built by default, but users can build it if they have TensorFlow Lite installed on their Raspberry Pi. It captures JPEG images when certain objects are detected.
Why should I use libcamera-apps?
We’d encourage users to move to Bullseye because libcamera-apps bring numerous benefits:
- They offer improved image quality, which, furthermore, a user can tweak to their own tastes.
- We can fix bugs and develop new features – all of which was extremely difficult in the proprietary Broadcom stack. For example, we’re very excited to be planning autofocus functionality in the near future.
- Because it’s open source, third parties can fix problems and add new features too – we’re happy to consider pull requests in our Github repository. (Contributions to the kernel or to libcamera itself need to be upstreamed in the usual way.)
- It’s much easier to add support for new cameras, and indeed for third-party cameras – a number are supported already (including the Sony imx290, imx327, and imx378, and the Omnivision ov9281). We’re keen to work with vendors, and more are already in the pipeline.
- The applications are designed to be easy to understand, so that users can customise them to suit their own requirements.
- We show how the camera applications can be augmented with powerful third-party image processing libraries, such as OpenCV and TensorFlow Lite.
- We provide an image post-processing framework that includes examples of motion detection, HDR (High Dynamic Range) imaging, face and object detection, pose estimation, and image segmentation. We’d be delighted if users would like to contribute more!
- It is fully supported in the 64-bit version of Raspberry Pi OS.
Nevertheless, libcamera and Raspberry Pi’s libcamera-apps remain a work in progress. Reasons for staying with an older OS release and continuing with the legacy camera system include:
- There is no Python interface yet. An alternative to the old Picamera, imaginatively named Picamera2, is in development. This too will integrate much more directly with established Python libraries to access things like windowing and graphics functions. Picamera2 will be developed by Raspberry Pi (unlike Picamera itself, which is actually third-party code), and this will facilitate both support and continuing future development.
- There is currently no support for stereo imaging within libcamera, though it is in our future development plans.
- Users of low-powered Raspberry Pis (such as Zero) who wish to use X Windows at the same time may get better camera performance with the legacy stack, as this does more work on the GPU and less on the ARM cores (which can struggle with X Windows in any case). Note that libcamera-apps still work well on these systems when X Windows is not running, or when a live video window is not required.
- There are still a few known issues in libcamera.
As mentioned further up, for those wishing to use it, the previous Buster release is still available to download.
How do I try libcamera?
Users of the new Bullseye OS will find that libcamera-apps are pre-installed and will work with no further intervention. You don’t even have to “enable the camera” any more, though you do still have to plug one in!
Note that there are a couple of known issues with the initial release:
- The preview when running under X Windows does not work on Raspberry Pi Zero or Raspberry Pi 1, 2 or 3 devices without this workaround. Users of Raspberry Pi 4, or those not using X Windows, are unaffected.
- The Raspberry Pi Camera Module 2, Raspberry Pi Camera Module 2 NoIR, and Raspberry Pi High Quality Camera are not yet working on the new Raspberry Pi Zero 2 W.
Both these problems will be fixed shortly.
Meanwhile Buster users with an up-to-date version of the OS can install libcamera-apps from the apt repositories.
More information on getting started with the new applications can be found on our official documentation page, where there are many example commands.