Raspberry Pi Documentation

Camera

Introducing the Raspberry Pi Cameras

Raspberry Pi currently sell two types of camera board: an 8MP device and a 12MP High Quality (HQ) camera. The 8MP device is also available in NoIR form without an IR filter. The original 5MP device is no longer available from Raspberry Pi.

All Raspberry Pi cameras are capable of taking high-resolution photographs, along with full HD 1080p video, and can be fully controlled programmatically. This documentation describes how to use the camera in various scenarios, and how to use the various software tools.

Once installed, there are various ways the cameras can be used. The simplest option is to use one of the provided camera applications, such as libcamera-still or raspistill.

libcamera and the legacy Raspicam camera stack

Note

Raspberry Pi is transitioning from a legacy camera software stack based on proprietary Broadcom GPU code to an open-source stack based on libcamera. Raspberry Pi OS images from Bullseye onwards will contain only the libcamera-based stack. Raspberry Pi OS images up to and including Buster will contain the legacy Raspicam stack, though the libcamera stack and applications can be installed using apt, or built by following the normal build instructions.

Users are encouraged to use the newest OS images and the libcamera-based stack because:

  • It will continue to be developed moving forward.

  • Raspberry Pi and 3rd parties can fix bugs and problems in the camera stack.

  • Raspberry Pi and 3rd parties can add new features to the camera stack.

  • It is much easier to add support for new cameras.

  • 3rd parties can add support directly for their own cameras.

  • Nearly all aspects of the camera tuning can be changed by users.

  • It integrates much more conveniently with other standard Linux APIs.

  • Raspberry Pi supply a set of libcamera-apps which emulate most of the features of the legacy applications.

  • It provdes a feature-rich post-processing framework integrating OpenCV and TensorFlow Lite.

  • Libcamera makes it easier to control the parameters of the image sensor and the camera system.

  • It is fully supported on 64-bit operating systems.

Reasons to consider staying with an older OS and using the legacy Raspicam stack might include:

  • It may perform better on Raspberry Pi 2 and Raspberry Pi Zero devices, as it offloads more to the GPU and is less dependent on the ARM cores.

  • libcamera is still missing certain features, most notably Python bindings. Whilst this work is in progress, users who need a Python interface (such as Picamera) will have to stay with the legacy stack for the time being.

Apart from some general information on setting up the camera, most of this documentation is divided into sections on libcamera and the libcamera-apps, and the legacy Raspicam apps. Once you’ve installed your camera module, please click on one of these two links.

Note
More information can be found in the blog post discussing the transition.

Camera Modules

The Raspberry Pi Camera Modules are official products from Raspberry Pi. The original 5-megapixel model was released in 2013, and an 8-megapixel Camera Module v2 was released in 2016. For both iterations, there are visible light and infrared versions. A 12-megapixel High Quality Camera was released in 2020. There is no infrared version of the HQ Camera, however the IR Filter can be removed if required.

Installing a Raspberry Pi camera

Warning
Cameras are sensitive to static. Earth yourself prior to handling the PCB. A sink tap or similar should suffice if you don’t have an earthing strap.

Connecting the Camera

The flex cable inserts into the connector labelled CAMERA on the Raspberry Pi, which is located between the Ethernet and HDMI ports. The cable must be inserted with the silver contacts facing the HDMI port. To open the connector, pull the tabs on the top of the connector upwards, then towards the Ethernet port. The flex cable should be inserted firmly into the connector, with care taken not to bend the flex at too acute an angle. To close the connector, push the top part of the connector towards the HDMI port and down, while holding the flex cable in place.

We have created a video to illustrate the process of connecting the camera. Although the video shows the original camera on the original Raspberry Pi 1, the principle is the same for all camera boards:

Depending on the model, the camera may come with a small piece of translucent blue plastic film covering the lens. This is only present to protect the lens while it is being mailed to you, and needs to be removed by gently peeling it off.

Preparing the Software

Before proceeding, we recommend ensuring that your kernel, GPU firmware and applications are all up to date. Please follow the instructions on keeping your operating system up to date.

Then, please follow the relevant setup instructions either for libcamera or for Raspicam.

Hardware Specification

Camera Module v1 Camera Module v2 HQ Camera

Net price

$25

$25

$50

Size

Around 25 × 24 × 9 mm

38 x 38 x 18.4mm (excluding lens)

Weight

3g

3g

Still resolution

5 Megapixels

8 Megapixels

12.3 Megapixels

Video modes

1080p30, 720p60 and 640 × 480p60/90

1080p30, 720p60 and 640 × 480p60/90

1080p30, 720p60 and 640 × 480p60/90

Linux integration

V4L2 driver available

V4L2 driver available

V4L2 driver available

C programming API

OpenMAX IL and others available

OpenMAX IL and others available

Sensor

OmniVision OV5647

Sony IMX219

Sony IMX477

Sensor resolution

2592 × 1944 pixels

3280 × 2464 pixels

4056 x 3040 pixels

Sensor image area

3.76 × 2.74 mm

3.68 x 2.76 mm (4.6 mm diagonal)

6.287mm x 4.712 mm (7.9mm diagonal)

Pixel size

1.4 µm × 1.4 µm

1.12 µm x 1.12 µm

1.55 µm x 1.55 µm

Optical size

1/4"

1/4"

Full-frame SLR lens equivalent

35 mm

S/N ratio

36 dB

Dynamic range

67 dB @ 8x gain

Sensitivity

680 mV/lux-sec

Dark current

16 mV/sec @ 60 C

Well capacity

4.3 Ke-

Depth of field

approx. 1 m to infinity

adjustable with supplied tool

N/A

Focal length

3.60 mm +/- 0.01

3.04 mm

Depends on lens

Horizontal field of view

53.50 +/- 0.13 degrees

62.2 degrees

Depends on lens

Vertical field of view

41.41 +/- 0.11 degrees

48.8 degrees

Depends on lens

Focal ratio (F-Stop)

2.9

2.0

Depends on lens

Software Features

libcamera stack

Picture formats

JPEG, JPEG + DNG (raw), BMP, PNG, YUV420, RGB888

Video formats

raw h.264 (accelerated), MJPEG

Post-processing

User-definable image effects, customisable DRC and HDR, motion detection, OpenCV integration, TensorFlowLite integration

Exposure modes

normal, short, long, fixed fps, customisable

Metering modes

centre-weighted, average, spot, customisable

Automatic white balance modes

off, auto, incandescent, tungsten, fluorescent, indoor, daylight, cloudy, customisable

Triggers

Keypress, UNIX signal, timeout

Extra modes

timelapse, circular buffer, motion detection, segmented video, many features through flexible post-processing

Legacy stack

Picture formats

JPEG (accelerated), JPEG + RAW, GIF, BMP, PNG, YUV420, RGB888

Video formats

raw h.264 (accelerated)

Effects

negative, solarise, posterize, whiteboard, blackboard, sketch, denoise, emboss, oilpaint, hatch, gpen, pastel, watercolour, film, blur, saturation

Exposure modes

auto, night, nightpreview, backlight, spotlight, sports, snow, beach, verylong, fixedfps, antishake, fireworks

Metering modes

average, spot, backlit, matrix

Automatic white balance modes

off, auto, sun, cloud, shade, tungsten, fluorescent, incandescent, flash, horizon

Triggers

Keypress, UNIX signal, timeout

Extra modes

demo, burst/timelapse, circular buffer, video with motion vectors, segmented video, live preview on 3D models

HQ Camera IR Filter Transmission

The HQ Camera uses a Hoya CM500 infrared filter. Its transmission characteristics are as represented in the following graph.

CM500 Transmission Graph

Maximum Exposure Times

The maximum exposure times of the three official Raspberry Pi cameras are given in the table below.

Module Max exposure (seconds)

V1 (OMx5647)

6

V2 (IMX219)

10

HQ (IMX417)

230

Mechanical Drawings

  • Camera Module v2 PDF

  • HQ Camera Module PDF

  • HQ Camera Module lens mount PDF

Schematics

  • Camera Module v2 PDF

  • HQ Camera Module PDF

Raspberry Pi HQ Camera Filter Removal

The High Quality Camera contains an IR filter, which is used to reduce the camera’s sensitivity to infrared light. This ensures that outdoor photos look more natural. However, some nature photography can be enhanced with the removal of this filter; the colours of sky, plants, and water can be affected by its removal. The camera can also be used without the filter for night vision in a location that is illuminated with infrared light.

Warning
This procedure cannot be reversed: the adhesive that attaches the filter will not survive being lifted and replaced, and while the IR filter is about 1.1mm thick, it may crack when it is removed. Removing it will void the warranty on the product. Nevertheless, removing the filter will be desirable to some users.

To remove the filter:

  1. Work in a clean and dust-free environment, as the sensor will be exposed to the air. camera sensor

  2. Unscrew the two 1.5 mm hex lock keys on the underside of the main circuit board. Be careful not to let the washers roll away. There is a gasket of slightly sticky material between the housing and PCB which will require some force to separate. camera gasket

  3. Lift up the board and place it down on a very clean surface. Make sure the sensor does not touch the surface.

  4. Before completing the next step, read through all of the steps and decide whether you are willing to void your warranty. Do not proceed unless you are sure that you are willing to void your warranty.

  5. Turn the lens around so that it is "looking" upwards and place it on a table. You may try some ways to weaken the adhesive, such as a little isopropyl alcohol and/or heat (~20-30 C). Using a pen top or similar soft plastic item, push down on the filter only at the very edges where the glass attaches to the aluminium - to minimise the risk of breaking the filter. The glue will break and the filter will detach from the lens mount. camera ir filter

  6. Given that changing lenses will expose the sensor, at this point you could affix a clear filter (for example, OHP plastic) to minimize the chance of dust entering the sensor cavity. camera protective filter

  7. Replace the main housing over the circuit board. Be sure to realign the housing with the gasket, which remains on the circuit board.

  8. The nylon washer prevents damage to the circuit board; apply this washer first. Next, fit the steel washer, which prevents damage to the nylon washer.

  9. Screw down the two hex lock keys. As long as the washers have been fitted in the correct order, they do not need to be screwed very tightly.

  10. Note that it is likely to be difficult or impossible to glue the filter back in place and return the device to functioning as a normal optical camera.

libcamera and libcamera-apps

Introduction

libcamera is a new software library aimed at supporting complex camera systems directly from the Linux operating system. In the case of the Raspberry Pi it enables us to drive the camera system directly from open source code running on ARM processors. The proprietary code running on the Broadcom GPU, and to which users have no access at all, is almost completely by-passed.

libcamera presents a C++ API to applications and works at the level of configuring the camera and then allowing an application to request image frames. These image buffers reside in system memory and can be passed directly to still image encoders (such as JPEG) or to video encoders (such as h.264), though such ancilliary functions as encoding images or displaying them are strictly beyond the purview of libcamera itself.

For this reason Raspberry Pi supplies a small set of example libcamera-apps. These are simple applications, built on top of libcamera, and are designed largely to emulate the function of the legacy stack built on Broadcom’s propretary GPU code (some users will recognise these legacy applications as raspstill and raspivid). The applications we provide are:

  • libcamera-hello A simple "hello world" application which starts a camera preview stream and displays it on the screen.

  • libcamera-jpeg A simple application to run a preview window and then capture high resolution still images.

  • libcamera-still A more complex still image capture application which emulates more of the features of raspistill.

  • libcamera-vid A video capture application.

  • libcamera-raw A basic application for capturing raw (unprocessed Bayer) frames directly from the sensor.

  • libcamera-detect This application is not built by default, but users can build it if they have TensorFlow Lite installed on their Pi. It captures JPEG images when certain objects are detected.

Raspberry Pi’s libcamera-apps are not only command line applications that make it easy to capture images and video from the camera, they are also examples of how users can create their own libcamera-based applications with custom functionality to suit their own requirements. The source code for the libcamera-apps is freely available under a BSD 2-Clause licence at https://github.com/raspberrypi/libcamera-apps.

More about libcamera

libcamera is an open source Linux community project. More information is available at the libcamera website.

The libcamera source code can be found and checked out from the official libcamera repository.

Underneath the libcamera core, Raspberry Pi provides a custom pipeline handler, which is the layer that libcamera uses to drive the sensor and ISP (Image Signal Processor) on the Raspberry Pi itself. Also part of this is a collection of well-known control algorithms, or IPAs (Image Processing Algorithms) in libcamera parlance, such as AEC/AGC (Auto Exposure/Gain Control), AWB (Auto White Balance), ALSC (Auto Lens Shading Correction) and so on.

All this code is open source and now runs on the Pi’s ARM cores. There is only a very thin layer of code on the GPU which translates Raspberry Pi’s own control parameters into register writes for the Broadcom ISP.

Raspberry Pi’s implementation of libcamera supports not only the three standard Raspberry Pi cameras (the OV5647 or V1 camera, the IMX219 or V2 camera and the IMX477 or HQ camera) but also third party senors such as the IMX290, IMX327, OV9281, IMX378. Raspberry Pi is keen to work with vendors who would like to see their sensors supported directly by libcamera.

Moreover, Raspberry Pi supplies a tuning file for each of these sensors which can be edited to change the processing performed by the Pi hardware on the raw images received from the image sensor, including aspects like the colour processing, the amount of noise suppression or the behaviour of the control algorithms.

For further information on libcamera for the Raspberry Pi, please consult the Tuning Guide for the Raspberry Pi cameras and libcamera.

Getting Started

Using the camera for the first time

When running a Raspberry Pi OS based on Bullseye, the 5 basic libcamera-apps are already installed. In this case, official Raspberry Pi cameras will also be detected and enabled automatically.

You can check that everything is working by entering:

libcamera-hello

You should see a camera preview window for about 5 seconds.

Users running Buster will need to install one of the libcamera-apps packages first and then configure their /boot/config.txt file with the appropriate overlay for the connected camera. Be aware that libcamera and the legacy raspicam stack cannot operate at the same time - to return to the legacy stack after using libcamera you will need to comment out the dtoverlay change you made and reboot the system.

Note
Pi 3 and older devices may not by default be using the correct display driver. Refer to the /boot/config.txt file and ensure that either dtoverlay=vc4-fkms-v3d or dtoverlay=vc4-kms-v3d is currently active. Please reboot if you needed to change this.

If you do need to alter the configuration

You may need to alter the camera configuration in your /boot/config.txt file if:

  • You are using a 3rd party camera (the manufacturer’s instructions should explain the changes you need to make).

  • You are using an official Raspberry Pi camera but wish to use a non-standard driver/overlay.

  • You are using an older version (Buster or earlier) of the operating system (in which case we would recommend upgrading first if this is possible).

Tip
You can find out what operating system version you have by entering cat /etc/os-release and checking (among others) the VERSION field. Note that Buster is version 10 and Bullseye is version 11.

If you do need to add your own dtoverlay, the following are currently recognised.

Camera Module In /boot/config.txt

V1 camera (OV5647)

dtoverlay=ov5647

V2 camera (IMX219)

dtoverlay=imx219

HQ camera (IMX477)

dtoverlay=imx477

IMX290 and IMX327

dtoverlay=imx290,clock-frequency=74250000 or dtoverlay=imx290,clock-frequency=37125000 (both modules share the imx290 kernel driver; please refer to instructions from the module vendor for the correct frequency)

IMX378

dtoverlay=imx378

OV9281

dtoverlay=ov9281

To override the automatic camera detection, Bullseye users will also need to delete the entry

camera_auto_detect=1

if present in the config.txt file. If you need to do edit this file then your Raspberry Pi will need to be rebooted.

Note
Setting camera_auto_detect=0 disables the boot time detection completely.

libcamera-hello

libcamera-hello is the equivalent of a "hello world" application for the camera. It starts the camera, displays a preview window, and does nothing else. For example

libcamera-hello

should display a preview window for about 5 seconds. The -t <duration> option lets the user select how long the window is displayed, where <duration> is given in milliseconds. To run the preview indefinitely, use:

libcamera-hello -t 0

The preview can be halted either by clicking the window’s close button, or using Ctrl-C in the terminal.

Options

libcamera-apps uses a 3rd party library to interpret command line options. This includes long form options where the option name consists of more than one character preceeded by --, and short form options which can only be a single character preceded by a single -. For the most part option names are chosen to match those used by the legacy raspicam applications with the exception that we can no longer handle multi-character option names with a single -. Any such legacy options have been dropped and the long form with -- must be used instead.

The options are classified broadly into 3 groups, namely those that are common, those that are specific to still images, and those that are for video encoding. They are supported in an identical manner across all the applications where they apply.

Please refer to the command line options documentation for a complete list.

The Tuning File

Raspberry Pi’s libcamera implementation includes a tuning file for each different type of camera module. This is a file that describes or "tunes" the parameters that will be passed to the algorithms and hardware to produce the best image quality. libcamera is only able to determine automatically the image sensor being used, not the module as a whole - even though the whole module affects the "tuning".

For this reason it is sometimes necessary to override the default tuning file for a particular sensor.

For example, the NOIR (no IR-filter) versions of sensors require different AWB settings to the standard versions, so the IMX219 NOIR should be run using

libcamera-hello --tuning-file /usr/share/libcamera/ipa/raspberrypi/imx219_noir.json

If you are using a Soho Enterprises SE327M12 module you should use

libcamera-hello --tuning-file /usr/share/libcamera/ipa/raspberrypi/se327m12.json

Notice how this also means that users can copy an existing tuning file and alter it according to their own preferences, so long as the --tuning-file parameter is pointed to the new version.

Finally, the --tuning-file parameter, in common with other libcamera-hello command line options, applies identically across all the libcamera-apps.

Preview Window

Most of the libcamera-apps display a preview image in a window. When X Windows is not running it will draw directly to the display using Linux DRM (Direct Rendering Manager), otherwise it will attempt to use X Windows. Both paths use zero-copy buffer sharing with the GPU, and a consequence of this is that X forwarding is not supported.

For this reason there is a third kind of preview window which does support X forwarding, and can be requested with the --qt-preview option. This implementation does not benefit from zero-copy buffer sharing nor from 3D acceleration which makes it computationally expensive (especially for large previews), and so is not normally recommended.

Note
Older systems using Gtk2 may, when linked with OpenCV, produce Glib-GObject errors and fail to show the Qt preview window. In this case please (as root) edit the file /etc/xdg/qt5ct/qt5ct.conf and replace the line containing style=gtk2 with style=gtk3.

The preview window can be suppressed entirely with the -n (--nopreview) option.

The --info-text option allows the user to request that certain helpful image information is displayed on the window title bar using "% directives". For example

libcamera-hello --info-text "red gain %rg, blue gain %bg"

will display the current red and blue gain values.

For the HQ camera, use --info-text "%focus" to display the focus measure, which will be helpful for focusing the lens.

A full description of the --info-text parameter is given in the command line options documentation.

libcamera-jpeg

libcamera-jpeg is a simple still image capture application. It deliberately avoids some of the additional features of libcamera-still which attempts to emulate raspistill more fully. As such the code is significantly easier to understand, and in practice still provides many of the same features.

To capture a full resolution JPEG image use

libcamera-jpeg -o test.jpg

which will display a preview for about 5 seconds, and then capture a full resolution JPEG image to the file test.jpg.

The -t <duration> option can be used to alter the length of time the preview shows, and the --width and --height options will change the resolution of the captured still image. For example

libcamera-jpeg -o test.jpg -t 2000 --width 640 --height 480

will capture a VGA sized image.

Exposure Control

All the libcamera-apps allow the user to run the camera with fixed shutter speed and gain. For example

libcamera-jpeg -o test.jpg -t 2000 --shutter 20000 --gain 1.5

would capture an image with an exposure of 20ms and a gain of 1.5x. Note that the gain will be applied as analogue gain within the sensor up until it reaches the maximum analogue gain permitted by the kernel sensor driver, after which the remainder will be applied as digital gain.

Raspberry Pi’s AEC/AGC algorithm allows applications to specify exposure compensation, that is, the ability to make images darker or brighter by a given number of stops, as follows

libcamera-jpeg --ev -0.5 -o darker.jpg
libcamera-jpeg --ev 0 -o normal.jpg
libcamera-jpeg --ev 0.5 -o brighter.jpg
Further remarks on Digital Gain

Digital gain is applied by the ISP (the Image Signal Processor), not by the sensor. The digital gain will always be very close to 1.0 unless:

  • The total gain requested (either by the --gain option, or by the exposure profile in the camera tuning) exceeds that which can be applied as analogue gain within the sensor. Only the extra gain required will be applied as digital gain.

  • One of the colour gains is less than 1 (note that colour gains are applied as digital gain too). In this case the advertised digital gain will settle to 1 / min(red_gain, blue_gain). This actually means that one of the colour channels - just not the green one - is having unity digital gain applied to it.

  • The AEC/AGC is changing. When the AEC/AGC is moving the digital gain will typically vary to some extent to try and smooth out any fluctuations, but it will quickly settle back to its "normal" value.

libcamera-still

libcamera-still is very similar to libcamera-jpeg but supports more of the legacy raspistill options. As before, a single image can be captured with

libcamera-still -o test.jpg

Encoders

libcamera-still allows files to be saved in a number of different formats. It supports both png and bmp encoding. It also allows files to be saved as a binary dump of RGB or YUV pixels with no encoding or file format at all. In these latter cases the application reading the files will have to understand the pixel arrangement for itself.

libcamera-still -e png -o test.png
libcamera-still -e bmp -o test.bmp
libcamera-still -e rgb -o test.data
libcamera-still -e yuv420 -o test.data

Note that the format in which the image is saved depends on the -e (equivalently --encoding) option and is not selected automatically based on the output file name.

Raw Image Capture

Raw images are the images produced directly by the image sensor, before any processing is applied to them either by the ISP (Image Signal Processor) or any of the CPU cores. For colour image sensors these are usually Bayer format images. Note that raw images are quite different from the processed but unencoded RGB or YUV images that we saw earlier.

To capture a raw image use

libcamera-still -r -o test.jpg

Here, the -r option (also --raw) indicates to capture the raw image as well as the JPEG. In fact, the raw image is the exact image from which the JPEG was produced. Raw images are saved in DNG (Adobe Digital Negative) format and are compatible with many standard applications, such as dcraw or RawTherapee. The raw image is saved to a file with the same name but the extension .dng, thus test.dng in this case.

These DNG files contain metadata pertaining to the image capture, including black levels, white balance information and the colour matrix used by the ISP to produce the JPEG. This makes these DNG files much more convenient for later "by hand" raw conversion with some of the aforementioned tools. Using exiftool shows all the metadata encoded into the DNG file:

File Name                       : test.dng
Directory                       : .
File Size                       : 24 MB
File Modification Date/Time     : 2021:08:17 16:36:18+01:00
File Access Date/Time           : 2021:08:17 16:36:18+01:00
File Inode Change Date/Time     : 2021:08:17 16:36:18+01:00
File Permissions                : rw-r--r--
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Make                            : Raspberry Pi
Camera Model Name               : /base/soc/i2c0mux/i2c@1/imx477@1a
Orientation                     : Horizontal (normal)
Software                        : libcamera-still
Subfile Type                    : Full-resolution Image
Image Width                     : 4056
Image Height                    : 3040
Bits Per Sample                 : 16
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Samples Per Pixel               : 1
Planar Configuration            : Chunky
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 2 1 1 0
Black Level Repeat Dim          : 2 2
Black Level                     : 256 256 256 256
White Level                     : 4095
DNG Version                     : 1.1.0.0
DNG Backward Version            : 1.0.0.0
Unique Camera Model             : /base/soc/i2c0mux/i2c@1/imx477@1a
Color Matrix 1                  : 0.8545269369 -0.2382823821 -0.09044229197 -0.1890484985 1.063961506 0.1062747385 -0.01334283455 0.1440163847 0.2593136724
As Shot Neutral                 : 0.4754476844 1 0.413686484
Calibration Illuminant 1        : D65
Strip Offsets                   : 0
Strip Byte Counts               : 0
Exposure Time                   : 1/20
ISO                             : 400
CFA Pattern                     : [Blue,Green][Green,Red]
Image Size                      : 4056x3040
Megapixels                      : 12.3
Shutter Speed                   : 1/20

We note that there is only a single calibrated illuminant (the one determined by the AWB algorithm even though it gets labelled always as "D65"), and that dividing the ISO number by 100 gives the analogue gain that was being used.

Very long exposures

To capture very long exposure images, we need to be careful to disable the AEC/AGC and AWB because these algorithms will otherwise force the user to wait for a number of frames while they converge. The way to disable them is to supply explicit values. Additionally, the entire preview phase of the capture can be skipped with the --immediate option.

So to perform a 100 second exposure capture, use

libcamera-still -o long_exposure.jpg --shutter 100000000 --gain 1 --awbgains 1,1 --immediate

For reference, the maximum exposure times of the three official Raspberry Pi cameras can be found in this table.

libcamera-vid

libcamera-vid is the video capture application. By default it uses the Raspberry Pi’s hardware H.264 encoder. It will display a preview window and write the encoded bitstream to the specified output. For example, to write a 10 second video to file use

libcamera-vid -t 10000 -o test.h264

The resulting file can be played with vlc (among other applications)

vlc test.h264

Note that this is an unpackaged video bistream, it is not wrapped in any kind of container format (such as an mp4 file). The --save-pts option can be used to output frame timestamps so that the bitstream can subsequently be converted into an appropriate format using a tool like mkvmerge.

libcamera-vid -o test.h264 --save-pts timestamps.txt

and then if you want an mkv file:

mkvmerge -o test.mkv --timecodes 0:timestamps.txt test.h264

Encoders

There is support for motion JPEG, and also for uncompressed and unformatted YUV420, for example

libcamera-vid -t 10000 --codec mjpeg -o test.mjpeg
libcamera-vid -t 10000 --codec yuv420 -o test.data

In both cases the --codec parameter determines the output format, not the extension of the output file.

The --segment parameter breaks output files up into chunks of the segment size (given in milliseconds). This is quite handy for breaking a motion JPEG stream up into individual JPEG files by specifying very short (1 millisecond) segments.

libcamera-vid -t 10000 --codec mjpeg --segment 1 -o test%05d.jpeg

Observe that the output file name is normally only sensible if we avoid over-writing the previous file every time, such as by using a file name that includes a counter (as above). More information on output file names is available below.

Network Streaming

UDP

To stream video using UDP, on the Raspberry Pi (server) use

libcamera-vid -t 0 --inline -o udp://<ip-addr>:<port>

where <ip-addr> is the IP address of the client, or multicast address (if appropriately configured to reach the client). On the client use (for example)

vlc udp://@:<port> :demux=h264

with the same <port> value.

TCP

Video can be streamed using TCP. To use the Pi as a server

libcamera-vid -t 0 --inline --listen -o tcp://0.0.0.0:<port>

and on the client

vlc tcp/h264://<ip-addr-of-server>:<port>

or alternatively

ffplay tcp://<ip-addr-of-server>:<port> -vf "setpts=N/30" -fflags nobuffer -flags low_delay -framedrop

for a 30 frames per second stream with low latency.

The Raspberry Pi will wait until the client connects, and then start streaming video.

RTSP

vlc is useful on the Pi for formatting an RTSP stream, though there are other RTSP servers available.

libcamera-vid -t 0 --inline -o - | cvlc stream://dev/stdin --sout '#rtp{sdp=rtsp://:8554/stream1}' :demux=h264

and this can be played with

vlc rtsp://<ip-addr-of-server>:8554/stream1

In all cases, the preview window on the server (the Raspberry Pi) can be suppressed with the -n (--nopreview) option. Note also the use of the --inline option which forces the stream header information to be included with every I (intra) frame. This is important so that a client can correctly understand the stream if it missed the very beginning.

libcamera-raw

libcamera-raw is like a video recording application except that it records raw Bayer frames directly from the sensor. It does not show a preview window. For a 2 second raw clip use

libcamera-raw -t 2000 -o test.raw

The raw frames are dumped with no formatting information at all, one directly after another. The application prints the pixel format and image dimensions to the terminal window so that the user can know how to interpret the pixel data.

By default the raw frames are saved in a single (potentially very large) file. As we saw previously, the --segement option can be used conveniently to direct each to a separate file.

libcamera-raw -t 2000 --segment 1 -o test%05d.raw

In good conditions (using a fast SSD) libcamera-raw can get close to writing 12MP HQ camera frames (18MB of data each) to disk at 10 frames per second. It writes the raw frames with no formatting in order to achieve these speeds; it has no capability to save them as DNG files (like libcamera-still). If you want to be sure not to drop frames you could reduce the framerate slightly using the --framerate option, for example

libcamera-raw -t 5000 --width 4056 --height 3040 -o test.raw --framerate 8

libcamera-detect

libcamera-detect is not supplied by default in any Raspberry Pi OS distribution, but can be built by users who have installed TensorFlow Lite. In this case, please refer to the libcamera-apps build instructions. You will need to run cmake with -DENABLE_TFLITE=1.

This application runs a preview window and monitors the contents using a Google MobileNet v1 SSD (Single Shot Detector) neural network that has been trained to identify about 80 classes of objects using the Coco dataset. It should recognise people, cars, cats and many other objects.

Its starts by running a preview window, and whenever the target object is detected it will perform a full resolution JPEG capture, before returning back to the preview mode to continue monitoring. It provides a couple of additional command line options that do not apply elsewhere:

--object <name>

Detect objects with the given <name>. The name should be taken from the model’s label file.

--gap <number>

Wait at least this many frames after a capture before performing another. This is necessary because the neural network does not run on every frame, so it is best to give it a few frames to run again before considering another capture.

Please refer to the TensorFlow Lite object detector section for more general information on how to obtain and use this model. But as an example, you might spy secretly on your cats while you are away with:

libcamera-detect -t 0 -o cat%04d.jpg --lores-width 400 --lores-height 300 --post-process-file object_detect_tf.json --object cat

Common Command Line Options

The following options apply across all the libcamera-apps with similar or identical semantics, unless noted otherwise.

	--help,		-h		Print help information for the application

The --help option causes every application to print its full set of command line options with a brief synopsis of each, and then quit.

	--version			Print out a software version number

All libcamera-apps will, when they see the --version option, print out a version string both for libcamera and libcamera-apps and then quit, for example:

libcamera-apps build: ca559f46a97a 27-09-2021 (14:10:24)
libcamera build: v0.0.0+3058-c29143f7
	--timeout,	-t		Delay before application stops automatically <milliseconds>

The --timeout option specifies how long the application runs before it stops, whether it is recording a video or showing a preview. In the case of still image capture, the application will show the preview window for this long before capturing the output image.

If unspecified, the default value is 5000 (5 seconds). The value zero causes the application to run indefinitely.

Example: libcamera-hello -t 0

Preview window

	--preview,	-p		Preview window settings <x,y,w,h>

Sets the size and location of the preview window (both X Windows and DRM versions). It does not affect the resolution or aspect ratio of images being requested from the camera. The camera images will be scaled to the size of the preview window for display, and will be pillar/letter-boxed to fit.

Example: libcamera-hello -p 100,100,500,500

Letterboxed preview image
	--fullscreen,	-f		Fullscreen preview mode

Forces the preview window to use the whole screen, and the window will have no border or title bar. Again the image may be pillar/letter-boxed.

Example libcamera-still -f -o test.jpg

	--qt-preview			Use Qt-based preview window

The preview window is switched to use the Qt-based implementation. This option is not normally recommended because it no longer uses zero-copy buffer sharing nor GPU acceleration and is therefore very expensive, however, it does support X forwarding (which the other preview implementations do not).

The Qt preview window does not support the --fullscreen option. Generally it is advised to try and keep the preview window small.

Example libcamera-hello --qt-preview

	--nopreview,	-n		Do not display a preview window

The preview window is suppressed entirely.

Example libcamera-still -n -o test.jpg

	--info-text			Set window title bar text <string>

The supplied string is set as the title of the preview window (when running under X Windows). Additionally the string may contain a number of % directives which are substituted with information from the image metadata. The permitted directives are

Directive Substitution

%frame

The sequence number of the frame

%fps

The instantaneous frame rate

%exp

The shutter speed used to capture the image, in microseconds

%ag

The analogue gain applied to the image in the sensor

%dg

The digital gain applied to the image by the ISP

%rg

The gain applied to the red component of each pixel

%bg

The gain applied to the blue component of each pixel

%focus

The focus metric for the image, where a larger value implies a sharper image

When not provided, the --info-text string defaults to "#%frame (%fps fps) exp %exp ag %ag dg %dg".

Example: libcamera-hello --info-test "Focus measure: %focus

Image showing focus measure

Camera Resolution and Readout

	--width				Capture image width <width>
	--height			Capture image height <height>

These numbers specify the output resolution of the camera images captured by libcamera-still, libcamera-jpeg and libcamera-vid.

For libcamera-raw, it affects the size of the raw frames captured. Where a camera has a 2x2 binned readout mode, specifying a resolution not larger than this binned mode will result in the capture of 2x2 binned raw frames.

For libcamera-hello these parameters have no effect.

Examples:

libcamera-vid -o test.h264 --width 1920 --height 1080 will capture 1080p video.

libcamera-still -r -o test.jpg --width 2028 --height 1520 will capture a 2028x1520 resolution JPEG. When using the HQ camera the sensor will be driven in its 2x2 binned mode so the raw file - captured in test.dng - will contain a 2028x1520 raw Bayer image.

	--viewfinder-width		Capture image width <width>
	--viewfinder-height		Capture image height <height>

These options affect only the preview (meaning both libcamera-hello and the preview phase of libcamera-jpeg and libcamera-still), and specify the image size that will be requested from the camera for the preview window. They have no effect on captured still images or videos. Nor do they affect the preview window as the images are resized to fit.

Example: libcamera-hello --viewfinder-width 640 --viewfinder-height 480

	--rawfull			Force sensor to capture in full resolution mode

This option forces the sensor to be driven in its full resolution readout mode for still and video capture, irrespective of the requested output resolution (given by --width and --height). It has no effect for libcamera-hello.

Using this option often incurs a frame rate penalty, as larger resolution frames are slower to read out.

Example: libcamera-raw -t 2000 --segment 1 --rawfull -o test%03d.raw will cause multiple full resolution raw frames to be captured. On the HQ camera each frame will be about 18MB in size. Without the --rawfull option the default video output resolution would have caused the 2x2 binned mode to be selected, resulting in 4.5MB raw frames.

	--lores-width			Low resolution image width <width>
	--lores-height			Low resolution image height <height>

libcamera allows the possibiliy of delivering a second lower resolution image stream from the the camera system to the application. This stream is available in both the preview and the video modes (i.e. libcamera-hello and the preview phase of libcamera-still, and libcamera-vid), and can be used, among other things, for image analysis. For still captures, the low resolution image stream is not available.

The low resolution stream has the same field of view as the other image streams. If a different aspect ratio is specified for the low resolution stream, then those images will be squashed so that the pixels are no longer square.

During video recording (libcamera-vid), specifying a low resolution stream will disable some extra colour denoise processing that would normally occur.

Example: libcamera-hello --lores-width 224 --lores-height 224

Note that the low resolution stream is not particularly useful unless used in conjunction with image post-processing.

	--hflip				Read out with horizontal mirror
	--vflip				Read out with vertical flip
	--rotation			Use hflip and vflip to create the given rotation <angle>

These options affect the order of read-out from the sensor, and can be used to mirror the image horizontally, and/or flip it vertically. The --rotation option permits only the value 0 or 180, so note that 90 or 270 degree rotations are not supported. Moreover, --rotation 180 is identical to --hflip --vflip.

Example: libcamera-hello --vflip --hflip

	--roi				Select a crop (region of interest) from the camera <x,y,w,h>

The --roi (region of interest) option allows the user to select a particular crop from the full field of view provided by the sensor. The coordinates are specified as a proportion of the available field of view, so that --roi 0,0,1,1 would have no effect at all.

The --roi parameter implements what is commonly referred to as "digital zoom".

Example libcamera-hello --roi 0.25,0.25,0.5,0.5 will select exactly a quarter of the total number of pixels cropped from the centre of the image.

Camera Control

The following options affect the image processing and control algorithms that affect the camera image quality.

	--sharpness			Set image sharpness <number>

The given <number> adjusts the image sharpness. The value zero means that no sharpening is applied, the value 1.0 uses the default amount of sharpening, and values greater than 1.0 use extra sharpening.

Example: libcamera-still -o test.jpg --sharpness 2.0

	--contrast			Set image contrast <number>

The given <number> adjusts the image contrast. The value zero produces minimum contrast, the value 1.0 uses the default amount of contrast, and values greater than 1.0 apply extra contrast.

Example: libcamera-still -o test.jpg --contrast 1.5

	--brightness			Set image brightness <number>

The given <number> adjusts the image brightness. The value -1.0 produces an (almost) black image, the value 1.0 produces an almost entirely white image and the value 0.0 produces standard image brightness.

Note that the brightness parameter adds (or subtracts) an offset from all pixels in the output image. The --ev option is often more appropriate.

Example: libcamera-still -o test.jpg --brightness 0.2

	--saturation			Set image colour saturation <number>

The given <number> adjusts the colour saturation. The value zero produces a greyscale image, the value 1.0 uses the default amount of sautration, and values greater than 1.0 apply extra colour saturation.

Example: libcamera-still -o test.jpg --saturation 0.8

	--ev				Set EV compensation <number>

Sets the EV compensation of the image in units of stops, in the range -10 to 10. Default is 0. It works by raising or lowering the target values the AEC/AGC algorithm is attempting to match.

Example: libcamera-still -o test.jpg --ev 0.3

	--shutter			Set the exposure time in microseconds <number>

The shutter time is fixed to the given value. The gain will still be allowed to vary (unless that is also fixed).

Note that this shutter time may not be achieved if the camera is running at a frame rate that is too fast to allow it. In this case the --framerate option may be used to lower the frame rate. The maximum possible shutter times for the official Raspberry Pi supported can be found in this table.

Using values above these maximums will result in undefined behaviour. Cameras will also have different minimum shutter times, though in practice this is not important as they are all low enough to expose bright scenes appropriately.

Example: libcamera-hello --shutter 30000

	--gain				Sets the combined analogue and digital gains <number>
	--analoggain			Synonym for --gain

These two options are actually identical, and set the combined analogue and digital gains that will be used. The --analoggain form is permitted so as to be more compatible with the legacy raspicam applications. Where the requested gain can be supplied by the sensor driver, then only analogue gain will be used. Once the analogue gain reaches the maximum permitted value, then extra gain beyond this will be supplied as digital gain.

Note that there are circumstances where the digital gain can go above 1 even when the analogue gain limit is not exceeded. This can occur when

  • Either of the colour gains goes below 1.0, which will cause the digital gain to settle to 1.0/min(red_gain,blue_gain). This means that the total digital gain being applied to any colour channel does not go below 1.0, as that would cause discolouration artifacts.

  • The digital gain can vary slightly while the AEC/AGC changes, though this effect should be only transient.

	--metering			Set the metering mode <string>

Sets the metering mode of the AEC/AGC algorithm. This may one of the following values

  • centre - centre weighted metering (which is the default)

  • spot - spot metering

  • average - average or whole frame metering

  • custom - custom metering mode which would have to be defined in the camera tuning file.

For more information on defining a custom metering mode, and also on how to adjust the region weights in the existing metering modes, please refer to the Tuning guide for the Raspberry Pi cameras and libcamera.

Example: libcamera-still -o test.jpg --metering spot

	--exposure			Set the exposure profile <string>

The exposure profile may be either normal or sport. Changing the exposure profile should not affect the overall exposure of an image, but the sport mode will tend to prefer shorter exposure times and larger gains to achieve the same net result.

Exposure profiles can be edited in the camera tuning file. Please refer to the Tuning guide for the Raspberry Pi cameras and libcamera for more information.

Example: libcamera-still -o test.jpg --exposure sport

	--awb				Set the AWB mode <string>

This option sets the AWB algorithm into the named AWB mode. Valid modes are:

Mode name Colour temperature

auto

2500K to 8000K

incandescent

2500K to 3000K

tungsten

3000K to 3500K

fluorescent

4000K to 4700K

indoor

3000K to 5000K

daylight

5500K to 6500K

cloudy

7000K to 8500K

custom

A custom range would have to be defined in the camera tuning file.

There is no mode that turns the AWB off, instead fixed colour gains should be specified with the --awbgains option.

Note that these values are only approximate, the values could vary according to the camera tuning.

For more information on AWB modes and how to define a custom one, please refer to the Tuning guide for the Raspberry Pi cameras and libcamera.

Example: libcamera-still -o test.jpg --awb tungsten

	--awbgains				Set fixed colour gains <number,number>

This option accepts a red and a blue gain value and uses them directly in place of running the AWB algorithm. Setting non-zero values here has the effect of disabling the AWB calculation.

Example: libcamera-still -o test.jpg --awbgains 1.5,2.0

	--denoise				Set the denoising mode <string>

The following denoise modes are supported:

  • auto - This is the default. It always enables standard spatial denoise. It uses extra fast colour denoise for video, and high quality colour denoise for stills capture. Preview does not enable any extra colour denoise at all.

  • off - Disables spatial and colour denoise.

  • cdn_off - Disables colour denoise.

  • cdn_fast - Uses fast color denoise.

  • cdn_hq - Uses high quality colour denoise. Not appropriate for video/viewfinder due to reduced throughput.

Note that even the use of fast colour denoise can result in lower framerates. The high quality colour denoise will normally result in much lower framerates.

Example: libcamera-vid -o test.h264 --denoise cdn_off

	--tuning-file				Specify the camera tuning to use <string>

This identifies the name of the JSON format tuning file that should be used. The tuning file covers many aspects of the image processing, including the AEC/AGC, AWB, colour shading correction, colour processing, denoising and so forth.

For more information on the camera tuning file, please consult the Tuning guide for the Raspberry Pi cameras and libcamera.

Example: libcamera-hello --tuning-file ~/my-camera-tuning.json

Output File Options

	--output,	-o			Output file name <string>

--output sets the name of the output file to which the output image or video is written. Besides regular file names, this may take the following special values:

  • - - write to stdout

  • udp:// - a string starting with this is taken as a network address for streaming

  • tcp:// - a string starting with this is taken as a network address for streaming

  • a string containing a %d directive is taken as a file name where the format directive is replaced with a count that increments for each file that is opened. Standard C format directive modifiers are permitted.

Examples:

libcamera-vid -t 100000 --segment 10000 -o chunk%04d.h264 records a 100 second file in 10 second segments, where each file is named chunk.h264 but with the inclusion of an incrementing counter. Note that %04d writes the count to a string, but padded up to a total width of at least 4 characters by adding leading zeroes.

libcamera-vid -t 0 --inline -o udp://192.168.1.13:5000 stream H.264 video to network address 192.168.1.13 on port 5000.

	--wrap					Wrap output file counter at <number>

When outputting to files with an incrementing counter (e.g. %d in the output file name), wrap the counter back to zero when it reaches this value.

Example: libcamera-vid -t 0 --codec mjpeg --segment 1 --wrap 100 -o image%d.jpg

	--flush					Flush output files immediately

--flush causes output files to be flushed to disk as soon as every frame is written, rather than waiting for the system to do it.

Example: libcamera-vid -t 10000 --flush -o test.h264

Post Processing Options

The --post-process-file option specifies a JSON file that configures the post-processing that the imaging pipeline applies to camera images before they reach the application. It can be thought of as a replacement for the legacy raspicam "image effects".

Post-processing is a large topic and admits the use of 3rd party software like OpenCV and TensorFlowLite to analyse and manipulate images. For more information, please refer to the section on post-processing.

Example: libcamera-hello --post-process-file negate.json

This might apply a "negate" effect to an image, if the file negate.json is appropriately configured.

Still Command Line Options

	--quality,	-q		JPEG quality <number>

Set the JPEG quality. 100 is maximum quality and 93 is the default. Only applies when saving JPEG files.

Example: libcamera-jpeg -o test.jpg -q 80

	--exif,		-x		Add extra EXIF tags <string>

The given extra EXIF tags are saved in the JPEG file. Only applies when saving JPEG files.

EXIF is supported using the libexif library and so there are some associated limitations. In particular, libexif seems to recognise a number of tags but without knowing the correct format for them. The software will currently treat these (incorrectly, in many cases) as ASCII, but will print a warning to the terminal. As we come across these they can be added to the table of known exceptions in the software.

Clearly the application needs to supply EXIF tags that contain specific camera data (like the exposure time). But for other tags that have nothing to do with the camera, a reasonable workaround would simply be to add them post facto, using something like exiftool.

Example: libcamera-still -o test.jpg --exif IDO0.Artist=Someone

	--timelapse			Time interval between timelapse captures <milliseconds>

This puts libcamera-still into timelapse mode where it runs according to the timeout (--timeout or -t) that has been set, and for that period will capture repeated images at the interval specified here. (libcamera-still only.)

Example: libcamera-still -t 100000 -o test%d.jpg --timelapse 10000 captures an image every 10s for about 100s.

	--framestart			The starting value for the frame counter <number>

When writing counter values into the output file name, this specifies the starting value for the counter.

Example: libcamera-still -t 100000 -o test%d.jpg --timelapse 10000 --framestart 1 captures an image every 10s for about 100s, starting at 1 rather than 0. (libcamera-still only.)

	--datetime			Use date format for the output file names

Use the current date and time to construct the output file name, in the form MMDDhhmmss.jpg, where MM = 2-digit month number, DD = 2-digit day number, hh = 2-digit 24-hour hour number, mm = 2-digit minute number, ss = 2-digit second number. (libcamera-still only.)

Example: libcamera-still --datetime

	--timestamp			Use system timestamps for the output file names

Uses the current system timestamp (the number of seconds since the start of 1970) as the the output file name. (libcamera-still only.)

Example: libcamera-still --timestamp

	--restart			Set the JPEG restart interval <number>

Sets the JPEG restart interval to the given value. Default is zero.

Example: libcamera-still -o test.jpg --restart 20

	--keypress,	-k		Capture image when Enter pressed

This switches libcamera-still into keypress mode. It will capture a still image either when the timeout expires or the Enter key is pressed in the terminal window. Typing x and Enter causes libcamera-still to quit without capturing.

Example: libcamera-still -t 0 -o test.jpg -k

	--signal,	-s		Capture image when SIGUSR1 received

This switches libcamera-still into signal mode. It will capture a still image either when the timeout expires or a SIGUSR1 is received. SIGUSR2 will cause libcamera-still to quit without capturing.

Example:

libcamera-still -t 0 -o test.jpg -s &

then

kill -SIGUSR1 $!

	--thumb				Set thumbnail parameters <w:h:q>

Sets the dimensions and quality parameter of the associated thumbnail image. The defaults are size 320x240 and quality 70.

Example: libcamera-still -o test.jpg --thumb 640:480:80

	--encoding,	-e		Set the still image codec <string>

Select the still image encoding to be used. Valid encoders are:

  • jpg - JPEG (the default)

  • png - PNG format

  • bmp - BMP format

  • rgb - binary dump of uncompressed RGB pixels

  • yuv420 - binary dump of uncompressed YUV420 pixels.

Note that this option determines the encoding and that the extension of the output file name is ignored for this purpose. However, for the --datetime and --timestamp options, the file extension is taken from the encoder name listed above. (libcamera-still only.)

Example: libcamera-still -e png -o test.png

	--raw,		-r		Save raw file

Save a raw Bayer file in DNG format alongside the usual output image. The file name is given by replacing the output file name extension by .dng. These are standard DNG files, and can be processed with standard tools like dcraw or RawTherapee, among others. (libcamera-still only.)

The image data in the raw file is exactly what came out of the sensor, with no processing whatsoever either by the ISP or anything else. The EXIF data saved in the file, among other things, includes:

  • exposure time

  • analogue gain (the ISO tag is 100 times the analogue gain used)

  • white balance gains (which are the reciprocals of the "as shot neutral" values)

  • the colour matrix used by the ISP.

	--latest			Make symbolic link to latest file saved <string>

This causes libcamera-still to make a symbolic link to the most recently saved file, thereby making it easier to identify. (libcamera-still only.)

Example: libcamera-still -t 100000 --timelapse 10000 -o test%d.jpg --latest latest.jpg

Video Command Line Options

	--quality,	-q		JPEG quality <number>

Set the JPEG quality. 100 is maximum quality and 50 is the default. Only applies when saving in MJPEG format.

Example: libcamera-vid --codec mjpeg -o test.mjpeg -q 80

	--bitrate,	-b		H.264 bitrate <number>

Set the target bitrate for the H.264 encoder, in bits per second. Only applies when encoding in H.264 format.

Example: libcamera-vid -b 10000000 --width 1920 --height 1080 -o test.h264

	--intra,	-g		Intra-frame period (H.264 only) <number>

Sets the frequency of I (Intra) frames in the H.264 bitstream, as a number of frames. The default value is 60.

Example: libcamera-vid --intra 30 --width 1920 --height 1080 -o test.h264

	--profile			H.264 profile <string>

Set the H.264 profile. The value may be baseline, main or high.

Example: libcamera-vid --width 1920 --height 1080 --profile main -o test.h264

	--level				H.264 level <string>

Set the H.264 level. The value may be 4, 4.1 or 4.2.

Example: libcamera-vid --width 1920 --height 1080 --level 4.1 -o test.h264

	--codec				Encoder to be used <string>

This can select how the video frames are encoded. Valid options are:

  • h264 - use H.264 encoder (the default)

  • mjpeg - use MJPEG encoder

  • yuv420 - output uncompressed YUV420 frames.

Examples:

libcamera-vid -t 10000 --codec mjpeg -o test.mjpeg

libcamera-vid -t 10000 --codec yuv420 -o test.data

	--keypress,	-k		Toggle between recording and pausing

Pressing Enter will toggle libcamera-vid between recording the video stream and not recording it (i.e. discarding it). The application starts off in the recording state, unless the --initial option specifies otherwise. Typing x and Enter causes libcamera-vid to quit.

Example: libcamera-vid -t 0 -o test.h264 -k

	--signal,	-s		Toggle between recording and pausing when SIGUSR1 received

The SIGUSR1 signal will toggle libcamera-vid between recording the video stream and not recording it (i.e. discarding it). The application starts off in the recording state, unless the --initial option specifies otherwise. SIGUSR2 causes libcamera-vid to quit.

Example:

libcamera-vid -t 0 -o test.h264 -s

then

kill -SIGUSR1 $!

	--initial			Start the application in the recording or paused state <string>

The value passed may be record or pause to start the application in, respectively, the recording or the paused state. This option should be used in conjunction with either --keypress or --signal to toggle between the two states.

Example: libcamera-vid -t 0 -o test.h264 -k --initial pause

	--split				Split multiple recordings into separate files

This option should be used in conjunction with --keypress or --signal and causes each recording session (inbetween the pauses) to be written to a separate file.

Example: libcamera-vid -t 0 --keypress --split --initial pause -o test%04d.h264

	--segment			Write the video recording into multiple segments <number>

This option causes the video recording to be split accross multiple files where the parameter gives the approximate duration of each file in milliseconds.

One convenient little trick is to pass a very small duration parameter (namely, --segment 1) which will result in each frame being written to a separate output file. This makes it easy to do "burst" JPEG capture (using the MJPEG codec), or "burst" raw frame capture (using libcamera-raw).

Example: libcamera-vid -t 100000 --segment 10000 -o test%04d.h264

	--circular			Write the video recording into a circular buffer.

The video recording is written to a circular buffer which is written to disk when the application quits. The size of the circular buffer is 4MB.

Example: libcamera-vid -t 0 --keypress --inline --circular -o test.h264

	--inline			Write sequence header in every I frame (H.264 only)

This option causes the H.264 sequence headers to be written into every I (Intra) frame. This is helpful because it means a client can understand and decode the video sequence from any I frame, not just from the very beginning of the stream. It is recommended to use this option with any output type that breaks the output into pieces (--segment, --split, --circular), or transmits the output over a network.

Example: libcamera-vid -t 0 --keypress --inline --split -o test%04d.h264

	--listen			Wait for an incoming TCP connection

This option is provided for streaming over a network using TCP/IP. Using --listen will cause libcamera-vid to wait for an incoming client connection before starting the video encode process, which will then be forwarded to that client.

Example: libcamera-vid -t 0 --inline --listen -o tcp://0.0.0.0:8123

Differences compared to Raspicam Apps

Whilst the libcamera-apps attempt to emulate most features of the legacy Raspicam applications, there are some differences. Here we list the principal ones that users are likely to notice.

  • The use of Boost program_options doesn’t allow multi-character short versions of options, so where these were present they have had to be dropped. The long form options are named the same, and any single character short forms are preserved.

  • libcamera-still and libcamera-jpeg do not show the capture image in the preview window.

  • libcamera performs its own camera mode selection, so the --mode option is not supported. It deduces camera modes from the resolutions requested. There is still work ongoing in this area.

  • The following features of the legacy apps are not supported as the code has to run on the ARM now. But note that a number of these effects are now provided by the post-processing mechanism.

    • opacity (--opacity)

    • image effects (--imxfx)

    • colour effects (--colfx)

    • annotation (--annotate, --annotateex)

    • dynamic range compression, or DRC (--drc)

  • stereo (--stereo, --decimate and --3dswap). There is no support in libcamera for stereo currently.

  • There is no image stabilisation (--vstab) (though the legacy implementation does not appear to do very much).

  • There are no demo modes (--demo).

  • The transformations supported are those that do not involve a transposition. 180 degree rotations, therefore, are among those permitted but 90 and 270 degree rotations are not.

  • There are some differences in the metering, exposure and AWB options. In particular the legacy apps conflate metering (by which we mean the "metering mode") and the exposure (by which we now mean the "exposure profile"). With regards to AWB, to turn it off you have to set a pair of colour gains (e.g. --awbgains 1.0,1.0).

  • libcamera has no mechanism to set the AWB into "grey world" mode, which is useful for "NOIR" camera modules. However, tuning files are supplied which switch the AWB into the correct mode, so for example, you could use libcamera-hello --tuning-file /usr/share/libcamera/ipa/raspberrypi/imx219_noir.json.

  • There is support for setting the exposure time (--shutter) and analogue gain (--analoggain or just --gain). There is no explicit control of the digital gain; you get this if the gain requested is larger than the analogue gain can deliver by itself.

  • libcamera has no understanding of ISO, so there is no --ISO option. Users should calculate the gain corresponding to the ISO value required (usually a manufacturer will tell you that, for example, a gain of 1 corresponds to an ISO of 40), and use the --gain parameter instead.

  • There is no support for setting the flicker period yet.

  • libcamera-still does not support burst capture. In fact, because the JPEG encoding is not multi-threaded and pipelined it would produce quite poor framerates. Instead, users are advised to consider using libcamera-vid in MJPEG mode instead (and --segment 1 can be used to force each frame into a separate JPEG file).

  • libcamera uses open source drivers for all the image sensors, so the mechanism for enabling or disabling on-sensor DPC (Defective Pixel Correction) is different. The imx477 (HQ cam) driver enables on-sensor DPC by default; to disable it the user should, as root, enter

echo 0 > /sys/module/imx477/parameters/dpc_enable

Post-Processing

libcamera-apps share a common post-processing framework. This allows them to pass the images received from the camera system through a number of custom image processing and image analysis routines. Each such routine is known as a post-processing stage and the description of exactly which stages should be run, and what configuration they may have, is supplied in a JSON file. Every stage, along with its source code, is supplied with a short example JSON file showing how to enable it.

For example, the simple negate stage (which "negates" all the pixels in an image, turning light pixels dark and vice versa) is supplied with a negate.json file that configures the post-processing pipeline to run it:

libcamera-hello --post-process-file /path/to/negate.json

Tip
Example JSON files can be found in the assets folder of the libcamera-apps repository at https://github.com/raspberrypi/libcamera-apps/tree/main/assets.

The negate stage is particularly trivial and has no configuration parameters of its own, therefore the JSON file merely has to name the stage, with no further information, and it will be run. Thus negate.json contains

{
    "negate":
    {
    }
}

To run multiple post-processing stages, the contents of the example JSON files merely need to be listed together, and the stages will be run in the order given. For example, to run the Sobel stage (which applies a Sobel filter to an image) followed by the negate stage we could create a custom JSON file containing

{
    "sobel_cv":
    {
        "ksize": 5
    },
    "negate":
    {
    }
}

The Sobel stage is implemented using OpenCV, hence cv in its name. Observe how it has a user-configurable parameter, ksize that specifies the kernel size of the filter to be used. In this case, the Sobel filter will produce bright edges on a black background, and the negate stage will turn this into dark edges on a white background, as shown.

Image with Sobel and negate

Some stages actually alter the image in some way, and this is their primary function (such as negate). Others are primarily for image analysis, and while they may indicate something on the image, all they really do is generate useful information. For this reason we also have a very flexible form of metadata that can be populated by the post-processing stages, and this will get passed all the way through to the application itself.

Image analysis stages often prefer to work on reduced resolution images. libcamera-apps are able to supply applications with a ready-made low resolution image provided directly by the ISP hardware, and this can be helpful in improving performance.

Furthermore, with the post-processing framework being completely open, Raspberry Pi welcomes the contribution of new and interesting stages from the community and would be happy to host them in our libcamera-apps repository. The stages that are currently available are documented below.

Note
The libcamera-apps supplied with the operating system will be built without any optional 3rd party libraries (such as OpenCV or TensorFlow Lite), meaning that certain post-processing stages that rely on them will not be enabled. To use these stages, please follow the instructions for building libcamera-apps for yourself.

negate stage

The negate stage requires no 3rd party libraries.

On a Raspberry Pi 3 device or a Raspberry Pi 4 running a 32-bit OS, it may execute more quickly if recompiled using -DENABLE_COMPILE_FLAGS_FOR_TARGET=armv8-neon. (Please see the build instructions.)

The negate stage has no user-configurable parameters.

Default negate.json file:

{
    "negate":
    {
    }
}

Example:

Image with negate

hdr stage

The hdr stage implements both HDR (high dynamic range) imaging and DRC (dynamic range compression). The terminology that we use here regards DRC as operating on single images, and HDR works by accumulating multiple under-exposed images and then performing the same algorithm as DRC.

The hdr stage has no dependencies on 3rd party libraries, but (like some other stages) may execute more quickly on Pi 3 or Pi 4 devices running a 32-bit OS if recompiled using -DENABLE_COMPILE_FLAGS_FOR_TARGET=armv8-neon (please see the build instructions). Specifically, the image accumulation stage will run quicker and result in fewer frame drops, though the tonemapping part of the process is unchanged.

The basic procedure is that we take the image (which in the case of HDR may be multiple images accumulated together) and apply an edge-preserving smoothing filter to generate a low pass (LP) image. We define the high pass (HP) image to be the difference between the LP image and the original. Next we apply a global tonemap to the LP image and add back the HP image. This procedure, in contrast to applying the tonemap directly to the original image, prevents us from squashing and losing all the local contrast in the resulting image.

It is worth noting that this all happens using fully-processed images, once the ISP has finished with them. HDR normally works better when carried out in the raw (Bayer) domain, as signals are still linear and have greater bit-depth. We expect to implement such functionality once libcamera exports an API for "re-processing" Bayer images that do not come from the sensor, but which application code can pass in.

In summary, the user-configurable parameters fall broadly into three groups: those that define the LP filter, those responsible for the global tonemapping, and those responsible for re-applying the local contrast.

num_frames

The number of frames to accumulate. For DRC (in our terminology) this would take the value 1, but for multi-frame HDR we would suggest a value such as 8.

lp_filter_strength

The coefficient of the low pass IIR filter.

lp_fiter_threshold

A piecewise linear function that relates the pixel level to the threshold that is regarded as being "meaningful detail".

global_tonemap_points

A list of points in the input image histogram and targets in the output range where we wish to move them. We define an inter-quantile mean (q and width), a target as a proportion of the full output range (target) and maximum and minimum gains by which we are prepared to move the measured inter-quantile mean (as this prevents us from changing an image too drastically).

global_tonemap_strength

Strength of application of the global tonemap.

local_pos_strength

A piecewise linear function that defines the gain applied to local contrast when added back to the tonemapped LP image, for positive (bright) detail.

local_neg_strength

A piecewise linear function that defines the gain applied to local contrast when added back to the tonemapped LP image, for negative (dark) detail.

local_tonemap_strength

An overall gain applied to all local contrast that is added back.

local_colour_scale

A factor that allows the output colours to be affected more or less strongly.

We note that the overall strength of the processing is best controlled by changing the global_tonemap_strength and local_tonemap_strength paramters.

The full processing takes between 2 and 3 seconds for a 12MP image on a Pi 4. The stage runs only on the still image capture, it ignores preview and video images. In particular, when accumulating multiple frames, the stage "swallows" the output images so that the application does not receive them, and finally sends through only the combined and processed image.

Default drc.json file for DRC:

{
    "hdr" :
    {
	"num_frames" : 1,
	"lp_filter_strength" : 0.2,
	"lp_filter_threshold" : [ 0, 10.0 , 2048, 205.0, 4095, 205.0 ],
	"global_tonemap_points" :
	[
	    { "q": 0.1, "width": 0.05, "target": 0.15, "max_up": 1.5, "max_down": 0.7 },
	    { "q": 0.5, "width": 0.05, "target": 0.5, "max_up": 1.5, "max_down": 0.7 },
	    { "q": 0.8, "width": 0.05, "target": 0.8, "max_up": 1.5, "max_down": 0.7 }
	],
	"global_tonemap_strength" : 1.0,
	"local_pos_strength" : [ 0, 6.0, 1024, 2.0, 4095, 2.0 ],
	"local_neg_strength" : [ 0, 4.0, 1024, 1.5, 4095, 1.5 ],
	"local_tonemap_strength" : 1.0,
	"local_colour_scale" : 0.9
    }
}

Example:

Without DRC:

Image without DRC processing

With full-strength DRC: (use libcamera-still -o test.jpg --post-process-file drc.json)

Image with DRC processing

Default hdr.json file for HDR:

{
    "hdr" :
    {
	"num_frames" : 8,
	"lp_filter_strength" : 0.2,
	"lp_filter_threshold" : [ 0, 10.0 , 2048, 205.0, 4095, 205.0 ],
	"global_tonemap_points" :
	[
	    { "q": 0.1, "width": 0.05, "target": 0.15, "max_up": 5.0, "max_down": 0.5 },
	    { "q": 0.5, "width": 0.05, "target": 0.45, "max_up": 5.0, "max_down": 0.5 },
	    { "q": 0.8, "width": 0.05, "target": 0.7, "max_up": 5.0, "max_down": 0.5 }
	],
	"global_tonemap_strength" : 1.0,
	"local_pos_strength" : [ 0, 6.0, 1024, 2.0, 4095, 2.0 ],
	"local_neg_strength" : [ 0, 4.0, 1024, 1.5, 4095, 1.5 ],
	"local_tonemap_strength" : 1.0,
	"local_colour_scale" : 0.8
    }
}

Example:

Without HDR:

Image without HDR processing

With HDR: (use libcamera-still -o test.jpg --ev -2 --denoise cdn_off --post-process-file hdr.json)

Image with DRC processing

motion_detect stage

The motion_detect stage works by analysing frames from the low resolution image stream, which must be configured for it to work. It compares a region of interest ("roi") in the frame to the corresponding part of a previous one and if enough pixels are sufficiently different, that will be taken to indicate motion. The result is added to the metadata under "motion_detect.result".

This stage has no dependencies on any 3rd party libraries.

It has the following tunable parameters. The dimensions are always given as a proportion of the low resolution image size.

roi_x

x-offset of the region of interest for the comparison

roi_y

y-offset of the region of interest for the comparison

roi_width

width of the region of interest for the comparison

roi_height

height of the region of interest for the comparison

difference_m

Linear coefficient used to construct the threshold for pixels being different

difference_c

Constant coefficient used to construct the threshold for pixels being different according to threshold = difference_m * pixel_value + difference_c

frame_period

The motion detector will run only this many frames

hskip

The pixel tests are subsampled by this amount horizontally

vksip

The pixel tests are subsampled by this amount vertically

region_threshold

The proportion of pixels (or "regions") which must be categorised as different for them to count as motion

verbose

Print messages to the console, including when the "motion"/"no motion" status changes

Default motion_detect.json configuration file:

{
    "motion_detect" :
    {
	"roi_x" : 0.1,
	"roi_y" : 0.1,
	"roi_width" : 0.8,
	"roi_height" : 0.8,
	"difference_m" : 0.1,
	"difference_c" : 10,
	"region_threshold" : 0.005,
	"frame_period" : 5,
	"hskip" : 2,
	"vskip" : 2,
	"verbose" : 0
    }
}

Note that the field difference_m and difference_c, and the value of region_threshold, can be adjusted to make the algorithm more or less sensitive to motion.

If the amount of computation needs to be reduced (perhaps you have other stages that need a larger low resolution image), the amount of computation can be reduced using the hskip and vskip parameters.

To use the motion_detect stage you might enter the following example commend:

libcamera-hello --lores-width 128 --lores-height 96 --post-process-file motion_detect.json

Post-Processing with OpenCV

Note
These stages all require OpenCV to be installed on your system. You may also need to rebuild libcamera-apps with OpenCV support - please see the instructions for building libcamera-apps for yourself.

sobel_cv stage

The sobel_cv stage has the following user-configurable parameters:

ksize

Kernel size of the Sobel filter

Default sobel_cv.json file:

{
    "sobel_cv":
    {
        "ksize": 5
    }
}

Example:

Image with Sobel filter

face_detect_cv stage

This stage uses the OpenCV Haar classifier to detect faces in an image. It returns the face locations in the metadata (under the key "face_detect.results"), and optionally draws them on the image.

The face_detect_cv stage has the following user-configurable parameters:

cascade_name

Name of the file where the Haar cascade can be found.

scaling_factor

Determines range of scales at which the image is searched for faces.

min_neighbors

Minimum number of overlapping neighbours required to count as a face.

min_size

Minimum face size.

max_size

Maximum face size.

refresh_rate

How many frames to wait before trying to re-run the face detector.

draw_features

Whether to draw face locations on the returned image.

The `face_detect_cv" stage runs only during preview and video capture; it ignores still image capture. It runs on the low resolution stream which would normally be configured to a resolution from about 320x240 to 640x480 pixels.

Default face_detect_cv.json file:

{
    "face_detect_cv":
    {
        "cascade_name" : "/usr/local/share/OpenCV/haarcascades/haarcascade_frontalface_alt.xml",
        "scaling_factor" : 1.1,
        "min_neighbors" : 2,
        "min_size" : 32,
        "max_size" : 256,
        "refresh_rate" : 1,
        "draw_features" : 1
    }
}

Example:

Image showing faces

annotate_cv stage

This stage allows text to be written into the top corner of images. It allows the same % substitutions as the --info-text parameter.

The stage does not output any metadata, but if it finds metadata under the key "annotate.text" it will write this text in place of anything in the JSON configuration file. This allows other post-processing stages to pass it text strings to be written onto the top of the images.

The annotate_cv stage has the following user-configurable parameters:

text

The text string to be written.

fg

Foreground colour.

bg

Background colour.

scale

A number proportional to the size of the text.

thickness

A number that determines the thickness of the text.

alpha

The amount of "alpha" to apply when overwriting the background pixels.

Default annotate_cv.json file:

{
    "annotate_cv" :
    {
	"text" : "Frame %frame exp %exp ag %ag dg %dg",
	"fg" : 255,
	"bg" : 0,
	"scale" : 1.0,
	"thickness" : 2,
	"alpha" : 0.3
    }
}

Example:

Image with text overlay

Post-Processing with TensorFlow Lite

Note
These stages require TensorFlow Lite (TFLite) libraries to be installed that export the C++ API. Unfortunately the TFLite libraries are not normally distributed conveniently in this form, however, one place where they can be downloaded is lindevs.com. Please follow the installation instructions given on that page. Subsequently you may need to recompile libcamera-apps with TensorFlow Lite support - please follow the instructions for building libcamera-apps for yourself.

object_classify_tf stage

object_classify_tf uses a Google MobileNet v1 model to classify objects in the camera image. It can be obtained from https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224_quant.tgz, which will need to be uncompressed. You will also need the labels.txt file which can be found in https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_1.0_224_frozen.tgz.

This stage has the following configuratble parameters.

top_n_results

How many results to show

refresh_rate

The number of frames that must elapse before the model is re-run

threshold_high

Confidence threshold (between 0 and 1) where objects are considered as being present

threshold_low

Confidence threshold which objects must drop below before being discarded as matches

model_file

Pathname to the tflite model file

labels_file

Pathname to the file containing the object labels

display_labels

Whether to display the object labels on the image. Note that this causes annotate.text metadata to be inserted so that the text can be rendered subsequently by the annotate_cv stage

verbose

Output more information to the console

Example object_classify_tf.json file:

{
    "object_classify_tf":
    {
        "top_n_results" : 2,
        "refresh_rate" : 30,
        "threshold_high" : 0.6,
        "threshold_low" : 0.4,
        "model_file" : "/home/pi/models/mobilenet_v1_1.0_224_quant.tflite",
        "labels_file" : "/home/pi/models/labels.txt",
        "display_labels" : 1
    },
    "annotate_cv" :
    {
	"text" : "",
	"fg" : 255,
	"bg" : 0,
	"scale" : 1.0,
	"thickness" : 2,
	"alpha" : 0.3
    }
}

The stage operates on a low resolution stream image of size 224x224, so it could be used as follows:

libcamera-hello --post-process-file object_classify_tf.json --lores-width 224 --lores-height 224

Image showing object classifier results

pose_estimation_tf stage

pose_estimation_tf uses a Google MobileNet v1 model posenet_mobilenet_v1_100_257x257_multi_kpt_stripped.tflite that can be found at https://github.com/Qengineering/TensorFlow_Lite_Pose_RPi_32-bits.

This stage has the following configurable parameters.

refresh_rate

The number of frames that must elapse before the model is re-run

model_file

Pathname to the tflite model file

verbose

Output more information to the console

Also provided is a separate plot_pose_cv stage which can be included in the JSON configuration file and which will draw the detected pose onto the main image. This stage has the following configuration parameters.

confidence_threshold

A confidence level determining how much is drawn. This number can be less than zero; please refer to the GitHub repository for more information.

Example pose_estimation_tf.json file:

{
    "pose_estimation_tf":
    {
        "refresh_rate" : 5,
        "model_file" : ""posenet_mobilenet_v1_100_257x257_multi_kpt_stripped.tflite",
    },
    "plot_pose_cv" :
    {
	"confidence_threshold" : -0.5
    }
}

The stage operates on a low resolution stream image of size 257x257 (but which must be rounded up to 258x258 for YUV420 images), so it could be used as follows:

libcamera-hello --post-process-file pose_estimation_tf.json --lores-width 258 --lores-height 258

Image showing pose estimation results

object_detect_tf stage

object_detect_tf uses a Google MobileNet v1 SSD (Single Shot Detector) model. The model and labels files can be downloaded from https://storage.googleapis.com/download.tensorflow.org/models/tflite/coco_ssd_mobilenet_v1_1.0_quant_2018_06_29.zip.

This stage has the following configurable parameters.

refresh_rate

The number of frames that must elapse before the model is re-run

model_file

Pathname to the tflite model file

labels_file

Pathname to the file containing the list of labels

confidence_threshold

Minimum confidence threshold because a match is accepted.

overlap_threshold

Determines the amount of overlap between matches for them to be merged as a single match.

verbose

Output more information to the console

Also provided is a separate object_detect_draw_cv stage which can be included in the JSON configuration file and which will draw the detected objects onto the main image. This stage has the following configuration parameters.

line_thickness

Thickness of the bounding box lines

font_size

Size of the font used for the label

Example object_detect_tf.json file:

{
    "object_detect_tf":
    {
	"number_of_threads" : 2,
	"refresh_rate" : 10,
	"confidence_threshold" : 0.5,
	"overlap_threshold" : 0.5,
	"model_file" : "/home/pi/models/coco_ssd_mobilenet_v1_1.0_quant_2018_06_29/detect.tflite",
	"labels_file" : "/home/pi/models/coco_ssd_mobilenet_v1_1.0_quant_2018_06_29/labelmap.txt",
	"verbose" : 1
    },
    "object_detect_draw_cv":
    {
	"line_thickness" : 2
    }
}

The stage operates on a low resolution stream image of size 300x300. The following example would pass a 300x300 crop to the detector from the centre of the 400x300 low resolution image.

libcamera-hello --post-process-file object_detect_tf.json --lores-width 400 --lores-height 300

Image showing detected objects

segmentation_tf stage

segmentation_tf uses a Google MobileNet v1 model. The model file can be downloaded from https://tfhub.dev/tensorflow/lite-model/deeplabv3/1/metadata/2?lite-format=tflite, whilst the labels file can be found in the assets folder, named segmentation_labels.txt.

This stage runs on an image of size 257x257. Because YUV420 images must have even dimensions, the low resolution image should be at least 258 pixels in both width and height. The stage adds a vector of 257x257 values to the image metadata where each value indicates which of the categories (listed in the labels file) that the pixel belongs to. Optionally, a representation of the segmentation can be drawn into the bottom right corner of the image.

This stage has the following configurable parameters.

refresh_rate

The number of frames that must elapse before the model is re-run

model_file

Pathname to the tflite model file

labels_file

Pathname to the file containing the list of labels

threshold

When verbose is set, the stage prints to the console any labels where the number of pixels with that label (in the 257x257 image) exceeds this threshold.

draw

Set this value to draw the segmentation map into the bottom right hand corner of the image.

verbose

Output more information to the console

Example segmentation_tf.json file:

{
    "segmentation_tf":
    {
	"number_of_threads" : 2,
	"refresh_rate" : 10,
	"model_file" : "/home/pi/models/lite-model_deeplabv3_1_metadata_2.tflite",
	"labels_file" : "/home/pi/models/segmentation_labels.txt",
	"draw" : 1,
	"verbose" : 1
    }
}

This example takes a square camera image and reduces it to 258x258 pixels in size. In fact the stage also works well when non-square images are squashed unequally down to 258x258 pixels without cropping. The image below shows the segmentation map in the bottom right hand corner.

libcamera-hello --post-process-file segmentation_tf.json --lores-width 258 --lores-height 258 --viewfinder-width 1024 --viewfinder-height 1024

Image showing segmentation in the bottom right corner

Writing your own Post-Processing Stages

The libcamera-apps post-processing framework is not only very flexible but is meant to make it easy for users to create their own custom post-processing stages. It is easy to include algorithms and routines that are already available both in OpenCV and TensorFlow Lite.

We are keen to accept and distribute interesting post-processing stages contributed by our users.

Basic Post-Processing Stages

Post-processing stages have a simple API, and users can create their own by deriving from the PostProcessingStage class. The member functions that must be implemented are listed below, though note that some be be unnecessary for simple stages.

char const *Name() const

Return the name of the stage. This is used to match against stages listed in the JSON post-processing configuration file.

void Read(boost::property_tree::ptree const &params)

This method will read any of the stage’s configuration parameters from the JSON file.

void AdjustConfig(std::string const &use_case, StreamConfiguration *config)

This method gives stages a chance to influence the configuration of the camera, though it is not often necessary to implement it.

void Configure()

This is called just after the camera has been configured. It is a good moment to check that the stage has access to the streams it needs, and it can also allocate any resources that it may require.

void Start()

Called when the camera starts. This method is often not required.

bool Process(CompletedRequest &completed_request)

This method presents completed camera requests for post-processing and is where the necessary pixel manipulations or image analysis will happen. The function returns true if the post-processing framework is not to deliver this request on to the application.

void Stop()

Called when the camera is stopped. Normally a stage would need to shut down any processing that might be running (for example, if it started any asynchronous threads).

void Teardown()

Called when the camera configuration is torn down. This would typically be used to de-allocate any resources that were set up in the Configure method.

Some helpful hints on writing your own stages:

  • Generally, the Process method should not take too long as it will block the imaging pipeline and may cause stuttering. When time-consuming algorithms need to be run, it may be helpful to delegate them to another asynchronous thread.

  • When delegating work to another thread, the way image buffers are handled currently means that they will need to be copied. For some applciations, such as image analysis, it may be viable to use the "low resolution" image stream rather than full resolution images.

  • The post-processing framework adds multi-threading parallelism on a per-frame basis. This is helpful in improving throughput if you want to run on every single frame. Some functions may supply parallelism within each frame (such as OpenCV and TFLite). In these cases it would probably be better to serialise the calls so as to suppress the per-frame parallelism.

  • Most streams, and in particular the low resolution stream, have YUV420 format. These formats are sometimes not ideal for OpenCV or TFLite so there may sometimes need to be a conversion step.

  • When images need to be altered, doing so in place is much the easiest strategy.

  • Implementations of any stage should always include a RegisterStage call. This registers your new stage with the system so that it will be correctly identified when listed in a JSON file. You will need to add it to the post-processing folder’s CMakeLists.txt too, of course.

The easiest example to start with is negate_stage.cpp, which "negates" an image (turning black white, and vice versa). Aside from a small amount of derived class boiler-plate, it contains barely half a dozen lines of code.

Next up in complexity is sobel_cv_stage.cpp. This implements a Sobel filter using just a few lines of OpenCV functions.

TFLite Stages

For stages wanting to analyse images using TensorFlowLite we provide the TfStage base class. This provides a certain amount of boilerplate code and makes it much easier to implement new TFLite-based stages by deriving from this class. In particular, it delegates the execution of the model to another thread, so that the full camera framerate is still maintained - it is just the model that will run at a lower framerate.

The TfStage class implements all the public PostProcessingStage methods that normally have to be redefined, with the exception of the Name method which must still be supplied. It then presents the following virtual methods which derived classes should implement instead.

void readExtras()

The base class reads the named model and certain other parameters like the refresh_rate. This method can be supplied to read any extra parameters for the derived stage. It is also a good place to check that the loaded model looks as expected (i.e. has right input and output dimensions).

void checkConfiguration()

The base class fetches the low resolution stream which TFLite will operate on, and the full resolution stream in case the derived stage needs it. This method is provided for the derived class to check that the streams it requires are present. In case any required stream is missing, it may elect simply to avoid processing any images, or it may signal a fatal error.

void interpretOutputs()

The TFLite model runs asynchronously so that it can run "every few frames" without holding up the overall framerate. This method gives the derived stage the chance to read and interpret the model’s outputs, running right after the model itself and in that same thread.

void applyResults()

Here we are running once again in the main thread and so this method should run reasonably quickly so as not to hold up the supply of frames to the application. It is provided so that the last results of the model (which might be a few frames ago) can be applied to the current frame. Typically this would involve attaching metadata to the image, or perhaps drawing something onto the main image.

For further information, readers are referred to the supplied example code implementing the ObjectClassifyTfStage and PoseEstimationTfStage classes.

libcamera and libcamera-apps Packages

A number of apt packages are provided for convenience. In order to access them, we recommend keeping your OS up to date in the usual way.

Binary Packages

There are two libcamera-apps packages available, that contain the necessary executables:

  • libcamera-apps contains the full applications with support for previews using X Windows. This package is pre-installed in the Bullseye release of Raspberry Pi OS, and can be installed in Buster using sudo apt install libcamera-apps.

  • libcamera-apps-lite omits X Windows support and only the DRM preview is available. This package is pre-installed in the Bullseye release of Raspberry Pi OS Lite, and can be installed in Buster using sudo apt install libcamera-apps-lite.

For Bullseye users, official Raspberry Pi cameras should be detected automatically. Other users will need to edit their /boot/config.txt file if they have not done so previously.

Dependencies

These applications depend on a number of library packages which are named library-name<n> where <n> is a version number (actually the ABI, or Application Binary Interface, version), and which stands at zero at the time of writing. Thus we have the following:

  • The package libcamera0 contains the libcamera libraries.

  • The package libepoxy0 contains the libepoxy libraries.

These will be installed automatically when needed.

Dev Packages

libcamera-apps can be rebuilt on their own without installing and building libcamera and libepoxy from scratch. To enable this, the following packages should be installed:

  • libcamera-dev contains the necessary libcamera header files and resources.

  • libepoxy-dev contains the necessary libepoxy header files and resources. You will only need this if you want support for the X11/GLES preview window.

Subsequently libcamera-apps can be checked out from Github and rebuilt.

Building libcamera and libcamera-apps

Building libcamera and libcamera-apps for yourself can bring the following benefits.

  • You can pick up the latest enhancements and features.

  • libcamera-apps can be compiled with extra optimisation for Pi 3 and Pi 4 devices running a 32-bit OS.

  • You can include the various optional OpenCV and/or TFLite post-processing stages (or add your own).

  • You can customise or add your own applications derived from libcamera-apps.

Note
When building for Pi 3 or earlier devices there is a risk that the device may run out of swap and fail. We recommend either increasing the amount of swap, or building with fewer threads (the -j option to ninja and to make).

Building libcamera-apps without rebuilding libcamera

You can rebuild libcamera-apps without first rebuilding the whole of libcamera and libepoxy. If you do not need support for the X11/GLES preview window then libepoxy can be omitted entirely. Mostly this will include Raspberry Pi OS Lite users, and they must be sure to use -DENABLE_X11=0 when running cmake later. These users should run:

sudo apt install -y libcamera-dev libjpeg-dev libtiff5-dev

All other users should execute:

sudo apt install -y libcamera-dev libepoxy-dev libjpeg-dev libtiff5-dev

If you want to use the Qt preview window, please also execute

sudo apt install -y qtbase5-dev libqt5core5a libqt5gui5 libqt5widgets5

Now proceed directly to the instructions for building libcamera-apps. Raspberry Pi OS Lite users should check that git is installed first (sudo apt install -y git).

Building libcamera

Rebuilding libcamera from scratch should be necessary only if you need the latest features that may not yet have reached the apt repositories, or if you need to customise its behaviour in some way.

First install all the necessary dependencies for libcamera.

Note
Raspberry Pi OS Lite users will first need to install the following additional packages if they have not done so previously:
sudo apt install -y python3-pip git
sudo pip3 install jinja2

All users should then install the following:

sudo apt install -y libboost-dev
sudo apt install -y libgnutls28-dev openssl libtiff5-dev
sudo apt install -y qtbase5-dev libqt5core5a libqt5gui5 libqt5widgets5
sudo apt install -y meson
sudo pip3 install pyyaml ply
sudo pip3 install --upgrade meson

In the meson commands below we have enabled the gstreamer plugin. If you do not need this you can set -Dgstreamer=disabled instead and the next pair of dependencies will not be required. But if you do leave gstreamer enabled, then you will need the following:

sudo apt install -y libglib2.0-dev libgstreamer-plugins-base1.0-dev

Now we can check out and build libcamera itself.

cd
git clone git://linuxtv.org/libcamera.git
cd libcamera

Next we recommend that Raspberry Pi OS Lite users run

meson build --buildtype=release -Dpipelines=raspberrypi -Dipas=raspberrypi -Dv4l2=true -Dgstreamer=enabled -Dtest=false -Dlc-compliance=disabled -Dcam=disabled -Dqcam=disabled -Ddocumentation=disabled

Users of Raspberry Pi OS can instead use

meson build --buildtype=release -Dpipelines=raspberrypi -Dipas=raspberrypi -Dv4l2=true -Dgstreamer=enabled -Dtest=false -Dlc-compliance=disabled -Dcam=disabled -Dqcam=enabled -Ddocumentation=disabled

The only difference is that the latter also builds the qcam test application, which has dependencies on Qt and X Windows (after completing the libcamera build users can run build/src/qcam/qcam to verify that libcamera is functioning correctly).

To complete the libcamera build, please run

ninja -C build   # use -j 2 on Pi 3 or earlier devices
sudo ninja -C build install
Note
At the time of writing libcamera does not yet have a stable binary interface. Therefore, if you have rebuilt libcamera we recommend continuing and rebuilding libcamera-apps from scratch too.

Building libepoxy

Rebuilding libepoxy should not normally be necessary as this library changes only very rarely. If you do want to build it from scratch, however, please follow the instructions below.

Start by installing the necessary dependencies.

sudo apt install -y libegl1-mesa-dev

Next, check out and build libepoxy.

cd
git clone https://github.com/anholt/libepoxy.git
cd libepoxy
mkdir _build
cd _build
meson
ninja
sudo ninja install

Building libcamera-apps

First fetch the necessary dependencies for libcamera-apps.

sudo apt install -y cmake libboost-program-options-dev libdrm-dev libexif-dev

The libcamera-apps build process begins with the following:

cd
git clone https://github.com/raspberrypi/libcamera-apps.git
cd libcamera-apps
mkdir build
cd build

At this point you will need to run cmake after deciding what extra flags to pass it. The valid flags are:

  • -DENABLE_COMPILE_FLAGS_FOR_TARGET=armv8-neon - you may supply this when building for Pi 3 or Pi 4 devices running a 32-bit OS. Some post-processing features may run more quickly.

  • -DENABLE_DRM=1 or -DENABLE_DRM=0 - this enables or disables the DRM/KMS preview rendering. This is what implements the preview window when X Windows is not running.

  • -DENABLE_X11=1 or -DENABLE_X11=0 - this enables or disables the X Windows based preview. You should disable this if your system does not have X Windows installed.

  • -DENABLE_QT=1 or -DENABLE_QT=0 - this enables or disables support for the Qt-based implementation of the preview window. You should disable it if you do not have X Windows installed, or if you have no intention of using the Qt-based preview window. The Qt-based preview is normally not recommended because it is computationally very expensive, however it does work with X display forwarding.

  • -DENABLE_OPENCV=1 or -DENABLE_OPENCV=0 - you may choose one of these to force OpenCV-based post-processing stages to be linked (or not). If you enable them, then OpenCV must be installed on your system. Normally they will be built by default if OpenCV is available.

  • -DENABLE_TFLITE=1 or -DENABLE_TFLITE=0 - choose one of these to enable TensorFlow Lite post-processing stages (or not). By default they will not be enabled. If you enable them then TensorFlow Lite must be available on your system. Depending on how you have built and/or installed TFLite, you may need to tweak the CMakeLists.txt file in the post_processing_stages directory.

For Raspberry Pi OS users we recommend the following cmake command:

cmake .. -DENABLE_DRM=1 -DENABLE_X11=1 -DENABLE_QT=1 -DENABLE_OPENCV=0 -DENABLE_TFLITE=0

and for Raspberry Pi OS Lite users:

cmake .. -DENABLE_DRM=1 -DENABLE_X11=0 -DENABLE_QT=0 -DENABLE_OPENCV=0 -DENABLE_TFLITE=0

In both cases, consider -DENABLE_COMPILE_FLAGS_FOR_TARGET=armv8-neon if you are using a 32-bit OS on a Pi 3 or Pi 4. Consider -DENABLE_OPENCV=1 if you have installed OpenCV and wish to use OpenCV-based post-processing stages. Finally also consider -DENABLE_TFLITE=1 if you have installed TensorFlow Lite and wish to use it in post-processing stages.

After executing the cmake command of your choice, the whole process concludes with the following:

make -j4  # use -j2 on Pi 3 or earlier devices
sudo make install
sudo ldconfig /usr/local/lib # this is only necessary on the first build
Note
If you are using an image where libcamera-apps have been previously installed as an apt package, and you want to run the new libcamera-apps executables from the same terminal window where you have just built and installed them, you may need to run hash -r to be sure to pick up the new ones over the system supplied ones.

Finally, if you have not already done so, please be sure to follow the dtoverlay and display driver instructions in the Getting Started section (and rebooting if you changed anything there).

Understanding and Writing your own Apps

libcamera-apps are not supposed to be a full set of all the applications with all the features that anyone could ever need. Instead, they are supposed to be easy to understand such that users who require specific but slightly different behaviour can implement it for themselves.

All the applications work by having a simple event loop which receives a message with a new set of frames from the camera system. This set of frames is called a CompletedRequest. It contains all the images that have been derived from that single camera frame (so perhaps a low resolution image in addition to the full size output), as well as metadata from the camera system and further metadata from the post-processing system.

libcamera-hello

libcamera-hello is much the easiest application to understand. The only thing it does with the camera images is extract the CompletedRequestPtr (a shared pointer to the CompletedRequest) from the message:

	CompletedRequestPtr &completed_request = std::get<CompletedRequestPtr>(msg.payload);

and forward it to the preview window:

	app.ShowPreview(completed_request, app.ViewfinderStream());

One important thing to note is that every CompletedRequest must be recycled back to the camera system so that the buffers can be reused, otherwise it will simply run out of buffers in which to receive new camera frames. This recycling process happens automatically when all references to the CompletedRequest are dropped, using C++'s shared pointer and custom deleter mechanisms.

In libcamera-hello therefore, two things must happen for the CompletedRequest to be returned to the camera.

  1. The event loop must go round again so that the message (msg in the code), which is holding a reference to the shared pointer, is dropped.

  2. The preview thread, which takes another reference to the CompletedRequest when ShowPreview is called, must be called again with a new CompletedRequest, causing the previous one to be dropped.

libcamera-vid

libcamera-vid is not unlike libcamera-hello, but it adds a codec to the event loop and the preview. Before the event loop starts, we must configure that encoder with a callback which says what happens to the buffer containing the encoded image data.

	app.SetEncodeOutputReadyCallback(std::bind(&Output::OutputReady, output.get(), _1, _2, _3, _4));

Here we send the buffer to the Output object which may write it to a file, or send it over the network, according to our choice when we started the application.

The encoder also takes a new reference to the CompletedRequest, so once the event loop, the preview window and the encoder all drop their references, the CompletedRequest will be recycled automatically back to the camera system.

libcamera-raw

libcamera-raw is not so very different from libcamera-vid. It too uses a encoder, though this time it is a "dummy" encoder called the NullEncoder. This just treats the input image directly as the output buffer and is careful not to drop its reference to the input until the output callback has dealt with it first.

This time, however, we do not forward anything to the preview window, though we could have displayed the (processed) video stream if we had wanted.

The use of the NullEncoder is possibly overkill in this application as we could probably just send the image straight to the Output object. However, it serves to underline the general principle that it is normally a bad idea to do too much work directly in the event loop, and time-consuming processes are often better left to other threads.

libcamera-jpeg

We discuss libcamera-jpeg rather than libcamera-still as the basic idea (that of switching the camera from preview into capture mode) is the same, and libcamera-jpeg has far fewer additional options (such as timelapse capture) that serve to distract from the basic function.

libcamera-jpeg starts the camera in preview mode in the usual way, but at the appropriate moment stops it and switches to still capture:

	app.StopCamera();
	app.Teardown();
	app.ConfigureStill();
	app.StartCamera();

Then the event loop will grab the first frame that emerges once it’s no longer in preview mode, and saves this as a JPEG.

Python Bindings for libcamera

Python bindings for libcamera are currently in development.

Full working examples will be provided here.

Camera Tuning and supporting 3rd Party Sensors

The Camera Tuning File

Most of the image procesing applied to frames from the sensor is done by the hardware ISP (Image Signal Procesor). This processing is governed by a set of control algorithms and these in turn must have a wide range of parameters supplied to them. These parameters are tuned specifically for each sensor and are collected together in a JSON file known as the camera tuning file.

This tuning file can be inspected and edited by users. Using the --tuning-file command line option, users can point the system at completely custom camera tuning files.

3rd Party Sensors

libcamera makes it possible to support 3rd party sensors (that is, sensors other than Raspberry Pi’s officially supported sensors) on the Raspberry Pi platform. To accomplish this, a working open source sensor driver must be provided, which the authors are happy to submit to the Linux kernel. There are a couple of extra files need to be added to libcamera which supply device-specific information that is available from the kernel drivers, including the previously discussed camera tuning file.

Raspberry Pi also supplies a tuning tool which automates the generation of the tuning file from a few simple calibration images.

Both these topics are rather beyond the scope of the documentation here, however, full information is available in the Tuning Guide for the Raspberry Pi cameras and libcamera.

Known Issues

We are aware of the following issues in libcamera and libcamera-apps.

  • Mode selection. libcamera attempts to guess the camera mode that you want from the output resolution specified, unfortunately this is far from foolproof. It can jump to cropped camera modes when you really wanted the full FoV, and it gives no way of requesting camera modes that frame faster (perhaps by sending raw images with a lower bitdepth). We remain in discussion with the libcamera team on this, and are working on finding a solution.

  • libcamera has no support for colour spaces. This means that all images, including video streams, are in the default JPEG ("full range BT601") colour space. Again, we are working with the libcamera team to find a solution for this.

  • On Pi 3s (and earlier devices) the graphics hardware can only support images up to 2048x2048 pixels which places a limit on the camera images that can be resized into the preview window. In practice this means that video encoding of images larger than 2048 pixels across (which would necessarily be using a codec other than h.264) will not support, or will produce corrupted, preview images. For Pi 4s the limit is 4096 pixels. We would recommend using the -n (no preview) option for the time being.

  • The preview window shows some display tearing when using X windows. This is not likely to be fixable.

Getting Help

For further help with libcamera and the libcamera-apps, the first port of call will usually be the Rasperry Pi Camera Forum. Before posting, it’s helpful to:

  • Ensure your software is up to date.

  • If you are using Buster or an earlier version of Raspberry Pi OS where you have had to build libcamera and libcamera-apps for yourself, or indeed if you are using a later version but have built them for yourself anyway, then you may need to update those git repositories and repeat the build process.

  • Make a note of your operating system version (uname -a).

  • Make a note of your libcamera and libcamera-apps versions (libcamera-hello --version).

  • Please report the make and model of the camera module you are using. Note that when third party camera module vendors supply their own software then we are normally unable to offer any support and all queries should be directed back to the vendor.

  • Please also provide information on what kind of a Raspberry Pi you have, including memory size.

  • If it seems like it might be relevant, please include any excerpts from the application’s console output.

When it seems likely that there are specific problems in the camera software (such as crashes) then it may be more appropriate to create an issue in the libcamera-apps Github repository. Again, please include all the helpful details that you can.

Raspicam commands

raspistill, raspivid and raspiyuv are command line tools for using the camera module.

Enabling the Camera

Before using any of the Raspicam applications, the camera must be enabled.

On the desktop

Select Preferences and Raspberry Pi Configuration from the desktop menu: a window will appear. Select the Interfaces tab, then click on the enable camera option. Click OK. You will need to reboot for the changes to take effect.

With the command line

Open the raspi-config tool from the terminal:

sudo raspi-config

Select Interfacing Options then Camera and press Enter. Choose Yes then Ok. Go to Finish and you’ll be prompted to reboot.

To test that the system is installed and working, try the following command:

raspistill -v -o test.jpg

The display should show a five-second preview from the camera and then take a picture, saved to the file test.jpg, whilst displaying various informational messages.

raspistill

raspistill is the command line tool for capturing still photographs with a Raspberry Pi camera module.

Basic usage of raspistill

With a camera module connected and enabled, enter the following command in the terminal to take a picture:

raspistill -o cam.jpg
Upside-down photo

In this example the camera has been positioned upside-down. If the camera is placed in this position, the image must be flipped to appear the right way up.

Vertical flip and horizontal flip

With the camera placed upside-down, the image must be rotated 180° to be displayed correctly. The way to correct for this is to apply both a vertical and a horizontal flip by passing in the -vf and -hf flags:

raspistill -vf -hf -o cam2.jpg
Vertical and horizontal flipped photo

Now the photo has been captured correctly.

Resolution

The camera module takes pictures at a resolution of 2592 x 1944 which is 5,038,848 pixels or 5 megapixels.

File size

A photo taken with the camera module will be around 2.4MB. This is about 425 photos per GB.

Taking 1 photo per minute would take up 1GB in about 7 hours. This is a rate of about 144MB per hour or 3.3GB per day.

Bash script

You can create a Bash script which takes a picture with the camera. To create a script, open up your editor of choice and write the following example code:

#!/bin/bash

DATE=$(date +"%Y-%m-%d_%H%M")

raspistill -vf -hf -o /home/pi/camera/$DATE.jpg

This script will take a picture and name the file with a timestamp.

You’ll also need to make sure the path exists by creating the camera folder:

mkdir camera

Say we saved it as camera.sh, we would first make the file executable:

chmod +x camera.sh

Then run with:

./camera.sh

More options

For a full list of possible options, run raspistill with no arguments. To scroll, redirect stderr to stdout and pipe the output to less:

raspistill 2>&1 | less

Use the arrow keys to scroll and type q to exit.

raspivid

raspivid is the command line tool for capturing video with a Raspberry Pi camera module.

Basic usage of raspivid

With a camera module connected and enabled, record a video using the following command:

raspivid -o vid.h264

Remember to use -hf and -vf to flip the image if required, like with raspistill

This will save a 5 second video file to the path given here as vid.h264 (default length of time).

Specify length of video

To specify the length of the video taken, pass in the -t flag with a number of milliseconds. For example:

raspivid -o video.h264 -t 10000

This will record 10 seconds of video.

More options

For a full list of possible options, run raspivid with no arguments, or pipe this command through less and scroll through:

raspivid 2>&1 | less

Use the arrow keys to scroll and type q to exit.

MP4 Video Format

The Pi captures video as a raw H264 video stream. Many media players will refuse to play it, or play it at an incorrect speed, unless it is "wrapped" in a suitable container format like MP4. The easiest way to obtain an MP4 file from the raspivid command is using MP4Box.

Install MP4Box with this command:

sudo apt install -y gpac

Capture your raw video with raspivid and wrap it in an MP4 container like this:

# Capture 30 seconds of raw video at 640x480 and 150kB/s bit rate into a pivideo.h264 file:
raspivid -t 30000 -w 640 -h 480 -fps 25 -b 1200000 -p 0,0,640,480 -o pivideo.h264
# Wrap the raw video with an MP4 container:
MP4Box -add pivideo.h264 pivideo.mp4
# Remove the source raw file, leaving the remaining pivideo.mp4 file to play
rm pivideo.h264

Alternatively, wrap MP4 around your existing raspivid output, like this:

MP4Box -add video.h264 video.mp4

raspiyuv

raspiyuv has the same set of features as raspistill but instead of outputting standard image files such as .jpgs, it generates YUV420 or RGB888 image files from the output of the camera ISP.

In most cases using raspistill is the best option for standard image capture, but using YUV can be of benefit in certain circumstances. For example if you just need a uncompressed black and white image for computer vision applications, you can simply use the Y channel of a YUV capture.

There are some specific points about the YUV420 files that are required in order to use them correctly. Line stride (or pitch) is a multiple of 32, and each plane of YUV is a multiple of 16 in height. This can mean there may be extra pixels at the end of lines, or gaps between planes, depending on the resolution of the captured image. These gaps are unused.

Troubleshooting

If the Camera Module isn’t working correctly, there are number of things to try:

  • Is the ribbon cable attached to the Camera Serial Interface (CSI), not the Display Serial Interface (DSI)? The ribbon connector will fit into either port. The Camera port is located near the HDMI connector.

  • Are the ribbon connectors all firmly seated, and are they the right way round? They must be straight in their sockets.

  • Is the Camera Module connector, between the smaller black Camera Module itself and the PCB, firmly attached? Sometimes this connection can come loose during transit or when putting the Camera Module in a case. Using a fingernail, flip up the connector on the PCB, then reconnect it with gentle pressure. It engages with a very slight click. Don’t force it; if it doesn’t engage, it’s probably slightly misaligned.

  • Have sudo apt update and sudo apt full-upgrade been run?

  • Has raspi-config been run and the Camera Module enabled?

  • Is your power supply sufficient? The Camera Module adds about 200-250mA to the power requirements of your Raspberry Pi.

If things are still not working, try the following:

  • Error : raspistill/raspivid command not found. This probably means your update/upgrade failed in some way. Try it again.

  • Error : ENOMEM. The Camera Module is not starting up. Check all connections again.

  • Error : ENOSPC. The Camera Module is probably running out of GPU memory. Check config.txt in the /boot/ folder. The gpu_mem option should be at least 128. Alternatively, use the Memory Split option in the Advanced section of raspi-config to set this.

  • If you’ve checked all the above issues and the Camera Module is still not working, try posting on our forums for more help.

Command Line Options

Preview window

	--preview,	-p		Preview window settings <'x,y,w,h'>

Allows the user to define the size of the preview window and its location on the screen. Note this will be superimposed over the top of any other windows/graphics.

	--fullscreen,	-f		Fullscreen preview mode

Forces the preview window to use the whole screen. Note that the aspect ratio of the incoming image will be retained, so there may be bars on some edges.

	--nopreview,	-n		Do not display a preview window

Disables the preview window completely. Note that even though the preview is disabled, the camera will still be producing frames, so will be using power.

	--opacity,	-op		Set preview window opacity

Sets the opacity of the preview windows. 0 = invisible, 255 = fully opaque.

Camera control options

	--sharpness,	-sh		Set image sharpness (-100 - 100)

Sets the sharpness of the image. 0 is the default.

	--contrast,	-co		Set image contrast (-100 - 100)

Sets the contrast of the image. 0 is the default.

	--brightness,	-br		Set image brightness (0 - 100)

Sets the brightness of the image. 50 is the default. 0 is black, 100 is white.

	--saturation,	-sa		Set image saturation (-100 - 100)

Sets the colour saturation of the image. 0 is the default.

	--ISO,	-ISO		Set capture ISO (100 - 800)

Sets the ISO to be used for captures.

	--vstab,	-vs		Turn on video stabilisation

In video mode only, turns on video stabilisation.

	--ev,	-ev		Set EV compensation (-10 - 10)

Sets the EV compensation of the image. Default is 0.

	--exposure,	-ex		Set exposure mode

Possible options are:

  • auto: use automatic exposure mode

  • night: select setting for night shooting

  • nightpreview:

  • backlight: select setting for backlit subject

  • spotlight:

  • sports: select setting for sports (fast shutter etc.)

  • snow: select setting optimised for snowy scenery

  • beach: select setting optimised for beach

  • verylong: select setting for long exposures

  • fixedfps: constrain fps to a fixed value

  • antishake: antishake mode

  • fireworks: select setting optimised for fireworks

Note that not all of these settings may be implemented, depending on camera tuning.

	--flicker, -fli		Set flicker avoidance mode

Set a mode to compensate for lights flickering at the mains frequency, which can be seen as a dark horizontal band across an image. Flicker avoidance locks the exposure time to a multiple of the mains flicker frequency (8.33ms for 60Hz, or 10ms for 50Hz). This means that images can be noisier as the control algorithm has to increase the gain instead of exposure time should it wish for an intermediate exposure value. auto can be confused by external factors, therefore it is preferable to leave this setting off unless actually required.

Possible options are:

  • off: turn off flicker avoidance

  • auto: automatically detect mains frequency

  • 50hz: set avoidance at 50Hz

  • 60hz: set avoidance at 60Hz

	--awb,	-awb		Set Automatic White Balance (AWB) mode

Modes for which colour temperature ranges (K) are available have these settings in brackets.

  • off: turn off white balance calculation

  • auto: automatic mode (default)

  • sun: sunny mode (between 5000K and 6500K)

  • cloud: cloudy mode (between 6500K and 12000K)

  • shade: shade mode

  • tungsten: tungsten lighting mode (between 2500K and 3500K)

  • fluorescent: fluorescent lighting mode (between 2500K and 4500K)

  • incandescent: incandescent lighting mode

  • flash: flash mode

  • horizon: horizon mode

  • greyworld: Use this on the NoIR camera to fix incorrect AWB results due to the lack of the IR filter.

Note that not all of these settings may be implemented, depending on camera type.

	--imxfx,	-ifx		Set image effect

Set an effect to be applied to the image:

  • none: no effect (default)

  • negative: invert the image colours

  • solarise: solarise the image

  • posterise: posterise the image

  • whiteboard: whiteboard effect

  • blackboard: blackboard effect

  • sketch: sketch effect

  • denoise: denoise the image

  • emboss: emboss the image

  • oilpaint: oil paint effect

  • hatch: hatch sketch effect

  • gpen: graphite sketch effect

  • pastel: pastel effect

  • watercolour: watercolour effect

  • film: film grain effect

  • blur: blur the image

  • saturation: colour saturate the image

  • colourswap: not fully implemented

  • washedout: not fully implemented

  • colourpoint: not fully implemented

  • colourbalance: not fully implemented

  • cartoon: not fully implemented

Note that not all of these settings may be available in all circumstances.

	--colfx,	-cfx		Set colour effect <U:V>

The supplied U and V parameters (range 0 - 255) are applied to the U and Y channels of the image. For example, --colfx 128:128 should result in a monochrome image.

	--metering,	-mm		Set metering mode

Specify the metering mode used for the preview and capture:

  • average: average the whole frame for metering

  • spot: spot metering

  • backlit: assume a backlit image

  • matrix: matrix metering

	--rotation,	-rot		Set image rotation (0 - 359)

Sets the rotation of the image in the viewfinder and resulting image. This can take any value from 0 upwards, but due to hardware constraints only 0, 90, 180, and 270 degree rotations are supported.

	--hflip,	-hf		Set horizontal flip

Flips the preview and saved image horizontally.

	--vflip,	-vf		Set vertical flip

Flips the preview and saved image vertically.

	--roi,	-roi		Set sensor region of interest

Allows the specification of the area of the sensor to be used as the source for the preview and capture. This is defined as x,y for the top-left corner, and a width and height, with all values in normalised coordinates (0.0 - 1.0). So, to set a ROI at halfway across and down the sensor, and a width and height of a quarter of the sensor, use:

-roi 0.5,0.5,0.25,0.25
	--shutter,	-ss		Set shutter speed/time

Sets the shutter open time to the specified value (in microseconds). Shutter speed limits are as follows:

Camera Version Max (microseconds)

V1 (OV5647)

6000000 (i.e. 6s)

V2 (IMX219)

10000000 (i.e. 10s)

HQ (IMX477)

200000000 (i.e. 200s)

Using values above these maximums will result in undefined behaviour.

	--drc,	-drc		Enable/disable dynamic range compression

DRC changes the images by increasing the range of dark areas, and decreasing the brighter areas. This can improve the image in low light areas.

  • off

  • low

  • med

  • high

By default, DRC is off.

	--stats,	-st		Use stills capture frame for image statistics

Force recomputation of statistics on stills capture pass. Digital gain and AWB are recomputed based on the actual capture frame statistics, rather than the preceding preview frame.

	--awbgains,	-awbg

Sets blue and red gains (as floating point numbers) to be applied when -awb off is set e.g. -awbg 1.5,1.2

	--analoggain,	-ag

Sets the analog gain value directly on the sensor (floating point value from 1.0 to 8.0 for the OV5647 sensor on Camera Module V1, and 1.0 to 12.0 for the IMX219 sensor on Camera Module V2 and the IMX447 on the HQ Camera).

	--digitalgain,	-dg

Sets the digital gain value applied by the ISP (floating point value from 1.0 to 64.0, but values over about 4.0 will produce overexposed images)

	--mode,	-md

Sets a specified sensor mode, disabling the automatic selection. Possible values depend on the version of the Camera Module being used:

Version 1.x (OV5647)

Mode Size Aspect Ratio Frame rates FOV Binning

0

automatic selection

1

1920x1080

16:9

1-30fps

Partial

None

2

2592x1944

4:3

1-15fps

Full

None

3

2592x1944

4:3

0.1666-1fps

Full

None

4

1296x972

4:3

1-42fps

Full

2x2

5

1296x730

16:9

1-49fps

Full

2x2

6

640x480

4:3

42.1-60fps

Full

2x2 plus skip

7

640x480

4:3

60.1-90fps

Full

2x2 plus skip

Version 2.x (IMX219)

Mode Size Aspect Ratio Frame rates FOV Binning

0

automatic selection

1

1920x1080

16:9

0.1-30fps

Partial

None

2

3280x2464

4:3

0.1-15fps

Full

None

3

3280x2464

4:3

0.1-15fps

Full

None

4

1640x1232

4:3

0.1-40fps

Full

2x2

5

1640x922

16:9

0.1-40fps

Full

2x2

6

1280x720

16:9

40-90fps

Partial

2x2

7

640x480

4:3

40-200fps1

Partial

2x2

1For frame rates over 120fps, it is necessary to turn off automatic exposure and gain control using -ex off. Doing so should achieve the higher frame rates, but exposure time and gains will need to be set to fixed values supplied by the user.

HQ Camera

Mode Size Aspect Ratio Frame rates FOV Binning/Scaling

0

automatic selection

1

2028x1080

169:90

0.1-50fps

Partial

2x2 binned

2

2028x1520

4:3

0.1-50fps

Full

2x2 binned

3

4056x3040

4:3

0.005-10fps

Full

None

4

1332x990

74:55

50.1-120fps

Partial

2x2 binned

	--camselect,	-cs

Selects which camera to use on a multi-camera system. Use 0 or 1.

	--annotate,	-a		Enable/set annotate flags or text

Adds some text and/or metadata to the picture.

Metadata is indicated using a bitmask notation, so add them together to show multiple parameters. For example, 12 will show time(4) and date(8), since 4+8=12.

Text may include date/time placeholders by using the '%' character, as used by strftime.

Value Meaning Example Output

-a 4

Time

20:09:33

-a 8

Date

10/28/15

-a 12

4+8=12 Show the date(4) and time(8)

20:09:33 10/28/15

-a 16

Shutter Settings

-a 32

CAF Settings

-a 64

Gain Settings

-a 128

Lens Settings

-a 256

Motion Settings

-a 512

Frame Number

-a 1024

Black Background

-a "ABC %Y-%m-%d %X"

Show some text

ABC %Y-%m-%d %X

-a 4 -a "ABC %Y-%m-%d %X"

Show custom formatted date/time

ABC 2015-10-28 20:09:33

-a 8 -a "ABC %Y-%m-%d %X"

Show custom formatted date/time

ABC 2015-10-28 20:09:33

	--annotateex,	-ae		Set extra annotation parameters

Specifies annotation size, text colour, and background colour. Colours are in hex YUV format.

Size ranges from 6 - 160; default is 32. Asking for an invalid size should give you the default.

Example Explanation

-ae 32,0xff,0x808000 -a "Annotation text"

gives size 32 white text on black background

-ae 10,0x00,0x8080FF -a "Annotation text"

gives size 10 black text on white background

	--stereo,	-3d

Select the specified stereo imaging mode; sbs selects side-by-side mode, tb selects top/bottom mode; off turns off stereo mode (the default).

	--decimate,	-dec

Halves the width and height of the stereo image.

	--3dswap,	-3dswap

Swaps the camera order used in stereoscopic imaging; NOTE: currently not working.

	--settings,	-set

Retrieves the current camera settings and writes them to stdout.

Application-specific Settings

raspistill

	--width,	-w		Set image width <size>

	--height,	-h		Set image height <size>

	--quality,	-q		Set JPEG quality <0 to 100>

Quality 100 is almost completely uncompressed. 75 is a good all-round value.

	--raw,	-r		Add raw Bayer data to JPEG metadata

This option inserts the raw Bayer data from the camera into the JPEG metadata.

	--output,	-o		Output filename <filename>

Specifies the output filename. If not specified, no file is saved. If the filename is '-', then all output is sent to stdout.

	--latest,	-l		Link latest frame to filename <filename>

Makes a file system link under this name to the latest frame.

	--verbose,	-v		Output verbose information during run

Outputs debugging/information messages during the program run.

	--timeout,	-t		Time before the camera takes picture and shuts down

The program will run for the specified length of time, entered in milliseconds. It then takes the capture and saves it if an output is specified. If a timeout value is not specified, then it is set to 5 seconds (-t 5000). Note that low values (less than 500ms, although it can depend on other settings) may not give enough time for the camera to start up and provide enough frames for the automatic algorithms like AWB and AGC to provide accurate results.

If set to 0, the preview will run indefinitely, until stopped with CTRL-C. In this case no capture is made.

	--timelapse,	-tl		time-lapse mode

The specific value is the time between shots in milliseconds. Note that you should specify %04d at the point in the filename where you want a frame count number to appear. So, for example, the code below will produce a capture every 2 seconds, over a total period of 30s, named image0001.jpg, image0002.jpg and so on, through to image0015.jpg.

-t 30000 -tl 2000 -o image%04d.jpg

Note that the %04d indicates a 4-digit number, with leading zeroes added to make the required number of digits. So, for example, %08d would result in an 8-digit number.

If a time-lapse value of 0 is entered, the application will take pictures as fast as possible. Note that there’s an minimum enforced pause of 30ms between captures to ensure that exposure calculations can be made.

	--framestart,	-fs

Specifies the first frame number in the timelapse. Useful if you have already saved a number of frames, and want to start again at the next frame.

	--datetime,	-dt

Instead of a simple frame number, the timelapse file names will use a date/time value of the format aabbccddee, where aa is the month, bb is the day of the month, cc is the hour, dd is the minute, and ee is the second.

	--timestamp,	-ts

Instead of a simple frame number, the timelapse file names will use a single number which is the Unix timestamp, i.e. the seconds since 1970.

	--thumb,	-th		Set thumbnail parameters (x:y:quality)

Allows specification of the thumbnail image inserted into the JPEG file. If not specified, defaults are a size of 64x48 at quality 35.

if --thumb none is specified, no thumbnail information will be placed in the file. This reduces the file size slightly.

	--demo,	-d		Run a demo mode <milliseconds>

This options cycles through the range of camera options. No capture is taken, and the demo will end at the end of the timeout period, irrespective of whether all the options have been cycled. The time between cycles should be specified as a millisecond value.

	--encoding,	-e		Encoding to use for output file

Valid options are jpg, bmp, gif, and png. Note that unaccelerated image types (GIF, PNG, BMP) will take much longer to save than jpg, which is hardware accelerated. Also note that the filename suffix is completely ignored when deciding the encoding of a file.

	--restart,	-rs

Sets the JPEG restart marker interval to a specific value. Can be useful for lossy transport streams because it allows a broken JPEG file to still be partially displayed.

	--exif,	-x		EXIF tag to apply to captures (format as 'key=value')

Allows the insertion of specific EXIF tags into the JPEG image. You can have up to 32 EXIF tag entries. This is useful for tasks like adding GPS metadata. For example, to set the longitude:

--exif GPS.GPSLongitude=5/1,10/1,15/1

would set the longitude to 5 degs, 10 minutes, 15 seconds. See EXIF documentation for more details on the range of tags available; the supported tags are as follows:

IFD0.<   or
IFD1.<
ImageWidth, ImageLength, BitsPerSample, Compression, PhotometricInterpretation, ImageDescription, Make, Model, StripOffsets, Orientation, SamplesPerPixel, RowsPerString, StripByteCounts, XResolution, YResolution, PlanarConfiguration, ResolutionUnit, TransferFunction, Software, DateTime, Artist, WhitePoint, PrimaryChromaticities, JPEGInterchangeFormat, JPEGInterchangeFormatLength, YCbCrCoefficients, YCbCrSubSampling, YCbCrPositioning, ReferenceBlackWhite, Copyright>

EXIF.<
ExposureTime, FNumber, ExposureProgram, SpectralSensitivity, ISOSpeedRatings, OECF, ExifVersion, DateTimeOriginal, DateTimeDigitized, ComponentsConfiguration, CompressedBitsPerPixel, ShutterSpeedValue, ApertureValue, BrightnessValue, ExposureBiasValue, MaxApertureValue, SubjectDistance, MeteringMode, LightSource, Flash, FocalLength, SubjectArea, MakerNote, UserComment, SubSecTime, SubSecTimeOriginal, SubSecTimeDigitized, FlashpixVersion, ColorSpace, PixelXDimension, PixelYDimension, RelatedSoundFile, FlashEnergy, SpatialFrequencyResponse, FocalPlaneXResolution, FocalPlaneYResolution, FocalPlaneResolutionUnit, SubjectLocation, ExposureIndex, SensingMethod, FileSource, SceneType, CFAPattern, CustomRendered, ExposureMode, WhiteBalance, DigitalZoomRatio, FocalLengthIn35mmFilm, SceneCaptureType, GainControl, Contrast, Saturation, Sharpness, DeviceSettingDescription, SubjectDistanceRange, ImageUniqueID>

GPS.<
GPSVersionID, GPSLatitudeRef, GPSLatitude, GPSLongitudeRef, GPSLongitude, GPSAltitudeRef, GPSAltitude, GPSTimeStamp, GPSSatellites, GPSStatus, GPSMeasureMode, GPSDOP, GPSSpeedRef, GPSSpeed, GPSTrackRef, GPSTrack, GPSImgDirectionRef, GPSImgDirection, GPSMapDatum, GPSDestLatitudeRef, GPSDestLatitude, GPSDestLongitudeRef, GPSDestLongitude, GPSDestBearingRef, GPSDestBearing, GPSDestDistanceRef, GPSDestDistance, GPSProcessingMethod, GPSAreaInformation, GPSDateStamp, GPSDifferential>

EINT.<
InteroperabilityIndex, InteroperabilityVersion, RelatedImageFileFormat, RelatedImageWidth, RelatedImageLength>

Note that a small subset of these tags will be set automatically by the camera system, but will be overridden by any EXIF options on the command line.

Setting --exif none will prevent any EXIF information being stored in the file. This reduces the file size slightly.

	--gpsdexif,	-gps

Applies real-time EXIF information from any attached GPS dongle (using GSPD) to the image; requires libgps.so to be installed.

	--fullpreview,	-fp		Full preview mode

This runs the preview window using the full resolution capture mode. Maximum frames per second in this mode is 15fps, and the preview will have the same field of view as the capture. Captures should happen more quickly, as no mode change should be required. This feature is currently under development.

	--keypress,	-k		Keypress mode

The camera is run for the requested time (-t), and a capture can be initiated throughout that time by pressing the Enter key. Pressing X then Enter will exit the application before the timeout is reached. If the timeout is set to 0, the camera will run indefinitely until the user presses X then Enter. Using the verbose option (-v) will display a prompt asking for user input, otherwise no prompt is displayed.

	--signal,	-s		Signal mode

The camera is run for the requested time (-t), and a capture can be initiated throughout that time by sending a USR1 signal to the camera process. This can be done using the kill command. You can find the camera process ID using the pgrep raspistill command.

kill -USR1 <process id of raspistill>

	--burst,	-bm

Sets burst capture mode. This prevents the camera from returning to preview mode in between captures, meaning that captures can be taken closer together.

raspivid

	--width,	-w		Set image width <size>

Width of resulting video. This should be between 64 and 1920.

	--height,	-h		Set image height <size>

Height of resulting video. This should be between 64 and 1080.

	--bitrate,	-b		Set bitrate

Use bits per second, so 10Mbits/s would be -b 10000000. For H264, 1080p30 a high quality bitrate would be 15Mbits/s or more. Maximum bitrate is 25Mbits/s (-b 25000000), but much over 17Mbits/s won’t show noticeable improvement at 1080p30.

	--output,	-o		Output filename <filename>

Specify the output filename. If not specified, no file is saved. If the filename is '-', then all output is sent to stdout.

To connect to a remote IPv4 host, use tcp or udp followed by the required IP Address. e.g. tcp://192.168.1.2:1234 or udp://192.168.1.2:1234.

To listen on a TCP port (IPv4) and wait for an incoming connection use --listen (-l) option, e.g. raspivid -l -o tcp://0.0.0.0:3333 will bind to all network interfaces, raspivid -l -o tcp://192.168.1.1:3333 will bind to a local IPv4.

	--listen,	-l

When using a network connection as the data sink, this option will make the system wait for a connection from the remote system before sending data.

	--verbose,	-v		Output verbose information during run

Outputs debugging/information messages during the program run.

	--timeout,	-t		Time before the camera takes picture and shuts down

The total length of time that the program will run for. If not specified, the default is 5000ms (5 seconds). If set to 0, the application will run indefinitely until stopped with Ctrl-C.

	--demo,	-d		Run a demo mode <milliseconds>

This options cycles through the range of camera options. No recording is done, and the demo will end at the end of the timeout period, irrespective of whether all the options have been cycled. The time between cycles should be specified as a millisecond value.

	--framerate,	-fps		Specify the frames per second to record

At present, the minimum frame rate allowed is 2fps, and the maximum is 30fps. This is likely to change in the future.

	--penc,	-e		Display preview image after encoding

Switch on an option to display the preview after compression. This will show any compression artefacts in the preview window. In normal operation, the preview will show the camera output prior to being compressed. This option is not guaranteed to work in future releases.

	--intra,	-g		Specify the intra refresh period (key frame rate/GoP)

Sets the intra refresh period (GoP) rate for the recorded video. H264 video uses a complete frame (I-frame) every intra refresh period, from which subsequent frames are based. This option specifies the number of frames between each I-frame. Larger numbers here will reduce the size of the resulting video, and smaller numbers make the stream less error-prone.

	--qp,	-qp		Set quantisation parameter

Sets the initial quantisation parameter for the stream. Varies from approximately 10 to 40, and will greatly affect the quality of the recording. Higher values reduce quality and decrease file size. Combine this setting with a bitrate of 0 to set a completely variable bitrate.

	--profile,	-pf		Specify H264 profile to use for encoding

Sets the H264 profile to be used for the encoding. Options are:

  • baseline

  • main

  • high

	--level,	-lev

Specifies the H264 encoder level to use for encoding. Options are 4, 4.1, and 4.2.

	--irefresh,	-if

Sets the H264 intra-refresh type. Possible options are cyclic, adaptive, both, and cyclicrows.

	--inline,	-ih		Insert PPS, SPS headers

Forces the stream to include PPS and SPS headers on every I-frame. Needed for certain streaming cases e.g. Apple HLS. These headers are small, so don’t greatly increase the file size.

	--spstimings,	-stm

Insert timing information into the SPS block.

	--timed,	-td		Do timed switches between capture and pause

This options allows the video capture to be paused and restarted at particular time intervals. Two values are required: the on time and the off time. On time is the amount of time the video is captured, and off time is the amount it is paused. The total time of the recording is defined by the timeout option. Note that the recording may take slightly over the timeout setting depending on the on and off times.

For example:

raspivid -o test.h264 -t 25000 -timed 2500,5000

will record for a period of 25 seconds. The recording will be over a timeframe consisting of 2500ms (2.5s) segments with 5000ms (5s) gaps, repeating over the 20s. So the entire recording will actually be only 10s long, since 4 segments of 2.5s = 10s separated by 5s gaps. So:

2.5 record — 5 pause - 2.5 record — 5 pause - 2.5 record — 5 pause — 2.5 record

gives a total recording period of 25s, but only 10s of actual recorded footage.

	--keypress,	-k		Toggle between record and pause on Enter keypress

On each press of the Enter key, the recording will be paused or restarted. Pressing X then Enter will stop recording and close the application. Note that the timeout value will be used to signal the end of recording, but is only checked after each Enter keypress; so if the system is waiting for a keypress, even if the timeout has expired, it will still wait for the keypress before exiting.

	--signal,	-s		Toggle between record and pause according to SIGUSR1

Sending a USR1 signal to the raspivid process will toggle between recording and paused. This can be done using the kill command, as below. You can find the raspivid process ID using pgrep raspivid.

kill -USR1 <process id of raspivid>

Note that the timeout value will be used to indicate the end of recording, but is only checked after each receipt of the SIGUSR1 signal; so if the system is waiting for a signal, even if the timeout has expired, it will still wait for the signal before exiting.

	--split,	-sp

When in a signal or keypress mode, each time recording is restarted, a new file is created.

	--circular,	-c

Select circular buffer mode. All encoded data is stored in a circular buffer until a trigger is activated, then the buffer is saved.

	--vectors,	-x

Turns on output of motion vectors from the H264 encoder to the specified file name.

	--flush,	-fl

Forces a flush of output data buffers as soon as video data is written. This bypasses any OS caching of written data, and can decrease latency.

	--save-pts,	-pts

Saves timestamp information to the specified file. Useful as an input file to mkvmerge.

	--codec,	-cd

Specifies the encoder codec to use. Options are H264 and MJPEG. H264 can encode up to 1080p, whereas MJPEG can encode up to the sensor size, but at decreased framerates due to the higher processing and storage requirements.

	--initial,	-i		Define initial state on startup

Define whether the camera will start paused or will immediately start recording. Options are record or pause. Note that if you are using a simple timeout, and initial is set to pause, no output will be recorded.

	--segment,	-sg		Segment the stream into multiple files

Rather than creating a single file, the file is split into segments of approximately the number of milliseconds specified. In order to provide different filenames, you should add %04d or similar at the point in the filename where you want a segment count number to appear e.g:

--segment 3000 -o video%04d.h264

will produce video clips of approximately 3000ms (3s) long, named video0001.h264, video0002.h264 etc. The clips should be seamless (no frame drops between clips), but the accuracy of each clip length will depend on the intraframe period, as the segments will always start on an I-frame. They will therefore always be equal or longer to the specified period.

The most recent version of Raspivid will also allow the file name to be time-based, rather than using a segment number. For example:

--segment 3000 -o video_%c.h264

will produce file names formatted like so: video_Fri Jul 20 16:23:48 2018.h264

There are many different formatting options available. Note than the %d and %u options are not available, as they are used for the segment number formatting, and that some combinations may produce invalid file names.

	--wrap,	-wr		Set the maximum value for segment number

When outputting segments, this is the maximum the segment number can reach before it’s reset to 1, giving the ability to keep recording segments, but overwriting the oldest one. So if set to 4, in the segment example above, the files produced will be video0001.h264, video0002.h264, video0003.h264, and video0004.h264. Once video0004.h264 is recorded, the count will reset to 1, and video0001.h264 will be overwritten.

	--start,	-sn		Set the initial segment number

When outputting segments, this is the initial segment number, giving the ability to resume a previous recording from a given segment. The default value is 1.

	--raw,	-r

Specify the output file name for any raw data files requested.

	--raw-format,	-rf

Specify the raw format to be used if raw output requested. Options as yuv, rgb, and grey. grey simply saves the Y channel of the YUV image.

raspiyuv

Many of the options for raspiyuv are the same as those for raspistill. This section shows the differences.

Unsupported options:

--exif, --encoding, --thumb, --raw, --quality

Extra options :

	--rgb,	-rgb		Save uncompressed data as RGB888

This option forces the image to be saved as RGB data with 8 bits per channel, rather than YUV420.

Note that the image buffers saved in raspiyuv are padded to a horizontal size divisible by 32, so there may be unused bytes at the end of each line. Buffers are also padded vertically to be divisible by 16, and in the YUV mode, each plane of Y,U,V is padded in this way.

	--luma,	-y

Only outputs the luma (Y) channel of the YUV image. This is effectively the black and white, or intensity, part of the image.

	--bgr,	-bgr

Saves the image data as BGR data rather than YUV.

Command Line Examples

Still Captures

By default, captures are done at the highest resolution supported by the sensor. This can be changed using the -w and -h command line options.

Take a default capture after 2s (times are specified in milliseconds) on the viewfinder, saving in image.jpg:

raspistill -t 2000 -o image.jpg

Take a capture at a different resolution:

raspistill -t 2000 -o image.jpg -w 640 -h 480

Reduce the quality considerably to reduce file size:

raspistill -t 2000 -o image.jpg -q 5

Force the preview to appear at coordinate 100,100, with width 300 pixels and height 200 pixels:

raspistill -t 2000 -o image.jpg -p 100,100,300,200

Disable preview entirely:

raspistill -t 2000 -o image.jpg -n

Save the image as a PNG file (lossless compression, but slower than JPEG). Note that the filename suffix is ignored when choosing the image encoding:

raspistill -t 2000 -o image.png –e png

Add some EXIF information to the JPEG. This sets the Artist tag name to Boris, and the GPS altitude to 123.5m. Note that if setting GPS tags you should set as a minimum GPSLatitude, GPSLatitudeRef, GPSLongitude, GPSLongitudeRef, GPSAltitude, and GPSAltitudeRef:

raspistill -t 2000 -o image.jpg -x IFD0.Artist=Boris -x GPS.GPSAltitude=1235/10

Set an emboss image effect:

raspistill -t 2000 -o image.jpg -ifx emboss

Set the U and V channels of the YUV image to specific values (128:128 produces a greyscale image):

raspistill -t 2000 -o image.jpg -cfx 128:128

Run preview for 2s, with no saved image:

raspistill -t 2000

Take a time-lapse picture, every 10 seconds for 10 minutes (10 minutes = 600000ms), naming the files image_num_001_today.jpg, image_num_002_today.jpg and so on, with the latest picture also available under the name latest.jpg:

raspistill -t 600000 -tl 10000 -o image_num_%03d_today.jpg -l latest.jpg

Take a picture and send the image data to stdout:

raspistill -t 2000 -o -

Take a picture and send the image data to a file:

raspistill -t 2000 -o - > my_file.jpg

Run the camera forever, taking a picture when Enter is pressed:

raspistill -t 0 -k -o my_pics%02d.jpg

Video captures

Image size and preview settings are the same as for stills capture. Default size for video recording is 1080p (1920x1080).

Record a 5s clip with default settings (1080p30):

raspivid -t 5000 -o video.h264

Record a 5s clip at a specified bitrate (3.5Mbits/s):

raspivid -t 5000 -o video.h264 -b 3500000

Record a 5s clip at a specified framerate (5fps):

raspivid -t 5000 -o video.h264 -f 5

Encode a 5s camera stream and send the image data to stdout:

raspivid -t 5000 -o -

Encode a 5s camera stream and send the image data to a file:

raspivid -t 5000 -o - > my_file.h264

Shell Error Codes

The applications described here will return a standard error code to the shell on completion. Possible error codes are:

C Define Code Description

EX_OK

0

Application ran successfully

EX_USAGE

64

Bad command line parameter

EX_SOFTWARE

70

Software or camera error

130

Application terminated by Ctrl-C

Long Exposures

The maximum exposure times of the three official Raspberry Pi cameras can be found in this table.

Due to the way the ISP works, by default asking for a long exposure can result in the capture process taking up to 7 times the exposure time, so a 200 second exposure on the HQ camera could take 1400 seconds to actually return an image. This is due to the way the camera system works out the correct exposures and gains to use in the image, using it’s AGC (automatic gain control) and AWB (automatic white balance) algorithms. The system needs a few frames to calculate these numbers in order to produce a decent image. When combined with frame discards at the start of processing (in case they are corrupt), and the switching between preview and captures modes, this can result in up to 7 frames needed to produce a final image. With long exposures, that can take a long time.

Fortunately, the camera parameters can be altered to reduce frame time dramatically; however this means turning off the automatic algorithms and manually providing values for the AGC.

The AWB gains can usually be omitted as the legacy stack is able to reprocess the camera data to work them out (the -st option), though it is fine to specify them as well. Additionally, burst mode (-bm) with a short timeout should be requested to suppress the initial preview phase, and the exposure mode also needs disabling (-ex off).

The following example will perform a 100 second exposure capture

raspistill -t 10 -bm -ex off -ag 1 -ss 100000000 -st -o long_exposure.jpg

Shooting RAW using the Camera Modules

The definition of raw images can vary. The usual meaning is raw Bayer data directly from the sensor, although some may regard an uncompressed image that has passed through the ISP (and has therefore been processed) as raw. For the latter, we recommend using the term unencoded so as to be clear about the difference.

Both options are available from the Raspberry Pi cameras.

Processed, Non-Lossy Images

The usual output from raspistill is a compressed JPEG file that has passed through all the stages of image processing to produce a high-quality image. However, JPEG, being a lossy format does throw away some information that the user may want.

raspistill has an encoding option that allows you to specify the output format: either jpg, bmp, png or gif. All but jpg are lossless, so no data is thrown away in an effort to improve compression, but do require conversion from the original YUV, and because these formats do not have hardware support they produce images slightly more slowly than JPEG.

e.g.

raspistill --encoding png -o fred.png

Another option is to output completely formatted YUV420 or RGB data using the raspiyuv application.

Unprocessed Images

For some applications, such as astrophotography, having the raw Bayer data direct from the sensor can be useful. This data will need to be post-processed to produce a useful image.

raspistill has a raw option that will cause the Bayer data to be output.

raspistill --raw -o fred.jpg

The raw data is appended to the end of the JPEG file and will need to be extracted.

Application Notes

Creating Timelapse Video

To create a time-lapse video, you simply configure the Raspberry Pi to take a picture at a regular interval, such as once a minute, then use an application to stitch the pictures together into a video. There are a couple of ways of doing this.

Using libcamera-still or raspistill Timelapse Mode

Both libcamera-still and raspistill have a built in time-lapse mode, using the --timelapse command line switch. The value that follows the switch is the time between shots in milliseconds:

libcamera-still -t 30000 --timelapse 2000 -o image%04d.jpg

or

raspistill -t 30000 --timelapse 2000 -o image%04d.jpg
Note

The %04d in the output filename: this indicates the point in the filename where you want a frame count number to appear. So, for example, the command above will produce a capture every two seconds (2000ms), over a total period of 30 seconds (30000ms), named image0001.jpg, image0002.jpg, and so on, through to image0015.jpg.

The %04d indicates a four-digit number, with leading zeros added to make up the required number of digits. So, for example, %08d would result in an eight-digit number. You can miss out the 0 if you don’t want leading zeros.

If a timelapse value of 0 is entered, the application will take pictures as fast as possible. Note that there’s an minimum enforced pause of approximately 30 milliseconds between captures to ensure that exposure calculations can be made.

Automating using cron Jobs

A good way to automate taking a picture at a regular interval is using cron. Open the cron table for editing:

crontab -e

This will either ask which editor you would like to use, or open in your default editor. Once you have the file open in an editor, add the following line to schedule taking a picture every minute (referring to the Bash script from the raspistill page, though you can use libcamera-still in exactly the same way):

* * * * * /home/pi/camera.sh 2>&1

Save and exit and you should see the message:

crontab: installing new crontab

Make sure that you use e.g. %04d to ensure that each image is written to a new file: if you don’t, then each new image will overwrite the previous file.

Stitching Images Together

Now you’ll need to stitch the photos together into a video. You can do this on the Pi using ffmpeg but the processing will be slow. You may prefer to transfer the image files to your desktop computer or laptop and produce the video there.

First you will need to install ffmpeg if it’s not already installed.

sudo apt install ffmpeg

Now you can use the ffmpeg tool to convert your JPEG files into an mp4 video:

ffmpeg -r 10 -f image2 -pattern_type glob -i 'image*.jpg' -s 1280x720 -vcodec libx264 timelapse.mp4

On a Raspberry Pi 3, this can encode a little more than two frames per second. The performance of other Pi models will vary. The parameters used are:

  • -r 10 Set frame rate (Hz value) to ten frames per second in the output video.

  • -f image2 Set ffmpeg to read from a list of image files specified by a pattern.

  • -pattern_type glob When importing the image files, use wildcard patterns (globbing) to interpret the filename input by -i, in this case image*.jpg, where * would be the image number.

  • -i 'image*.jpg' The input file specification (to match the files produced during the capture).

  • -s 1280x720 Scale to 720p. You can also use 1920x1080, or lower resolutions, depending on your requirements.

  • -vcodec libx264 Use the software x264 encoder.

  • timelapse.mp4 The name of the output video file.

ffmpeg has a comprehensive parameter set for varying encoding options and other settings. These can be listed using ffmpeg --help.

Using Gstreamer

Gstreamer is a Linux framework for reading, processing and playing multimedia files. There is a lot of information and many tutorials at the gstreamer website. Here we show how libcamera-vid (and similarly raspivid) can be used to stream video over a network.

On the server we need libcamera-vid to output an encoded h.264 bitstream to stdout and can use the gstreamer fdsrc element to receive it. Then extra gstreamer elements can send this over the network. As an example we can simply send and receive the stream on the same device over a UDP link. On the server:

libcamera-vid -t 0 -n --inline -o - | gst-launch-1.0 fdsrc fd=0 ! udpsink host=localhost port=5000

For the client (type this into another console window) we can use:

gst-launch-1.0 udpsrc address=localhost port=5000 ! h264parse ! v4l2h264dec ! autovideosink

Using RTP

To stream using the RTP protocol, on the server you could use:

libcamera-vid -t 0 -n --inline -o - | gst-launch-1.0 fdsrc fd=0 ! h264parse ! rtph264pay ! udpsink host=localhost port=5000

And in the client window:

gst-launch-1.0 udpsrc address=localhost port=5000 caps=application/x-rtp ! rtph264depay ! h264parse ! v4l2h264dec ! autovideosink

We conclude with an example that streams from one machine to another. Let us assume that the client machine has the IP address 192.168.0.3. On the server (a Raspberry Pi) the pipeline is identical, but for the destination address:

libcamera-vid -t 0 -n --inline -o - | gst-launch-1.0 fdsrc fd=0 ! h264parse ! rtph264pay ! udpsink host=192.168.0.3 port=5000

If the client is not a Raspberry Pi it may have different gstreamer elements available. For a Linux PC we might use:

gst-launch-1.0 udpsrc address=192.168.0.3 port=5000 caps=application/x-rtp ! rtph264depay ! h264parse ! avdec_h264 ! autovideosink

The libcamerasrc element

libcamera provides a libcamerasrc gstreamer element which can be used directly instead of libcamera-vid. On the server you could use:

gst-launch-1.0 libcamerasrc ! capsfilter caps=video/x-raw,width=1280,height=720,format=NV12 ! v4l2convert ! v4l2h264enc extra-controls="controls,repeat_sequence_header=1" ! h264parse ! rtph264pay ! udpsink host=localhost port=5000

and on the client we use the same playback pipeline as previously.

V4L2 Drivers

V4L2 drivers provide a standard Linux interface for accessing camera and codec features. They are loaded automatically when the system is started, though in some non-standard situations you may need to load camera drivers explicitly.

Driver differences when using libcamera or the legacy stack

On systems using libcamera, /dev/video0 and /dev/video1 are V4L2 drivers for Unicam, the Pi’s CSI-2 receiver. The Pi has two CSI-2 receivers, each managed by one of these device nodes.

On systems using the legacy stack, /dev/video0 is a V4L2 driver that gives access to the full camera system using the proprietary Broadcom driver on the GPU. There is no /dev/video1. There are no Unicam drivers, though there is a legacy MMAL Rawcam component.

The other device nodes are always the same, and are listed in the table below.

/dev/videoX Default Action

video10

Video decode.

video11

Video encode.

video12

Simple ISP. Can perform conversion and resizing between RGB/YUV formats, and also Bayer to RGB/YUV conversion.

video13

Input to fully programmable ISP.

video14

High resolution output from fully programmable ISP.

video15

Low result output from fully programmable ISP.

video16

Image statistics from fully programmable ISP.

video19

HEVC Decode

Using the Driver

Please see the V4L2 documentation for details on using this driver.

Camera Serial Interface 2 (CSI2) "Unicam"

The SoC’s used on the Raspberry Pi range all have two camera interfaces that support either CSI-2 D-PHY 1.1 or CCP2 (Compact Camera Port 2) sources. This interface is known by the codename "Unicam". The first instance of Unicam supports 2 CSI-2 data lanes, whilst the second supports 4. Each lane can run at up to 1Gbit/s (DDR, so the max link frequency is 500MHz).

However, the normal variants of the Raspberry Pi only expose the second instance, and route out only 2 of the data lanes to the camera connector. The Compute Module range route out all lanes from both peripherals.

Software Interfaces

There are 3 independent software interfaces available for communicating with the Unicam peripheral:

Firmware

Note
This interface is available only when using the legacy camera stack.

The closed source GPU firmware has drivers for Unicam and three camera sensors plus a bridge chip. They are the Raspberry Pi Camera v1.3 (Omnivision OV5647), Raspberry Pi Camera v2.1 (Sony IMX219), Raspberry Pi HQ camera (Sony IMX477), and an unsupported driver for the Toshiba TC358743 HDMI->CSI2 bridge chip.

This driver integrates the source driver, Unicam, ISP, and tuner control into a full camera stack delivering processed output images. It can be used via MMAL, OpenMAX IL and V4L2 using the bcm2835-v4l2 kernel module. Only Raspberry Pi cameras are supported via this interface.

MMAL rawcam component

Note
This interface is available only when using the legacy camera stack.

This was an interim option before the V4L2 driver was available. The MMAL component vc.ril.rawcam allows receiving of the raw CSI2 data in the same way as the V4L2 driver, but all source configuration has to be done by userland over whatever interface the source requires. The raspiraw application is available on github. It uses this component and the standard I2C register sets for OV5647, IMX219, and ADV7282M to support streaming.

V4L2

Note
The V4L2 interface for Unicam is available only when using libcamera.

There is a fully open source kernel driver available for the Unicam block; this is a kernel module called bcm2835-unicam. This interfaces to V4L2 subdevice drivers for the source to deliver the raw frames. This bcm2835-unicam driver controls the sensor, and configures the CSI-2 receiver so that the peripheral will write the raw frames (after Debayer) to SDRAM for V4L2 to deliver to applications. Except for this ability to unpack the CSI-2 Bayer formats to 16bits/pixel, there is no image processing between the image source (e.g. camera sensor) and bcm2835-unicam placing the image data in SDRAM.

|------------------------|
|     bcm2835-unicam     |
|------------------------|
     ^             |
     |      |-------------|
 img |      |  Subdevice  |
     |      |-------------|
     v   -SW/HW-   |
|---------|   |-----------|
| Unicam  |   | I2C or SPI|
|---------|   |-----------|
csi2/ ^             |
ccp2  |             |
    |-----------------|
    |     sensor      |
    |-----------------|

Mainline Linux has a range of existing drivers. The Raspberry Pi kernel tree has some additional drivers and device tree overlays to configure them that have all been tested and confirmed to work. They include:

Device Type Notes

Omnivision OV5647

5MP Camera

Original Raspberry Pi Camera

Sony IMX219

8MP Camera

Revision 2 Raspberry Pi camera

Sony IMX477

12MP Camera

Raspberry Pi HQ camera

Toshiba TC358743

HDMI to CSI-2 bridge

Analog Devices ADV728x-M

Analog video to CSI-2 bridge

No interlaced support

Infineon IRS1125

Time-of-flight depth sensor

Supported by a third party

As the subdevice driver is also a kernel driver, with a standardised API, 3rd parties are free to write their own for any source of their choosing.

Developing Third-Party Drivers

This is the recommended approach to interfacing via Unicam.

When developing a driver for a new device intended to be used with the bcm2835-unicam module, you need the driver and corresponding device tree overlays. Ideally the driver should be submitted to the linux-media mailing list for code review and merging into mainline, then moved to the Raspberry Pi kernel tree, but exceptions may be made for the driver to be reviewed and merged directly to the Raspberry Pi kernel.

Please note that all kernel drivers are licensed under the GPLv2 licence, therefore source code MUST be available. Shipping of binary modules only is a violation of the GPLv2 licence under which the Linux kernel is licensed.

The bcm2835-unicam has been written to try and accommodate all types of CSI-2 source driver as are currently found in the mainline Linux kernel. Broadly these can be split into camera sensors and bridge chips. Bridge chips allow for conversion between some other format and CSI-2.

Camera sensors

The sensor driver for a camera sensor is responsible for all configuration of the device, usually via I2C or SPI. Rather than writing a driver from scratch, it is often easier to take an existing driver as a basis and modify it as appropriate.

The IMX219 driver is a good starting point. This driver supports both 8bit and 10bit Bayer readout, so enumerating frame formats and frame sizes is slightly more involved.

Sensors generally support V4L2 user controls. Not all these controls need to be implemented in a driver. The IMX219 driver only implements a small subset, listed below, the implementation of which is handled by the imx219_set_ctrl function.

  • V4L2_CID_PIXEL_RATE / V4L2_CID_VBLANK / V4L2_CID_HBLANK: allows the application to set the frame rate.

  • V4L2_CID_EXPOSURE: sets the exposure time in lines. The application needs to use V4L2_CID_PIXEL_RATE, V4L2_CID_HBLANK, and the frame width to compute the line time.

  • V4L2_CID_ANALOGUE_GAIN: analogue gain in sensor specific units.

  • V4L2_CID_DIGITAL_GAIN: optional digital gain in sensor specific units.

  • V4L2_CID_HFLIP / V4L2_CID_VFLIP: flips the image either horizontally or vertically. Note that this operation may change the Bayer order of the data in the frame, as is the case on the imx219.

  • V4L2_CID_TEST_PATTERN / V4L2_CID_TEST_PATTERN_*: Enables output of various test patterns from the sensor. Useful for debugging.

In the case of the IMX219, many of these controls map directly onto register writes to the sensor itself.

Further guidance can be found in libcamera’s sensor driver requirements, and also in chapter 3 of the Raspberry Pi Camera Tuning Guide.

Device Tree

Device tree is used to select the sensor driver and configuren parameters such as number of CSI-2 lanes, continuous clock lane operation, and link frequency (often only one is supported).

Bridge chips

These are devices that convert an incoming video stream, for example HDMI or composite, into a CSI-2 stream that can be accepted by the Raspberry Pi CSI-2 receiver.

Handling bridge chips is more complicated, as unlike camera sensors they have to respond to the incoming signal and report that to the application.

The mechanisms for handling bridge chips can be broadly split into either analogue or digital.

When using ioctls in the sections below, an S in the ioctl name means it is a set function, whilst G is a get function and _ENUM enumerates a set of permitted values.

Analogue video sources

Analogue video sources use the standard ioctls for detecting and setting video standards. VIDIOC_G_STD, VIDIOC_S_STD, VIDIOC_ENUMSTD, and VIDIOC_QUERYSTD

Selecting the wrong standard will generally result in corrupt images. Setting the standard will typically also set the resolution on the V4L2 CAPTURE queue. It can not be set via VIDIOC_S_FMT. Generally requesting the detected standard via VIDIOC_QUERYSTD and then setting it with VIDIOC_S_STD before streaming is a good idea.

Digital video sources

For digital video sources, such as HDMI, there is an alternate set of calls that allow specifying of all the digital timing parameters (VIDIOC_G_DV_TIMINGS, VIDIOC_S_DV_TIMINGS, VIDIOC_ENUM_DV_TIMINGS, and VIDIOC_QUERY_DV_TIMINGS).

As with analogue bridges, the timings typically fix the V4L2 CAPTURE queue resolution, and calling VIDIOC_S_DV_TIMINGS with the result of VIDIOC_QUERY_DV_TIMINGS before streaming should ensure the format is correct.

Depending on the bridge chip and the driver, it may be possible for changes in the input source to be reported to the application via VIDIOC_SUBSCRIBE_EVENT and V4L2_EVENT_SOURCE_CHANGE.

Currently supported devices

There are 2 bridge chips that are currently supported by the Raspberry Pi Linux kernel, the Analog Devices ADV728x-M for analogue video sources, and the Toshiba TC358743 for HDMI sources.

Analog Devices ADV728x(A)-M Analogue video to CSI2 bridge

These chips convert composite, S-video (Y/C), or component (YPrPb) video into a single lane CSI-2 interface, and are supported by the ADV7180 kernel driver.

Product details for the various versions of this chip can be found on the Analog Devices website.

Because of some missing code in the current core V4L2 implementation, selecting the source fails, so the Raspberry Pi kernel version adds a kernel module parameter called dbg_input to the ADV7180 kernel driver which sets the input source every time VIDIOC_S_STD is called. At some point mainstream will fix the underlying issue (a disjoin between the kernel API call s_routing, and the userspace call VIDIOC_S_INPUT) and this modification will be removed.

Please note that receiving interlaced video is not supported, therefore the ADV7281(A)-M version of the chip is of limited use as it doesn’t have the necessary I2P deinterlacing block. Also ensure when selecting a device to specify the -M option. Without that you will get a parallel output bus which can not be interfaced to the Raspberry Pi.

There are no known commercially available boards using these chips, but this driver has been tested via the Analog Devices EVAL-ADV7282-M evaluation board

This driver can be loaded using the config.txt dtoverlay adv7282m if you are using the ADV7282-M chip variant; or adv728x-m with a parameter of either adv7280m=1, adv7281m=1, or adv7281ma=1 if you are using a different variant. e.g.

dtoverlay=adv728x-m,adv7280m=1

Toshiba TC358743 HDMI to CSI2 bridge

This is a HDMI to CSI-2 bridge chip, capable of converting video data at up to 1080p60.

Information on this bridge chip can be found on the Toshiba Website

The TC358743 interfaces HDMI in to CSI-2 and I2S outputs. It is supported by the TC358743 kernel module.

The chip supports incoming HDMI signals as either RGB888, YUV444, or YUV422, at up to 1080p60. It can forward RGB888, or convert it to YUV444 or YUV422, and convert either way between YUV444 and YUV422. Only RGB888 and YUV422 support has been tested. When using 2 CSI-2 lanes, the maximum rates that can be supported are 1080p30 as RGB888, or 1080p50 as YUV422. When using 4 lanes on a Compute Module, 1080p60 can be received in either format.

HDMI negotiates the resolution by a receiving device advertising an EDID of all the modes that it can support. The kernel driver has no knowledge of the resolutions, frame rates, or formats that you wish to receive, therefore it is up to the user to provide a suitable file. This is done via the VIDIOC_S_EDID ioctl, or more easily using v4l2-ctl --fix-edid-checksums --set-edid=file=filename.txt (adding the --fix-edid-checksums option means that you don’t have to get the checksum values correct in the source file). Generating the required EDID file (a textual hexdump of a binary EDID file) is not too onerous, and there are tools available to generate them, but it is beyond the scope of this page.

As described above, use the DV_TIMINGS ioctls to configure the driver to match the incoming video. The easiest approach for this is to use the command v4l2-ctl --set-dv-bt-timings query. The driver does support generating the SOURCE_CHANGED events should you wish to write an application to handle a changing source. Changing the output pixel format is achieved by setting it via VIDIOC_S_FMT, however only the pixel format field will be updated as the resolution is configured by the dv timings.

There are a couple of commercially available boards that connect this chip to the Raspberry Pi. The Auvidea B101 and B102 are the most widely obtainable, but other equivalent boards are available.

This driver is loaded using the config.txt dtoverlay tc358743.

The chip also supports capturing stereo HDMI audio via I2S. The Auvidea boards break the relevant signals out onto a header, which can be connected to the Pi’s 40 pin header. The required wiring is:

Signal B101 header Pi 40 pin header BCM GPIO

LRCK/WFS

7

35

19

BCK/SCK

6

12

18

DATA/SD

5

38

20

GND

8

39

N/A

The tc358743-audio overlay is required in addition to the tc358743 overlay. This should create an ALSA recording device for the HDMI audio. Please note that there is no resampling of the audio. The presence of audio is reflected in the V4L2 control TC358743_CID_AUDIO_PRESENT / "audio-present", and the sample rate of the incoming audio is reflected in the V4L2 control TC358743_CID_AUDIO_SAMPLING_RATE / "Audio sampling-frequency". Recording when no audio is present will generate warnings, as will recording at a sample rate different from that reported.