In this guide, our friends from Google walk you through how to install LiteRT and run high-performance on-device AI models using the LiteRT CLI tool. You can read more about the work Google has done to improve LiteRT performance on the Raspberry Pi platform here.

Step 1. Raspberry Pi OS Setup

This step is meant for those starting with a fresh Raspberry Pi device or those looking to install an updated version of Raspberry Pi OS.

1. Download the Raspberry Pi Imager tool from raspberrypi.com/software.
2. Launch the application.
3. Select Raspberry Pi 5 in the ‘Choose device’ menu.
4. Select Raspberry Pi OS (64-bit) under ‘Choose OS’.
5. Insert your SD card and select it under the ‘Choose storage’ column.
6. Enter a hostname (e.g. dino-pi), username (e.g. dino), and password (e.g. dinopi).

Note: Customize the hostname, username, and password to your preferences.

7. Enable SSH.

Note: You may wish to skip enabling Raspberry Pi Connect.

8. Click Save, then select Yes to begin the write process.
9. Click Finish once writing and verification are complete.

Step 2. Hardware and connection

1. Remove the SD card from your laptop and connect it via the SD card slot on your Raspberry Pi.
2. Connect the Raspberry Pi and your laptop together via an Ethernet cable.

Note: You may need to use an adapter if your laptop does not have a matching port.

3. Connect the Raspberry Pi to a power outlet using its power supply.

Step 3. Accessing the Raspberry Pi

1. Open your terminal and ping your Raspberry Pi to confirm it is reachable on your network.

Note: Replace ‘dino-pi’ with the hostname chosen during installation (Step 1.6).

ping dino-pi.local

2. SSH to the Raspberry Pi via SSH.

Note: When prompted, enter the password chosen during installation (Step 1.6).

ssh [email protected]

3. Verify the hardware architecture is aarch64.

uname -m

Step 4. Set up Hugging Face account and access

1. If you don’t already have one, create an account on Hugging Face. 
2. Go to your Hugging Face settings, click ‘Access Tokens’, then click ‘Create New Token’. This is referred to as <your_hugging_face_token_here> going forward.
3. Follow the instructions for generating a token with the token type of Read.

Step 5. LiteRT CLI installation

1. Update the system and refresh the available software packages.

sudo apt update

sudo apt full-upgrade -y

2. Download and install the uv tool manager to handle your AI environments.

curl -LsSf https://astral.sh/uv/install.sh | sh

3. Refresh your shell environment to ensure it recognizes the newly installed tools.

source $HOME/.local/bin/env

4. Use the uv manager to install the LiteRT CLI, which can run and benchmark various models such as language models (via LiteRT-LM) or vision models.

uv venv --clear --python=3.13 --seed

source .venv/bin/activate

uv pip install litert-cli-nightly

Step 6.a. Run large language models 

1. Navigate to the desired Gemma model (e.g. gemma-4-E2B-it-litert-lm).

Note: You may be required to accept the Gemma model terms for certain models.

2. Export your token from Step 4 and run the litert lm command to start a conversation.

export HUGGING_FACE_HUB_TOKEN=<your_hugging_face_token_here>

litert lm run \

  --from-huggingface-repo=litert-community/gemma-4-E2B-it-litert-lm \

  gemma-4-E2B-it.litertlm \

  --prompt="What is the capital of France?"

Example output

dino@dino:~ $ litert lm run \

  --from-huggingface-repo=litert-community/gemma-4-E2B-it-litert-lm \

  gemma-4-E2B-it.litertlm \

  --prompt="What is the capital of France?"

Downloading gemma-4-E2B-it.litertlm from litert-community/gemma-4-E2B-it-litert-lm...

gemma-4-E2B-it.litertlm: 100%|██████████████████████| 2.59G/2.59G [02:06<00:00, 20.4MB/s]

The capital of France is **Paris**.

Step 6.b. Run classic machine learning models 

Find, download, and run the desired LiteRT model (e.g. efficientnet_b1). For example, you can download and run EfficicientNet for image classification:

litert download litert-community/efficientnet_b1 --output efficientnet

litert run efficientnet/efficientnet_b1.tflite --input <your-image.JPEG>

Example output

dino@dino:~ $ litert run efficientnet/efficientnet_b1.tflite --input 

shark.JPEG 

...

Outputs:

  linear (Top 5 Predictions):

    1: index 3 (tiger shark, Galeocerdo cuvieri) - score 7.4043

    2: index 2 (great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias) - score 4.7619

    3: index 4 (hammerhead, hammerhead shark) - score 3.3590

    4: index 395 (gar, garfish, garpike, billfish, Lepisosteus osseus) - score 1.7937

    5: index 0 (tench, Tinca tinca) - score 1.2683

Step 7. Run model with Raspberry Pi’s GPU (optional)

LiteRT uses WebGPU for GPU acceleration. It takes advantage of the Dawn WebGPU implementation, which can run on a Vulkan driver. Raspberry Pi 5 uses the V3DV open source Vulkan driver shipped with Mesa.

Experimental WebGPU support is available as of today. It can be enabled with the V3D_WEBGPU_OVERRIDE=1 environment variable using the updated official Raspberry Pi OS Mesa package.

WARNING: The GPU provides lower performance than the CPU. This is expected behavior, as WebGPU support is experimental in the V3DV Vulkan driver. Raspberry Pi’s CPU currently outperforms the GPU on this workload.

1. Run LiteRT-LM using your Raspberry Pi’s GPU.

export V3D_WEBGPU_OVERRIDE=1

litert download litert-community/efficientnet_b1 --output efficientnet

litert run efficientnet/efficientnet_b1.tflite \

  --input <your-image.JPEG> \

  --gpu

Note: You need to set export V3D_WEBGPU_OVERRIDE=1 to activate GPU optimization.

What’s next?

We are excited to share that LiteRT integration and Gemma models are coming soon to Hailo AI accelerators! This update will allow you to seamlessly offload model inference to the Raspberry Pi AI HAT+ and AI HAT+ 2, delivering massive hardware acceleration benefits through the exact same LiteRT workflows you use today.

Explore our resources and start your journey with LiteRT:

• Official documentation: Access installation guides, API references, and quick-start tutorials on the LiteRT Developer Site.
• GitHub repos: Find the latest source code, implementation details, and updates on the LiteRT and LiteRT-LM GitHub repos.
• Samples and templates: Check out the LiteRT-Samples GitHub repo for reference code. Kick-start your own application using the Google AI Edge Gallery app.
• Ready-to-use models: Download optimized, open-weight models like the lightweight and powerful Gemma 4 E2B directly from the LiteRT Hugging Face Community.

We value your input. Please share your thoughts, feedback, or feature requests by opening an issue on our GitHub issue tracker. Share your cool Raspberry Pi + LiteRT + Gemma projects with @googlegemma. We can’t wait to see what you build!

Acknowledgements

Google: Changming Sun, Chintan Parikh, Cormac Brick, Daisuke Majima, Dillon Sharlet, Erin Walsh, Frank Barchard, Glenn Cameron, Ian Ballantyne, Jingjiang Li, Jun Jiang, Kimish Patel, Lu Wang, Matthias Grundmann, Rodney Witcher, Sachin Kotwani, Sasha Denisov, Scott Loftin, Shuangfeng Li, Somdatta Banerjee, Terry (Woncheol) Heo, Volodymyr Kysenko, Weiyi Wang, Yi-Chun Kuo, Yu-hui Chen, and the gTech team

Raspberry Pi: Ashley Whittaker, Naushir Patuck, and Sarah Cunningham

Igalia: José María Casanova

Hailo: Eldad Rubinstein

Ultralytics: Francesco Mattioli, Lakshantha Dissanayake, and Onuralp Sezer

Moonshine AI: Pete Warden