This LEGO robot arm copies your hand gestures

We love this project from Creator Academy in New South Wales, Australia. It’s a LEGO robot arm using the LEGO Mindstorms Robot Inventor (51515) and Raspberry Pi Build HAT.

How does it work?

A Raspberry Pi 4 powers the whole thing and a Raspberry Pi camera acts as the eyes for the robot arm. Open CV and MediaPipe take care of the machine learning that allows this build to recognise different finger and hand gestures.

Computer Vision Robot Arm Tutorial (Build HAT)
You can see each of the white finger and thumb motor cables plugged into the Build HAT on the left. And our Raspberry Pi camera sits perfectly snugly in LEGO front and centre

Each of the robot arm’s finger and thumb motor cables connects to one of the four ports on the Raspberry Pi Build HAT. The middle finger and ring finger have to share a port, but then, have you tried moving these two fingers independently on your own hand?

The robot arm mirrors the hand gestures it detects in front of the Raspberry Pi camera.

I want to build a robot arm!

Here is Garry Law’s (aka Mr Code) full tutorial video to help you build your own:

Go to this page if you prefer written PDF instructions rather than a video. Buying the PDF instructions helps support Mr Code to make more project videos. And all the code you’ll need is available on GitHub.

Computer Vision Robot Arm Tutorial (Build HAT)
It’s learning!

You will need:

What is Creator Academy?

Visit to to find out about robotics classes for young people and professional development courses for teachers. Some of the professional development courses can be delivered remotely but you’ll need to be lucky enough to live in beautiful New South Wales to access the classes for young learners.

No comments

Comments are closed