Every dog has its day, and this is FREISA’s time to shine. This #MagPiMonday, David Crookes takes a look at an innovative plant-watering system.

Real dogs tend to be fluffy and cute. Robodogs rather less so. But when you meet FREISA, a robotic dog created for a recent competition, you’ll still want to give it a pat on the back and heap it with praise. For, just like its living, breathing canine counterparts, this impressive little thing is intelligent and skilled. It’s capable of spotting a thirsty plant and making sure it gets enough water.

FREISA – an acronym for Four-legged Robot Ensuring Intelligent Sprinkler Automation – is the brainchild of the B-AROL-O Team. This group won the popular vote of the OpenCV Spatial AI Contest in 2022 by creating a packaging machine called ARNEIS using Raspberry Pi 4B, a Spatial AI camera and an assortment of LEGO Technic parts. 

Seeking to make a similar impact in last year’s competition, they got their heads together once again. “As implied by our team name, our familiarity with vineyards inspired us to develop an automated watering system aimed at assisting on a farm,” says one of the group, Gianluca Teti. The idea was to use computer vision to water plants and come up with a device capable of carrying out the work.

They ruled out creating a robot running on wheels and tracks, noting that it would be poor at clearing obstacles over rough ground. The group also briefly considered a quadcopter but felt its battery and payload would be poor. A quadruped legged platform was chosen because of its ability to navigate complex terrain. 

“We initiated a brainstorming session with the team to assess our available hardware options for the project,” Gianluca adds. “Orso Eric, one of our team members, owned a Mini Pupper 2. Consequently, we collectively agreed to utilise that specific model for our project.”

Re-tail therapy

The Mini Pupper 2 is an open source robot kit that sprang into life off the back of a crowdfunding campaign. The quadrupedal robot supports Raspberry Pi, which the team planned to use to handle FREISA’s navigation. To ensure it would be suitable as a robot plant waterer, an OAK-D Lite camera boasting SpatialAI functionality was connected. This camera can run computer vision pipelines on dedicated hardware which, the team surmised, would reduce the burden on Raspberry Pi’s resources.

To ensure the project would actually be suitable for its intended purpose, the team carried out some research. “We conducted interviews with several experienced farmers and gardeners,” says Gianluca. “Their insights suggested that visually inspecting plants can offer a reliable assessment of their condition.” 

With this in mind, the team got to work, training two neural networks to work out which plants would need watering and which ones would not. By this point, they had ditched the idea of analysing vines because they noted the leaves would grow too high for the robot to see them, and the vineyard terrain would be too uneven. Instead, they decided to concentrate their efforts on a household garden populated with tomato plants.

“The objective was to assess a plant’s condition by analysing the state of its leaves and stems,” Gianluca explains. “Our approach began with collecting a vast array of images and then training various networks and models. Our infrastructure consisted of one desktop computer – featuring an AMD Ryzen 2600X CPU, 48 GB of RAM, and an NVidia GTX 1660 Super GPU – running Ubuntu 23.04 and accessible remotely via a JupyterLab container.”

Petting on success

For the time being, the team decided that using visuals would be sufficient, although there is scope to further improve accuracy. “Integrating additional sensors – such as those measuring humidity, temperature and so on – into the FREISA project’s framework could be beneficial,” Gianluca says. “The architecture of the FREISA project appears adaptable enough to readily accommodate these supplementary sensors.”

By using Ubuntu rather than Raspberry Pi OS, the team says their work was made a little easier. “Our decision leaned towards an Ubuntu server over Raspberry Pi OS, despite their potential equivalence for our current application, because Ubuntu stands as the primary supported platform for ROS (Robot Operating System), a framework we intend to implement soon to expand the capabilities of our platform for additional use cases,” Gianluca explains.

Having Raspberry Pi on board, however, has worked extremely well. “Raspberry Pi serves as an exceptional hardware foundation, notably due to its extensive array of open-source software and programming tools,” Gianluca continues. “Furthermore, the Mini Pupper 2 is constructed using the Raspberry Pi Compute Module 4, aiding in space and weight reduction, consequently enhancing the available payload on the robot dog.”

Round of a-paws

The great news is that the project – which makes use of Docker (a software platform that lets developers quickly build, test and deploy applications) and OpenCV – has been effective. The robot has a sprinkler in a pelvic position with the water reservoir above it, and it’s able to move freely to quench a plant’s thirst.

“Our robot demonstrates good abilities in identifying a target plant, navigating towards it, and examining its leaves before precisely adjusting its position to efficiently water the plant,” Gianluca explains. Even so, there have been challenges; some practical and some logistical. “The primary hurdle we encountered was identifying the most suitable terrain for deploying the watering system,” he continues.

“A secondary challenge involved effectively coordinating tasks among a team comprising four active members and two advisors situated across various locations worldwide: the US, France and Italy.

“To overcome these challenges, we implemented an agile development workflow based on weekly sprints. With the help of a CI/CD pipeline built with GitHub and other online resources, team members collaborated effectively, enabling successful project management and task allocation despite geographical distances.”

FREISA won the Grand Prize in 2023’s OpenCV AI Competition. Still, the team isn’t going to rest on its laurels. It’s already identified potential improvements such as using the OAK-D Lite’s stereo camera to estimate a target’s distance. Just goes to show, you can’t keep a good dog down.