These photos show images of different weed species that the researchers trained the artificial intelligence (AI) machine vision model to recognize. That model is intended to guide an automated robotic precision herbicide spraying unit under development in the Department of Agricultural and Biological Engineering to control weeds in apple orchards. Credit: Penn State

Weed control is essential in apple orchards because weeds compete with trees for nutrients, water and sunlight, which can reduce fruit yields. However, physically removing weeds is not only labor-intensive, but it also can damage soil structure and tree roots. Using chemical sprays to kill weeds can lead to other problems, such as pollution, herbicide resistance and excess chemical residues on apples.

Precision weed management and robotics

Another option called precision weed management—detecting and measuring weeds with high accuracy then applying small amounts of herbicide to control them efficiently—can help farmers avoid wasting chemicals or causing injury to crops or the environment, according to a team of researchers at Penn State. To help growers achieve such precise management, the researchers are developing an automated, robotic weed-management system.

The researchers reported on an early step in that process in the December issue of Computers and Electronics in Agriculture: an AI machine vision model they developed that can accurately find, outline, interpret and estimate the density of weeds in apple orchards.

Machine vision and side-view cameras

The system, intended to guide the eventual robotic precision sprayer, uses a machine vision innovation that allows a side-view camera to detect and identify weeds for treatment—even weeds that are partially obscured.

"In complex environments like apple orchards, it is difficult to develop weed-detection mechanisms because the tree canopy and low branches block the view from above, precluding traditional top-view camera systems, like drones, because they can’t clearly see the weeds on the ground," said team leader and study senior author Long He, associate professor of agricultural and biological engineering. His research group in the College of Agricultural Sciences has been studying and developing robotic precision agricultural systems over the last decade.

"A side-view camera can help, but weeds might be partially visible or hidden behind untargeted objects or tree trunks. This causes problems such as misidentifying weeds or losing track of a weed in real time."

Improving deep-learning models for detection

To overcome those challenges, study first author Lawrence Arthur, doctoral candidate in the Department of Agricultural and Biological Engineering led the team in improving a commercially available deep-learning model for the machine vision computer program. The model was already capable of fast object detection and segmentation, meaning it can find the weed and outline its exact shape, pixel by pixel.

To make it better, the researchers added a module that helps the model "pay attention" to the most important image features while suppressing irrelevant feature information in the scene. This innovation improved accuracy when parts of weeds are hidden or hard to distinguish, He explained. Also, they integrated a tracking algorithm with a filtering mechanism for more effective weed tracking.

The algorithm preserves weed identity across video frames and prevents counting the same weed multiple times, He said. This allows the system to track weeds when they disappear briefly due to being blocked by the apple trees or even other weeds.

Testing and results in apple orchards

The study data were collected at Penn State’s Fruit Research and Extension Center in Biglerville and nearby apple orchards. Weed species included dandelion, common sow thistle, horseweed and Carolina horsenettle. The researchers took high-resolution photos of these weeds to form the dataset for training and testing the AI model.

"The model achieved high accuracy and recall for weed detection, making it suitable for automated weed management in orchards," He pointed out. It achieved 84.9% average precision for detecting segmentation and 83.6% average precision for localization. These numbers indicate strong accuracy for finding weeds and outlining them.

In tracking images across frames, the model scored 82% in multiple object tracking accuracy—meaning it achieved high accuracy in tracking multiple weeds; 78% in multiple object tracking precision—meaning it achieved good precision in estimating weed positions; and 88% in identification score—showing a strong ability to correctly maintain weed identities across video frames. Finally, the model only recorded six identity switches in the study, He noted, meaning the model rarely confuses one weed for another as it tracks them.

Future impact and research team

The research is a big step toward automated, precise weed control in agriculture, according to He.

"By combining better detection and stronger tracking with added density estimation, the model we developed provides more accurate, consistent weed detection, even in difficult orchard conditions," He said. "By providing actionable data for site-specific weed management, this approach will improve herbicide efficiency and reduce waste."

Contributing to the research were Caio Brunharo, assistant professor of weed science; Paul Heinemann, professor of agricultural and biological engineering; Magni Hussain, assistant research professor of electronics, instrumentation and control systems; and Sadjad Mahnan, graduate assistant in the Department of Agricultural and Biological Engineering.

More information: Lawrence Arthur et al, YOLOv7-CBAM and DeepSORT with pixel grid analysis for Real-Time weed localization and Intra-Row density estimation in apple orchards, Computers and Electronics in Agriculture (2025). DOI: 10.1016/j.compag.2025.111071

Citation: Eyes for an agricultural robot: AI system identifies weeds in apple orchards (2025, December 4) retrieved 4 December 2025 from https://phys.org/news/2025-12-eyes-agricultural-robot-ai-weeds.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.