Is it possible for a physically intelligent person to break down the paradox of harvesting lithium e

Traditional single-arm harvesting schemes, limited operational coverage, frequent adjustment of machine position throughout the harvest, slow pace of operation, and difficulty in matching the demand for scale-up greenhouse production efficiency.

Simply increasing the number of mechanical arms cannot be directly translated into improving operational efficiency, and the densely planted environment is extremely vulnerable to arm movement interference, misallocation of tasks, inefficient waiting, and so on.

Planned multi-armed design for robot reception

R&D using collaborative arm+double module three-axis straight-line skateboard complex multi-arm harvesting schemes, the hardware is divided into mobile skateboard modules, 6-axis collaborative mechanical arm modules, visual perception modules, end-user modules, each unit using standardized quick-switching interfaces for easy maintenance and quick replacement of end-user parts.

STPE collaboration framework

On top of the hardware, innovation builds on the STPE closed-loop integration collaboration planning framework, which addresses multi-arms and concurrent operations conflicts from an algorithmic level, with four key components to achieve secure and efficient collaboration scheduling:

1. Spatial structuring: to separate the collection of target clusters, to generate gateway candidates, to structure and dismantle complex spatial collection tasks, and to reduce the computational expense of solving algorithms in high density planting scenarios.

2· Timeline synchronization: A two-layer alternating solution strategy at the gate-level, completing task allocation and mis-peak scheduling, avoiding armor space collisions from the source.

3. Route planning: Combine the actual distribution of the field strings to generate the best reference movement trajectory for the various mechanical arms.

4. Execute correction: high frequency safety distance monitoring + multi-speed closed-loop correction mechanism for repairing low-frequency trajectories, for site changes such as plant disruption, strain vibration, dynamic modification of mechanical arm movement instructions.

Plans to collect robot test scenes

The results of the simulation and sampling tests showed that the average multi-arm pulse rate of up to 3.8 s/string, compared to traditional sports planning schemes, shortened the total production cycle by 7.3%-14.2%, and significantly improved system robustness.

Displays running a single-key autonomously labeled software

Multi-arm systems have strict requirements for the consistency of the multi-machine coordinate system, and coordinate deviations can directly cause the diversion of the pickup point, causing leakage and damage to the impact. For industrial batch deployment of low-efficiency labeling engineering pain points, the development of a key autonomous hand-eye labeling software based on the ArUco identifier and a depth-point cloud matching, completes the coordinate mapping solution between the camera and the multi-machine arm, compressing the overall hand-eye timing from 1 hour to 15 minutes.

Detection of measuring devices such as laser trackers

At the same time, using laser trackers, modular mechanical arm coordinate systems are being corrected, correcting axis orientation deviations and spatial coordinate errors, ensuring positioning consistency and laying the foundation for efficient and stable harvesting.

Greenhouse environmental lighting fluctuates sharply, leaves are shaded in large quantities, cranberries continue to vibrate with the wind, which is very likely to cause target leakage, location drift at the point of collection.

The precision of identification and positioning of the visual system, which directly determines the accuracy of the collection of the work, while also controlling the preconditions of the damage to the fruit.

Image of the two-mode visual system

The R&D team collected a large number of field samples, built a dedicated data set of greenhouse real-life working conditions, and developed a system of perceptual algorithms to adapt to complex agricultural scenarios:

1. Lightweight maturity recognition algorithm: combining LAMP channel cutting with SegNeXt attention mechanism, completing model lightweighting, achieving high-speed reasoning on embedded hardware, determining the maturity of the fruit sequence, and harvesting only for mature fruit execution.

Map of network institutions

Two, dual-collection critical point detection algorithms: combined with the characteristics of tomato biology students, output the two best cut points at the endpoints, providing the end institution with precise execution targets.

Figure for the process of calculating the double extraction point

3 √ Detailed depth estimation method: Filtering depth noise with coordinate mapping based on ArUco real-value labeling, obtaining the true three-dimensional spatial location of the depth noise, solving the problem of depth camera distance noise measurement.

Accurate conversion of coordinates from visual positioning to robot execution

4 Synchronous rotational trajectory prediction algorithm: using adaptive Kalman wave propagation to recommend the position and speed of the rotation of the strings, constructing MS‐OTG multi-joint synchronous rolling update mechanism, continuously updating the trajectory of the mechanical arm movement in visual sampling intervals, and achieving continuous tracking of the positioning of the dynamic rotation of the strings.

Graph of multi-junction synchronous online trajectory generation methods for tomato harvesting

5-Integrated assay data: average fruit recognition accuracy is 97.8%, positioning accuracy is 96%, and algorithm reasoning speed is 22 FPS. High-precision sensing can guide the end device to accurately align the fruit, reducing the risk of damage to the fruit due to mistaken touching from the source.

The end-hand claws are the core components of the direct interaction between robots and crops, and are the last link in achieving low-damaged harvesting. Both to achieve reliable cutting, and to avoid holding, squeezing, squeezing, and touching the fruit.

The project has been self-developed and integrated with claw harvesting, integrated holding mechanism with cutting blade to achieve claw limiting, claw cutting synchronous completion. The mechanism uses aluminum alloy lightweight modulating structure, knives, claw block supporting rapid disassembly and replacement, claw course adjustable, suitable for different claws, different shapes and strains of tomato harvesting.

It is designed to integrate cutting to remove the claws.

In collaboration with the three institutions of multi-arm coordinated scheduling, high-precision visual positioning and cutting-edge terminal, the integrated field test: harvesting speed 50 kg/h, harvesting completeness 92%, fruit damage rate controlled at 3%, integrated stability of continuous operation for more than 4 hours, achieving high efficiency, high completeness, low damage harvesting targets.

Continuous operational tests to collect the scene of the machine's production

Hardware institutions and algorithm models require support for human-machine interaction software to land on the production site. Support for self-developed APP extraction assistants, which can be used on tablets, mobile terminals, open robot deployments, mission launches, state monitoring, operational data statistics, and reduce the use threshold.

Here's how to extract the APP software interface

The main core technologies and functions of the software are divided into four main modules:

1. Localized deployment and map management modules: support map loading, point marking, region selection. Operators can select the area of operation to be collected in the visualized interface, launch the task of collection, and complete the rapid deployment of the robot's operational scenarios.

2. Integrated real-time state monitoring module: real-time retransmission of information to alert the system failure. In the event of an exceptional situation, the APP immediately pops the window to alert the police, facilitating quick disposal by the on-site personnel.

3 Visualization of operational processes and manual intervention modules: real-time viewing of the progress of the collection of operations, support for one-click start, stop and finish tasks.

4. Operations data statistics modules: automatically record single-series harvesting time, harvesting success rate, harvesting completeness, and other core indicators. Facilitates maintenance personnel to replicate operational results, and provides real field data support for algorithm parameter optimization and crop conditions optimization.

Image of the human-machine interface

The entire set of APPs is deeply coupled with the robot-based ROS general control system, achieving a sensing-decision-executing-human-control loop, allowing the collection system not only to have strong intrinsic hardware and algorithmic capabilities, but also to be operable in agricultural settings.

Focus on innovation

New road to smart agriculture

The difficulty of agricultural robots has never been limited to laboratory simulations, but to engineering capabilities in complex, unstructured field environments.

The laboratory project will transfer the engineering capabilities of industrial collaboration robots, multi-axis motion control, systems integration to the facility's agricultural track, around the full chain of perception-decision-implementation, and break down the common problems of the agricultural robotics industry: uncertainty, instability, collaboration difficulties, and costly landing.

In the future, the laboratories will continue to deepen the agricultural robotics trajectory, continuously iterate the tomato harvesting robot series of products optimized facilities, further refine the agricultural harvesting integration solutions, promote the wider deployment of intelligent harvesting equipment, facilitate the transformation of facilities and upgrade agriculture, and empower rural communities with robotics technology!

* Articles translated from Chinese to Chinese