• 🚩End-to-end: end-to-end direct output of the four corner points of the armor.
• ⚡High speed: can reach 80+ FPS.
• 🏆Easy Deployment: Rapid Model Deployment with OpenVINO.

Project Introduction

This is a four-points Armor plate recognition algorithm for RoboMaster based on NanoDet. In the RoboMaster Robot Self-Aiming System Recognition section we also tried an end-to-end object detection scheme. Due to the need to calculate the distance between the enemy robot and our robot, we have to get the four corner points of the enemy robot’s armor, the traditional object detection algorithms get the upper-left corner points of the rectangular box as well as the length and width of the box, so that the output will always be a positive rectangle, but our armor may be tilted in the field of play, so we modified the loss function of the traditional object detection and the detection head, changing the final output directly to the four corner points of the quadrilateral, which greatly improves the operational efficiency of the recognition network. This project is based on the modification on NanoDet-plus, and we are very grateful to the authors for the open source!

Project WorkFlow

The overall flow of the project is as follows:

Our project makes changes based on NanoDet-Plus, the main parts changed are the loss function during training and the detection head of the network.

The backbone is a lightweight ShuffleNetV2, which can greatly improve the speed of the network while guaranteeing the recognition stability. In the feature fusion part, we directly adopt the method provided by the authors, and use the lightweight and high-performance GhostPAN.The detection head part finally has three scales of detection head outputs, and each scale uses the same set of convolutions for edge regression and classification, and finally splits them into two parts, and based on the task we want to achieve, we keep the classification part of the original detection head, and change the original four outputs to the corresponding regression part. The original four outputs are changed to eight outputs corresponding to the coordinates of the four points of the rectangle.

In the training part, we also use the training assistance module Assign Guidance Module (AGM) provided by the authors together with the Dynamic Soft Label Assigner (DSLA) to solve the optimal label matching problem in the lightweight model. This module is only used during the training process and discarded directly after training without affecting the speed of the network. The original authors used classification loss, IoU loss, and dis loss in the loss function part, but due to the specificity of our detection task, the IoU calculation of the rotated rectangle is very complicated, and because the regression of the four corner points needs to consider whether the generated polygon is convex or concave polygon, the IoU loss is very difficult to converge, so we delete this part, and we also change the original dis Loss to L1 Loss to regress the coordinates of the corner points.

Project Performance

Based on our own photographed and labeled armor dataset for testing, the model accuracy is as follows:

The inference speeds deployed on Intel’s NUC platform after OpenVINO optimization are as follows: