Armor detection algorithm based on OpenCV and CNN

• 📚Unique architecture: using a traditional algorithm to extract armors’ features and then using an armor classifier to get classification of armors.
• ⚡️High Efficiency: Compact network architecture with a streamlined framework that can achieve 150+ FPS.
• 🔝High accuracy: 97% precision and 98% recall can be achieved.

Project Introduction

Armor plate recognition algorithm based on OpenCV and CNN. The algorithm can reach 97% precision, 98% recall and 150+FPS. This project is mainly used in RoboMaster, a national college robotics competition, where self-aiming of enemy robots is required. The traditional algorithm of self-aiming requires the use of traditional image processing algorithms and armors classifier to achieve the recognition and classification of enemy armors.

Project WorkFlow

The overall flow of the project is shown in the figure below:

The overall project can be divided into 6 steps. Firstly the image to be detected needs to be acquired, in the competition we use an industrial camera for image acquisition. In the second step we need to binarize the image for the subsequent detection process. The third step is to find the contours in the binarized image and filter the contours to get all the possible light bars in the image. The fourth step is to match all possible light bars by setting certain matching conditions, screening out the light bars that fail to match or the armor plate parameters obtained after matching do not meet the conditions at all, and obtaining all possible armors regions in the image. The fifth step inputs the ROIs of all possible regions of the armor into the CNN for classification to get the categories, and filters out the incorrect armors that are classified as negative samples. The sixth step gives the coordinates of all the correct armors in the figure along with their categories.

LightBar Screening and Armor Matching

In order to obtain the position of the armor stably, we first need to binarize the image. In the competition, the light bar of the armor of the target robot will only be red or blue, so we first take the image of the blue channel/red channel for binarization, at the same time, in order to further improve the stability of the binarization, we then convert the original image to grayscale and then binarize it, and then finally, we will do the sum operation with the two binarized images to get the binarized image that we will use in the end.

Then find the contours in the binarized image and get the outer rectangles of all the contours, and filter out all the regions that do not satisfy the conditions by the area, aspect ratio and tilt angle of the outer rectangles. Finally, all the light bars that pass the screening are matched two by two as an armor, and the armor is screened again by the distance between the two light bars, the tilt angle and the aspect ratio of the armor formed after matching, etc., to get all the regions in the figure that may be an armor and input them into the classifier for the final judgment.

Armor Classifier

In the game, there existed a total of eight types of armor plate, and we also added a class for negative samples, the armor looks like numbers and some simple images, so we chose to go to build a very small CNN to carry out classification. The network consists of three convolutional and downsampling layers, and two fully connected layers, the specific network structure is as follows:

Project effect

Algorithm performance

Overall effect of algorithm

CNN accuracy and confusion matrix