🏆 How MPWARE topped the leaderboard in MosquitoAlert Challenge 🦟

Today, we’re diving into the story of Eric, a seasoned software engineer, whose journey through the MosquitoAlert Challenge is an example of the power of continuous learning and practical application in the realm of artificial intelligence. Eric’s unique approach to tackling machine learning challenges combines effective problem-solving with tangible, hands-on experience. This challenge has enhanced his expertise in areas such as algorithm development, data analysis, and model optimization. Let’s hear how he ranked third in the competition and dive deeper into his solutions.

🦟 Discovering the MosquitoAlert Challenge

It all started when Eric was looking for a new project to sink his teeth into during the summer and found the MosquitoAlert challenge. The detailed problem statement, coupled with an engaging townhall event featuring seasoned experts, sparked his curiosity.

🔍 Technical Exploration

In this challenge, Team MPWare led by Eric embarked on a two-pronged solution to efficiently detect and classify mosquitoes.

Stage 1: Tackling Mosquito Detection Using Yolo

The first stage was critical for the initial identification and localization of mosquitoes in images, employing a Yolo-based model to detect mosquito bounding boxes.

Stage 2: The Ensemble Approach for Mosquito Classification

In the second stage, the team utilized an ensemble of two distinct models for the classification task. One model was built on the Vision Transformer (ViT) architecture, renowned for its efficacy in image recognition tasks. The second model leveraged the EfficientNetv2 architecture, celebrated for its efficiency and accuracy in image classification.

🧩 Inference Challenges and Solutions

Eric and his team faced a challenge — a strict 2-second limit per image for processing on a CPU-only setup. This constraint restricted the number of models Team MPWare could ensemble and limited Test-Time Augmentation (TTA) options. But with strategic thinking, they turned to the OpenVino conversion of the YoloV8 model. This moved helped them maintain and improve the model’s mean Average Precision (mAP) on cross-validation without affecting the inference time, which was around 400–500ms.

In the final stages, the team employed a weighted boxes fusion method for final bounding box computation, followed by feeding these images into both classifiers. This helped them ensure a refined and precise localization. The images corresponding to these bounding boxes were then fed into both classifiers. The classifiers were run through Pytorch, and despite not having runtime optimizations, they operated efficiently within 600–700ms.

🏋️ Training Approach

Eric’straining was tailored to fit within the competition’s stringent runtime limits, focusing on efficient model selection and optimization. The choice of the YoloV8 nano model for bounding box detection was a strategic choice to blend speed and accuracy.

Stage 1: Efficient Bounding Box Detection

Eric’s team began with the YoloV8 nano model for bounding box detection. This model was chosen for its balance of speed and accuracy. The task did not require the complexity of larger Yolo models. The YoloV8 nano’s performance was comparable to its larger counterparts but had a faster inference time, crucial for this stage.

📏 Training Details for Bounding Box Detector

The team used a cross-validation method divided into 4 folds, based by species and mosquito surface area in the images. Training used 10,000 images, plus 1,000 background images to enhance model robustness. Training parameters included a 768x768 image size and augmentations such as rotation, shearing, and mixup. The training lasted 128 epochs with a batch size of 16, using an SGD optimizer and a linear learning rate scheduler. In inference, a weighted Boxes Fusion method was used to select final bounding boxes, aiming to optimize the “filtered F1” metric. This method achieved a mean Intersection over Union (mIoU) of 0.83, with only 225 images removed on the public leaderboard.

Stage 2: Classifier Training with Diverse Architectures

Next, Eric and his team used various model backbones from the Timm library, including EfficientNet, MaxViT, TinyViT, ResNet, MobileNet, and DenseNet, with image sizes from 224 to 512.

Training procedure and hyper-parameters:

📚 Classifier Training Specifics

• The best performing models for cross-validation F1 scores were TinyViT 384 and EfficientNetV2s 512.
• The team maintained consistency with the same 4-fold cross-validation strategy used in Stage 1.
• Over time, the training procedure was refined, gaining significant improvements from techniques like mixup augmentations, Exponential Moving Average (EMA), and label smoothing. Label smoothing, in particular, was effective in mitigating overconfidence due to noisy labels.

❌ What did not work?

In the relentless pursuit of an optimal solution for the MosquitoAlert challenge, not all paths led to success. Increasing image size to 1024x1024 or adjusting aspect ratios to 1.0 did not improve model performance. Single-stage models for classifying six classes, using Yolo and RTDETR models, did not meet expectations. Stochastic Weight Averaging (SWA) and model pruning did not yield gains. Adjusting margins of predicted bounding boxes also did not lead to better results. The team realized focusing on the mosquito’s head, thorax, and dorsal parts was insufficient. Recognizing the value of mosquito legs and wings marked a significant learning point.

Eric and his team’s experience highlights the iterative process of research and development in AI, showing strategic thinking, technical skill, and resilience. Their story is a guide for enthusiasts in AI, promoting continuous learning and exploration.

So, what are your thoughts on Eric’s journey and his approach to the MosquitoAlert challenge? Let us know in the comments, or tweet us @AIcrowdHQ. And remember, whether you’re just starting out or are an experienced hand in AI, there’s always something new to learn and explore in this ever-evolving field!