Evaluation of deep convolutional neural network models for the identification of fly maggot species of forensic significance based on images of posterior spiracles
Of which at the third stage, the external morphology of the larvae is quite similar; thus, the morphological identification used to differentiate its genera or species usually includes the cephalopharyngeal skeleton, anterior spiracle, and posterior spiracles. The morphology of the posterior stigma is one of the important characteristics for identification. A typical morphology of the posterior part of the spiracle of third instar larvae has been shown in Fig. 2. Based on the optical microscope study, the posterior part of the spiracle of M. domestica clearly stood out from the others. On the other hand, the morphology of the posterior spiracle of C. megacephalus and A. rufifacies was quite similar. For C. megacephalus and C. rufifacies, the peritreme, a structure encircling the three spiracular openings (slits), was incomplete and the slits were straight, as shown in Figs. 2A, B, respectively. The complete peritreme encircling three slits was found in L.cuprina and M. domestica as shown in Figs. 2C, D, respectively. However, only the slits of M. domestica were curvy like the letter M (Fig. 2D). Their morphological characteristics found in this study were like the descriptions in previous reports23,24,25.
For model training, four of the CNN models used for species-level identification of maggots provided 100% accuracy rates and 0% loss. The number of parameters (#Params), model speed, model size, macro precision, macro recall, f1 score and support value were also presented in Table 1. The result demonstrated that the AlexNet model offered the best performance for all indicators when compared. among four models. The AlexNet model used the fewest parameters while the Resnet101 model used the most. For model speed, the AlexNet model provided the fastest speed, while the Densenet161 model provided the slowest speed. For model size, the AlexNet model was the smallest, while the Resnet101 model was the largest, which matched the number of parameters used. Macro precision, macro recall, f1 score and support value of all models were the same.
Like the training results presented in Supplementary Data (Fig. S1), all models provided 100% accuracy and 0% loss at the start of training (16. Therefore, we have focused on AlexNet results for the remainder of this article.
Using tSNE visualization, the AlexNet model can explicitly separate species into distinct groups based on features extracted from the model, as shown in Fig. 3. The four species were separated with overlapping data from C. megacephalus and C. rufifacies. This could be due to the similarity of the morphological characteristics of the two species24. This result indicated that the performance of the AlexNet model was equal to human practice.
The five AlexNet hidden convolutional layers on four sample posterior stigmata images were visualized as shown in Fig. 4. Within each layer, the posterior stigma of each species was randomly selected for visualization. Color images have been generated to clearly show the patterns. The multiple hidden layers could extract a rich hierarchy of patterns from the input image, including low-level, mid-level, and high-level patterns that can be observed. The lower layers extracted the detailed local patterns from the images, such as textures, margins and others. The complexity and variation of the visualized patterns increased with respect to the intermediate layers. Top layers extracted specific pattern features.
The classification results (validation and test) for each image are displayed in confusion matrices (Fig. 5) that show how the predicted species (columns) correspond to the actual species (rows). Values along the diagonal indicate the number of correct predictions, while values off the diagonal indicate classification errors. Interestingly, no misclassifications were found after testing the model using the test images (Fig. 5A). Therefore, the results indicated that the predictions of the AlexNet model correspond to the classification of taxonomic experts. The confusion matrix showed misclassification between C. megacephalus and C. rufifacies (Fig. 5B), corresponding to the results of the tSNE visualization. When the model was tested with the externalized images, the classification accuracy for C. megacephalus, C. rufifacies, L. cuprinaand M. domestica was 94.94, 98.02, 98.35, 100%, respectively (Fig. 5B). The results of using the Heatmap program showed that the prediction accuracy of this model was still high (98.70–100%), depending on the image conditions (Fig. 6).
The AlexNet model framework has been demonstrated in Fig. 7. This model consists of 5 convolutional layers (Conv1, Conv2, Conv3, Conv4 and Conv5) and 3 fully connected layers (FC6, FC7 and FC8).19. Convolutional layers extract the features, then fully connected layers combine the features to model the outputs. The example of the features of the images presented in the 5 convolutional layers has been shown in Fig. 4.
Previously, CNNs have been successfully used to identify different cells or species11,12,13,14,15,16,17,18,26,27. This study also confirmed the effectiveness of CNNs in the identification of fly species.
Finally, we created a web application called “The Fly” using our classification model to identify maggot species. The web application is available at https://thefly.ai. Users can identify maggot species by uploading their posterior spiracle images and the result with associated probability percentage will then be displayed. This web application can be viewed and used on desktop and mobile browsers. In terms of limited performance, this web application was designed to identify only four species of fly maggots using images of posterior spiracles. This web application is the first step in the development of the automatic identification of fly species of the order Diptera. More images of these four species and other species should be studied in the future. Moreover, the results of this study will be applied to develop a functionality as a microservice for the identification of maggots in a mobile application called iParasites which is currently available on AppStore and GooglePlay. Nevertheless, we would like to project that taxonomic experts are still important and essential for the development of this AI-based automatic image identification system, as mentioned in a previous report.11.