Market-approved convolutional neural network tasked with classifying skin lesions under suspicion of melanoma: Performance across primary care clinics within Australia

Background. Artificial intelligence (AI) is poised to revolutionise how melanoma is detected in clinical practice, yet few studies have been published with patient data at the forefront. Objective. The primary aim of this study was to investigate the clinical performance of a market-approved convolutional neural network (CNN) to better differentiate skin lesions suspicious of being malignant melanoma (MM). A secondary aim of this study was to compare the diagnostic performance of the CNN across two separate general practices, that are skin cancer focused clinics. Methods. Multicentre, cross-sectional study using a commercially available CNN on 373 melanocytic lesions (114 melanoma, 259 non-melanoma) from participants attending a skin examination within two Australian specialised, general practice clinics. Performance metrics included sensitivity, specificity, predictive values, diagnostic odds ratios, accuracy and area under the curve (AUC) of receiver operating characteristics (ROC) used for classification of images. Results. The CNN average sensitivity [Gold Coast vs Townsville] was calculated as 63.2% [61.5% vs 68.6%], specificity as 53.9% [52.5% vs 55.1%], positive predictive value as 37.8% [28.9% vs 44.0%] and negative predictive value as 76.8% [71.4% vs 84.2%]. Likelihood ratios were 1.4 for positive likelihood ratio, 0.7 for negative likelihood ratio and a diagnostic odds ratio of 2.0 across both clinics. Accuracy was calculated as 56.6% [56.1% vs 57.5%] and the AUC of ROC for both clinics was 0.602 and 0.615 for Townsville and Gold Coast, respectively. Conclusions. Improvement of the performance of this CNN for the classification of images, particularly when suspectingMMis necessary before it may be used in a clinical setting in Australia. Other validated AI systems used internationally may also require review for use in an Australian setting.