Dynamics of the Neural Network Accuracy in the Context of Modernization of the Algorithms of Skin Pathology Recognition

Abstract

Background: The lack of objective methodologies and open datasets for the evaluation of the algorithms complicates the objective evaluation by specialists and hinders the widespread use of this technology in health care. The purpose of this study was to estimate the accuracy of the Skinive’s algorithm 2020 version, then, after an algorithm improvement in 2020-2021, to show a statistically significant decrease in neural network errors in the risk assessment of skin pathologies in 2021.

Methods: Skinive neural network uses a machine-learning algorithm to calculate the risk rating of skin pathologies. For this study, we used Skinive’s algorithm 2020 and 2021 versions trained on 64 000 and 115 000 images respectively. Three validation datasets were used to assess the sensitivity of the algorithm: precancer + cancer, HPV skin pathology, acne, containing 285 images in each set. The specificity has been calculated on a separate validation set containing 6,000 benign neoplasm cases.

Results: The sensitivity of the Skinive neural network in detecting malignant neoplasms was 89.1% and 95.4% in 2020 and 2021 respectively. The specificity of Skinive’s neural network in determining benign neoplasms was 95.3% in 2020 and 97.9% in 2021. For all skin neoplasms: in 2020, the sensitivity was 95.3%, for specificity 93.5%; in 2021, it was 97.9% and 97.1% respectively.

Conclusions: The results of sensitivity and specificity of the Skinive neural network indicate that the algorithm is highly accurate in detecting various neoplasms and skin diseases. After improving the algorithm, we showed a statistically significant decrease in the number of neural network errors in determining the risks of skin pathologies.

Note:
Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. This study was conducted as part of the employment of the authors.

Declaration of Interests: All authors declare no conflicts of interest.

Ethics Approval Statement: All photos of patients' skin in our manuscript with the histological report, are provided to the authors in accordance with the Skinive MD application approbation agreement (the agreement includes consent to the publication of photos an impersonal form). The approbation was carried out in the Republic of Belarus by deramatologists and oncologists under the leadership of the Ministry of Health of the Republic of Belarus (more information: https://skinive.com/skinive-belarus-aprobation/ ). The approbation agreement can be provided upon request. The photos were provided to the authors in an impersonal form, it is impossible to identify the patient from these photos, therefore, additional consent for the publication of these photos from the patients is not required.

Keywords: Neural Network, Artificial Intelligence, Machine Learning, Skin Detection, Skin Diseases

JEL Classification: n/a