Avey: An Accurate AI Algorithm for Self-Diagnosis

Abstract

Medical self-diagnosis algorithms (or symptom checkers) are increasingly becoming an integral part of digital health and our daily lives. In this paper, we present Avey, our Artificial Intelligence (AI) based symptom checker. Alongside, we propose a comprehen- sive experimentation methodology that capitalizes on the standard clinical vignette approach to evaluate symptom checkers.

Based on this methodology, we compiled and peer-reviewed the largest benchmark vignette suite, to our knowledge, in the domain thus far. Afterwards, we defined seven accuracy metrics and leveraged this vignette suite to assess the performance of Avey and five other popular symptom checkers from different angles. Furthermore, we compared Avey’s accuracy against three highly seasoned primary care physicians with an average experience of 16.6 years. Results show that Avey significantly outperforms the five symptom check- ers and compares favourably to the physicians.

Medical self-diagnosis algorithms (or symptom checkers) are in- creasingly becoming an integral part of digital health and our daily lives. In this paper, we present Avey, our Artificial Intelligence (AI) based symptom checker. Alongside, we propose a comprehen- sive experimentation methodology that capitalizes on the standard clinical vignette approach to evaluate symptom checkers.