Journal of Medical Research and Innovation

Assessment of ‘Florence’ in Addressing Inquiries on Nicotine Replacement Therapy

Meer Sadad Billah, Samia Amin, Oishi Barua — Sat, 17 Feb 2024 10:23:56 +0000

Dear editor,

Artificial intelligence (AI) assisted chatbots, or conversational agents are new digital tools that mimic instantaneous human conversation. As AI assistants become more prevalent, evaluating their accuracy and consistency in providing health information is important. Evidence suggests that role of chatbots in smoking cessation is promising particularly in participant’s engagement.¹ The World Health Organization has launched a digital health worker ‘Florence’ (a virtual human), powered by AI as its newest resource for providing the general population with accurate health information on COVID-19 vaccines and treatments, mental health, including smoking cessation.²

Background

Nicotine replacement therapy (NRT) is a widely recommended approach for smoking cessation to manage withdrawal symptoms associated with quitting smoking, such as irritability, cravings, and mood swings.³ It is recommended to use these products under the guidance of a healthcare professional.³ However, it is evident that people often search for information about addiction help-seeking queries from AI assistants.⁴ It is critical to understand the role and reliability of ‘Florence,’ especially in smoking cessation. The objective of this research was to elucidate whether ‘Florence’ provides evidence-based information in response to common NRT questions.

Methodology

A rigorous, methodical process was followed to develop an effective evaluation scale.⁵ The evaluation scale was developed to comprehensively assess the performance of an AI system across 3 parameters. In the first parameter, the AI evaluated on ‘voice recognition’ and ‘question understanding’. Voice recognition is scored on a scale from 0 to 2, where 0 represents a failure to differentiate between male and female voices, 1 indicates inconsistent recognition, and 2 signifies reliable recognition. Similarly, ‘question understanding’ is assessed on the same scale, with 0 denoting a lack of understanding, 1 representing inconsistent understanding, and 2 indicating consistent comprehension of questions. The second parameter was ‘consistency in answers between researchers’, the AI's performance is measured by answer consistency. Scores range from 0 to 2, where 0 signifies completely different answers between researchers, 1 suggests somewhat different answers, and 2 denotes identical answers. The third parameter, ‘accuracy of answers’, evaluated the AI's precision in providing correct responses. The scale ranges from 0 to 2, with 0 indicating completely inaccurate answers, 1 representing somewhat accurate answers with significant errors, and 2 signifying entirely accurate responses. The overall assessment is derived from the total score, where a cumulative score of 0-2 indicates ‘poor’ performance, 3-4 reflects ‘fair’ performance, 5-6 signifies ‘good’ performance, and 7-8 represents ‘excellent’ performance.

The scoring guidance was provided to support consistency across evaluators. We pilot tested this matrix before main data collection by collecting 20 questions from ‘Quora’ platform related to smoking cessation. Finally, the team had then read through the ACS FAQ webpage responses to evaluate consistency and accuracy and compare them with the responses from the ‘Florence.’

Fifty-six NRT questions were obtained from the American Cancer Society website.⁶ Two researchers independently queried ‘Florence’ and recorded the responses over a two-week period in January 2024. Responses were compared to the American Cancer Society answers to evaluate accuracy and between researchers to assess consistency. An 8-point rating scale was used across 3 evaluation parametres: voice recognition, question understanding, answer consistency, and accuracy.

Results

Out of 56 NRT questions asked, 11 questions (19.6%) were answered with excellent accuracy and depth of knowledge, demonstrating a strong command of the topics covered. Total 44 questions (78.6%) were rated as fair performance. Responses to these questions had some minor flaws in accuracy, comprehensiveness of information, or depth of explanation. There is room for improvement to address gaps in knowledge. Only 1 question (1.8%) received a poor performance rating. Approximately one-fifth of responses met excellence criteria, over three-fourths still have space for improving quality in content, detail, precision, or accuracy.

The results indicate a mixed performance of 'Florence' in addressing NRT-related queries. The identified gaps in knowledge, as evidenced by the ‘fair’ performance ratings, underscore the need for continuous improvement in the AI system. Addressing these gaps could enhance the quality of content, detail, precision, and overall accuracy of responses. As the field of AI-assisted chatbots in health information provision evolves, ongoing evaluations and refinements are essential to ensure these tools meet the highest standards in accuracy and reliability. This research contributes valuable insights that can guide future enhancements in AI-assisted health information tools, ultimately benefiting individuals seeking reliable guidance in their health journeys.

This study's strengths lie in its comprehensive evaluation methodology, real-world application, and actionable insights for improvement. Yet, limitations such as potential biases in comparison sources and subjectivity in evaluations emphasize the need for careful consideration in interpreting 'Florence's' performance.

Conclusion

In summary, while ‘Florence’ excelled in linguistic processing like speech and question comprehension, supplemental training focused on strengthening NRT knowledge itself would help address shortcomings in consistency, precision, completeness, and depth when answering domain-specific questions. Targeted improvement tuning both language mastery and core subject matter competencies could boost overall performance from fair to excellent across evaluations.

Artificial Intelligence in Medicine: Revolutionizing Healthcare for Improved Patient Outcomes

Varshil Mehta — Sat, 03 Jun 2023 00:00:00 +0000

Introduction: Artificial intelligence (AI) has emerged as a groundbreaking technology with the potential to transform various sectors, and the field of medicine is no exception. With its ability to process vast amounts of data and perform complex tasks, AI has begun to revolutionize healthcare, offering promising avenues for diagnosis, treatment, and patient care. In this editorial article, we will explore the significant impact of AI in medicine, highlighting its potential benefits and the challenges that lie ahead.

AI-Driven Diagnosis

One of the most remarkable applications of AI in medicine is its capacity to assist in accurate and efficient diagnosis. By leveraging machine learning algorithms, AI systems can analyze medical imaging, such as X-rays, MRIs, and CT scans, with a level of precision that rivals human experts. Studies have demonstrated the effectiveness of AI in detecting various conditions, including lung cancer, cardiovascular diseases, and neurological disorders, leading to earlier and more accurate diagnoses.

For instance, a study published in Nature Medicine by McKinney et al. revealed that an AI model trained on a large dataset of mammograms outperformed radiologists in breast cancer detection. The AI system achieved a lower false-negative rate and reduced the number of false positives, thereby potentially reducing unnecessary biopsies [1]. Similarly, a study by Esteva et al., showed that a deep learning algorithm outperformed dermatologists in diagnosing skin cancer based on images [2]. Such advancements in AI-driven diagnosis hold immense promise for improving patient outcomes and reducing healthcare costs.

Personalized Treatment and Precision Medicine

AI has also opened doors to personalized treatment strategies, enabling healthcare professionals to tailor therapies to individual patients. By analyzing vast amounts of patient data, including genetic information, medical history, and treatment outcomes, AI algorithms can identify patterns, predict responses to specific treatments, and recommend personalized interventions. This approach, known as precision medicine, has the potential to revolutionize disease management.

An example of AI's impact on precision medicine is showcased in the work of Poplin et al. The study demonstrated how a deep learning algorithm could predict the onset of cardiovascular events by analyzing electronic health records. The algorithm outperformed traditional risk models by incorporating a broader range of patient data, allowing for more accurate and timely interventions to prevent adverse events [3]. Similarly, Obermeyer et al., demonstrated that an AI model outperformed traditional methods in predicting acute kidney injury in hospitalized patients [4] while a study by Che et al., demonstrated the effectiveness of an AI model in predicting sepsis, allowing for early intervention and improved patient outcomes [5].

Enhanced Clinical Decision-Making and Workflow

AI has the capacity to enhance clinical decision-making by assisting healthcare providers in analyzing complex data and generating evidence-based recommendations. AI systems can process and interpret vast amounts of medical literature, patient records, and clinical guidelines, providing healthcare professionals with timely insights and decision support. This augmentation of human expertise can lead to more accurate diagnoses, improved treatment plans, and enhanced patient care.

A notable example is the work of Rajkomar et al., published in The New England Journal of Medicine. The authors developed an AI algorithm capable of predicting patient deterioration within the next few hours, based on electronic health record data. By alerting healthcare providers in advance, this AI system helped to prevent adverse events and facilitated proactive interventions [6].

Drug Discovery and Clinical Research

The drug discovery and development process is notoriously expensive and time-consuming. AI has the potential to accelerate this process by analyzing vast amounts of biomedical literature, genomic data, and clinical trial outcomes. Machine learning models can identify potential drug targets, predict drug toxicity, and optimize drug formulations. In fact, a study by Aliper et al., demonstrated that an AI system outperformed human researchers in designing new drugs to target age-related diseases [7].

Virtual Assistants and Telemedicine

AI-powered virtual assistants and chatbots are transforming the way patients interact with healthcare providers. These virtual assistants can provide instant medical advice, answer queries, and triage patients based on their symptoms. Furthermore, telemedicine platforms integrated with AI algorithms can enhance remote patient monitoring, enabling healthcare professionals to monitor patients' vital signs and provide timely interventions [8,9].

Challenges and Ethical Considerations

While the potential benefits of AI in medicine are substantial, it is important to address the challenges and ethical considerations associated with its implementation. Privacy and data security remain critical concerns when handling vast amounts of patient data. Maintaining patient confidentiality and ensuring secure data sharing frameworks must be prioritized to protect patient privacy.

Moreover, the need for transparency and interpretability of AI algorithms is vital to build trust between healthcare professionals and AI systems. Understanding how AI arrives at its recommendations or diagnoses is crucial for healthcare providers to make informed decisions and ensure accountability.

Conclusion: Artificial intelligence holds tremendous potential to revolutionize healthcare and improve patient outcomes. From enhancing diagnostic accuracy to enabling personalized treatment strategies and augmenting clinical decision-making, AI is transforming the field of medicine. However, to fully realize the benefits, it is essential to address the challenges surrounding privacy, data security, and algorithm transparency. By leveraging the power of AI responsibly, healthcare providers can usher in a new era of precision medicine, advancing the quality and effectiveness of patient care.

Low back pain oswestry disability index changes following 8-week movement proficiency exercise program – A retrospective cohort study

Brogan Samuel Williams, David Johnson — Tue, 30 May 2023 00:00:00 +0000

Chronic low back pain (CLBP) is a worldwide epidemic, with a prevalence rate of 75–84% in developed countries. With the prevalence increasing, health-care professionals must question current best practice guidelines. In 2014, spinal neurosurgeon and back pain rehabilitation specialist Dr. David Johnson developed a unique back pain rehabilitation program referred to as NearoHAB®. The program’s uniqueness is founded on the principle that effective rehabilitation must eliminate the root cause of pain symptoms. The NeuroHAB® 8-week Movement therapy program aims to reverse movement dysfunction by restoring central nervous system-derived motor patterns based on proficient spinopelvic biomechanics for bending activities of daily living. To date, no other rehabilitation methodology adopts a movement dysfunction cause-based clinical model for back pain symptoms or includes a framework for what healthy lumbar pelvic movement should resemble. Over the course of the 8-week program, each participant is gradually upskilled, developing new default movement proficiency and improved biomechanics, in efforts to downregulate pain, improve disability, and increase functional movement capacity, creating a positive feedback loop for further progress. The leading question of this study is “How does functional movement-based therapy impact chronic low back pain?” Ten sets of participant details were selected at random and retrieved from the NeuroHAB® 8-week program database of 2020. All participants presented with CLBP, and two oswestry disability index (ODI) scores were documented – the first at the beginning of the 8-week program, and the second after the NeuroHAB® intervention. ODI scores were collated and the pre- and post-program results were measured and compared quantitatively through a paired t-test to determine the statistical significance of improvement. Results showed a two-tailed P=0.05 indicating that there was a significant difference between the pre- and post-data (0.0024). The pre- and post-group intervention ODI means were 25.80 and 13.30, respectively, resulting in a difference of 12.50 (95% CI: 5.73–19.27); determining the mean data between the pre- and post-intervention decreased by 48.4496%. The results from this study support the alternative hypothesis, concluding an 8-week intervention of functional movement therapy represented by NeuroHAB® results in a significant reduction of LBP ODI scores.

Outcome of the first health skills simulation laboratory in the kingdom of Saudi Arabia

Mohammad Ibrahim Fatani, Abdulmajeed Khan, Ammar Albokhari — Mon, 12 Dec 2022 00:00:00 +0000

Simulation advanced medical education and medical personnel evaluation across the world.^[^1] For improving the healthcare skills of medical staff and clinical performance of health-care practitioners, the Ministry of Health issued a budget of three million Saudi Riyals to establish the first clinical skills and simulation center in the Kingdom of Saudi Arabia, in Makkah, at the beginning of 2003. The objectives were to establish a state-of-the-art simulation laboratory to improve the clinical performance of health-care practitioners, facilitate consistent formal clinical training, overcome the difficulties encountered by practitioners in actual practice in either the pre- or post-graduate period, and provide jobs through specialized training and education.

The first phase of the project consisted of searching for a location, establishing a structured 3-year plan, and convincing stakeholders of the concept of clinical training. The second phase included preparation of the venue, building up of human resources, interior designing of showrooms, and documentation of each single action. The third and final phase consisted of marketing and advertising, official accreditation of short- and long-term courses, and postgraduate medical professional training. Each phase required an entire year of planning to complete from 2002 to 2004 [Figure 1].

Types of courses conducted at the skill laboratory since 2004: Anatomy and Physiology, Basic Cardiac Life Support (BCLS) and Advanced Cardiac Life Support, Pediatric Advanced Life Support and Neonatal Resuscitation Program, Advanced Trauma Life Support, Fundamental Critical Care Support, Clinical Nursing Skills, Difficult Intubation Course, Peripheral and Central IV Course, Gyn and PV Examination, Normal Deliveries and Its Complications, Diabetic Foot Care, Cardiac Catheterization, Middle Ear Diseases, Arthroscopy, Public Health Education, Upper and Lower GI Endoscopies, Bronchoscopy, and Endoscopic Retrograde Cholangiopancreatography [Figure 2].

As of 2022, the center is under the administration of Makkah Healthcare Cluster and named The Simulation Center of Makkah Healthcare Cluster, and it is affiliated with Hera General Hospital, Al Noor Specialist Hospital, Maternity and Children Hospital, and King Abdullah Medical City Simulation Centers. Each of the simulation centers has a vast majority of stations, with specialized stations related to the availability of medical specialty, for instance, Gyn/Obstetric-related stations at Hera General Hospital and Maternal and Children Hospital, Cardiac Catheterization station at Al Noor Specialist Hospital and King Abdullah Medical City, and Oncology-related stations at King Abdullah Medical City. The BCLS training centers at the institutions of Makkah Healthcare Cluster are supervised by The Simulation Center of Makkah Healthcare Cluster.

Figure 1. First purchase of the equipment in year 2002.

Figure 2. Types of courses conducted at the skill laboratory since 2004.

The Clinical Skills and Simulation Center at the Ministry of Health of Saudi Arabia has adopted various programs and specialized courses for its healthcare practitioners.^[2] The vision of The Simulation Center of Makkah Healthcare Cluster is to render every simulation station accessible to its employees, with an interactive website to help provide the best experience in browsing through the courses that will be of immense support to the staff ’s field of expertise and to upgrade their knowledge and skills.

Authors contributions

Study conception, AK and MIF; manuscript review and editing, AK and MIF; manuscript writing, AAA.

The kaleidoscope of autoimmunity: A report of 10 cases of multiple autoimmune syndrome

Sharang Gupta, Shivali Aggarwal, Dimple Chopra — Mon, 12 Dec 2022 00:00:00 +0000

Dear Sir,

Multiple autoimmune syndrome has been defined as the coexistence of at least three autoimmune disorders in a single patient.^[1] Dermatological autoimmune disorders, especially vitiligo, take a special seat in the setting of multiple autoimmune syndrome.^[2] There is strong evidence to suggest that the various autoimmune disorders have a common origin and share similar pathophysiological mechanisms. Therefore, the presence of one autoimmune disorder in an individual makes him/her susceptible to the development of other autoimmune disorders.^[^3] Herein, we report 10 cases of multiple autoimmune syndrome that presented to the dermatology department of our hospital.

The patients’ details are shown in [Table 1]. Females made up majority of the cases in our series with seven patients while there were only three male patients. The age of the patients ranged from 19 to 60 years. Seven patients had three autoimmune disorders, two patients had four, and one patient had five coexisting autoimmune disorders. Autoimmune thyroid disease was the most common disorder in our case series which was present in seven patients. The diagnoses of vitiligo, psoriasis, alopecia areata, and lichen planus were confirmed with clinical examination and dermoscopy along with histopathological examination. In addition to these, specific antibody testing was carried out for systemic sclerosis, systemic lupus erythematosus, pemphigus vulgaris, and bullous pemphigoid. Triiodothyronine, tetraiodothyronine, thyroid-stimulating hormone, and antithyroid peroxidase antibody levels were carried out for autoimmune thyroid disease while fasting blood glucose, serum insulin levels, and serum C-peptide levels were measured for type 1 diabetes mellitus. The diagnosis of rheumatoid arthritis was based on clinical, radiological, and immunological testing.

Disorders with an autoimmune pathogenesis occur with increased frequency in patients already suffering from another autoimmune disease.^[^4] The term polyautoimmunity has been proposed for this association of autoimmune disorders. This term also encompasses the probable common origin of these disorders.^[^5] Multiple autoimmune syndrome showcases the kaleidoscope of autoimmunity with at least three autoimmune disorders being present in a single patient. The presence of one autoimmune disease should alert one to watch for presence or subsequent development of another one.

To sum up, this study indicates that autoimmune hypothyroidism, type 1 diabetes mellitus, and rheumatoid arthritis are the most common systemic associations of the autoimmune dermatological disorders. Hence, a patient presenting with features of skin autoimmunity must be investigated for these conditions.

Declaration of patient consent

Patient’s consent not required as patient’s identity is not disclosed or compromised.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.