Low Literacy Chatbot
My internship project at the Albert Schweitzer hospital in my 6th semester. Our team did research and made a prototype for a medical chatbot adjusted for low literacy, aimed at exploring a solution to the problem of treatment leaflets which are almost never read.
Before or after a treatment a patient gets a leaflet in their online patient dossier, where important information about preperations or complications are written down. It's been found that only 2-5% of patients actually read these leaflets online. Logging in to your online patient dossier with DigID, navigating to the leaflet and then having to attentively read it was understandably too big of a hassle for many patients. Another part of the project revolved around low literacy, because the area of Dordrecht (where the hospital is located) experiences a high percentage of this (around 20% of the population). We we're tasked with researching if Large Language Models could be used to help these patients by offering a more approachable option to acces the information inside of the leaflets.
My role within the team was the 'software developer', for which I held myself busy with doing research on the technical side of this project, programming a custom made chatbot and (with the help of another teammate) setting up and integrating the final prototype.
📌 Project specifications
- Goal:
-
Research and prototype an alternative digital solution to provide public healthcare information from the Albert Schweitzer Hospital (ASz) in an accessible way for low-literate patients.
- Scope:
-
- Our focus was specifically on treatment leaflets from the hospital, as research indicated that these documents were rarely accessed (as detailed in the introduction), thereby streamlining our project.
- Privacy, security, and the legal aspects of medical chatbots were beyond the project's scope. Our primary objective was to investigate the feasibility of leveraging AI for enhancing accessibility to healthcare information.
- Timeline:
-
15 Weeks
🖥️ Technical Specifications
- Technologies, tools and frameworks used (Custom chatbot):
-
- Python with the langchain Library
- Chroma vector database
- Open-source LLM and embeddingsmodel from Huggingface
- Technologies, tools and frameworks used (Final chatbot):
-
- Voiceflow
- ManyChat
- Data:
-
The 2 medical leaflets used for testing the final prototype where one about a broken upper arm and one about an after cataract laser operation from the ASz website.
🎯 Results
The project was successful in achieving its primary goal of researching, prototyping, and testing a potential AI solution for low-literate patients. We completed a comprehensive research project covering various aspects of the problem, thanks to our collaborative efforts as a team.
My role focused on developing a custom chatbot, which, while not perfect, provided valuable insights. Testing this chatbot allowed us to draw conclusions that informed our technology recommendations for the final prototype, a Whatsapp chatbot powered by Voiceflow. This final prototype was successfully tested with Stichting ABC, an organization for and by low-literate individuals. Additionally, we presented it to the client panel and employees from different hospital sections, receiving positive feedback.
Image 1: Testing the custom chatbot.
Image 2: Testing a Voiceflow chatbot prototype with Stichting ABC.
Video: Testing the final Whatsapp chatbot prototype, which we showed in our final presentations.
Our project was not intended to implement a final product but to explore different solutions, prototype the most promising one, and test its feasibility. We succeeded in demonstrating that an AI solution could be effective, sparking interest and showing its potential to the hospital staff. Further research, testing, and development are necessary for any future rollout, but we laid a strong foundation for continued exploration.
💭 Reflection
Reflecting on the project, balancing school requirements with project tasks presented challenges, but overall, it was a fulfilling experience where I learned a lot. The initial misalignment between the project's goals and the practice supervisor's expectations required finding a balance between development work and research. This overlap often made it hard to demonstrate progress in both areas.
During the first two sprints, I focused on supporting the team to establish a clear direction. By the third sprint, I was able to develop and test the custom chatbot application effectively, even though it coincided with interim assessments. This period was intense but rewarding.
Despite these challenges, the project was highly educational and meaningful. Working on an innovative solution for a vulnerable group was deeply satisfying. The support from my instructor, practice supervisor, and teammates was invaluable, allowing me to navigate the complexities and achieve successful outcomes.