Engineering Trust: Inside Project Diabot's Approach to Safe, Fair Diabetes AI

Authors: Asil Ercan (1), Bilge Pinar (2), Charlotte Lieu (3), Kensley Lovius(4), Nergis Teke (5), Olivia Torcedo (6), Ruhan Shamim (7), and Sinem Yilmaz (8)

Mentors: Fatma Zehra Tunca and Gulsima Karsli

Author Affiliations: (2) Gainesville High School , (3) Meridian High School, (4) Parsippany Hills High School, (5) Central Bucks High School West, (6) Riverside High School, (7) Battlefield Highschool, (8) State College Area High School

Keywords

healthcare chatbots, artificial intelligence, diabetes management, algorithmic bias, patient trust, medical ethics, data privacy, human oversight

Project Diabot is a diabetes-support chatbot built to give patients reliable, everyday guidance on managing their condition. But putting an AI system between a patient and their health information only works if that system can be trusted. Here is how the team behind Diabot is approaching trust, evidence, fairness, safety, and the road ahead.

Why Trust and Evidence Matter

As artificial intelligence becomes part of everyday healthcare, earning genuine trust is essential, since these tools directly shape decisions about diagnosis, treatment, and patient care. Trust describes a person's willingness to rely on a system, while trustworthiness refers to whether that system actually has the qualities, accuracy, reliability, transparency, and fairness, needed to earn that reliance (Asan et al., 2020). Transparency matters most: when AI provides medical information without a clear explanation, patients and clinicians are left questioning whether the result even applies to their situation. When a widely used U.S. healthcare algorithm underestimated the needs of Black patients because it measured spending rather than actual health, it showed how even advanced AI can produce harmful outcomes when people cannot verify how its decisions are made. Without that trust, patients may hesitate to use AI health tools, and clinicians may be reluctant to bring them into practice. Trustworthy AI does not replace clinicians; it supports their judgment while giving patients confidence in the information they receive.

That confidence starts with evidence. Diabot's responses are grounded in guidance from the World Health Organization, the CDC, and the National Institutes of Health, whose recommendations reflect expert review and current scientific consensus, and digital health tools should be designed to maintain safety, effectiveness, and quality (World Health Organization, 2021). Training on peer-reviewed research matters because independent experts examine the methods, data, and conclusions behind a study before publication, catching errors and improving reliability, so a chatbot trained this way is more likely to explain diseases and treatments accurately than one trained on unreviewed sources (National Academies of Sciences, Engineering, and Medicine, 2015). These matters given the risk of AI "hallucination," when a general-purpose model, rather than a purpose-built healthcare chatbot, generates a confident-sounding answer with no real evidence behind it (Hatem et al., 2023). Because healthcare decisions directly affect patient outcomes, anchoring Diabot in vetted sources rather than open-ended AI keeps its guidance dependable.

Building and Testing Diabot's Knowledge

Diabot's reliability also depends on the quality of the Q&A data used to train it. Well-designed question-and-answer sets, drawn from scientifically factual sources, give the chatbot a dependable baseline for handling more complex questions and make information easier for patients to understand. The first mark of a high-quality dataset is domain specificity: information tailored closely to the research it is drawn from. More advanced chatbots are also trained to handle variations in vernacular between different users, so the same underlying answer can reach people who ask about it in different ways, and a larger volume of Q&A pairs generally improves both reliability and accuracy. Diabot's researchers wrote multiple versions of the same question in different phrasing and prioritized the topics people ask about most. From there, raw datasets move through a production pipeline: foundation data first passes through Supervised Fine-Tuning (SFT), which generalizes the domain, and then optionally through Reinforcement Fine-Tuning and Model Distillation to sharpen response quality before deployment (Figure 1), though for smaller, domain-specific bots SFT alone is sometimes enough. Every dataset was also built to reinforce one rule: patients should consult a healthcare professional before making any serious decision or adjusting a dosage.

Figure 1. Diabot's Q&A datasets move through Supervised Fine-Tuning and, optionally, Reinforcement Fine-Tuning and Model Distillation before deployment. Credit: Original illustration, Project Diabot Research Team.

Building good data is only half the work; Diabot must also be tested against it. Researchers typically compare chatbot answers on topics like glucose monitoring, medication, diet, exercise, and diabetes complications against established clinical guidelines, with healthcare professionals judging factual accuracy, completeness, and clinical appropriateness. A systematic review by Wu et al. (2024) organized this kind of evaluation into three categories, technical performance, user experience, and health outcomes, and found across 25 studies that chatbot interventions significantly improved blood glucose control, though they showed no significant effect on patient weight. Other researchers assess chatbots using detailed technical metrics: Abd-Alrazaq et al. (2020) catalogued 27 measures covering usability, response speed, comprehensibility, and how accurately a bot interprets user input, including metrics like word error rate and concept error rate. More recently, Hua et al. (2025) introduced HAICEF, a 271-question framework developed with input from clinicians, patients, developers, epidemiologists, and policymakers, spanning safety and fairness, trustworthiness and usefulness, and design and operational effectiveness. No single evaluation method is sufficient alone, so researchers combine several complementary approaches to confirm a chatbot is safe and reliable before it reaches patients.

Fairness, Safety, and Human Oversight

Even accurate data can carry bias. AI bias occurs when a system produces systematically skewed outcomes because of flaws in its training data or algorithms, and in healthcare this can mean unfairly favoring or harming particular patient groups based on race, gender, or socioeconomic status. One widely cited example is a U.S. algorithm that favored healthier white patients over sicker Black patients because it was trained to predict healthcare costs rather than actual health needs, meaning less money was ultimately spent treating Black patients with the same level of illness (Obermeyer et al., 2019). This kind of bias traces back to people, not just code: historical inequalities in healthcare access, biased clinical judgments, and insufficiently diverse training data all feed into a model's blind spots. Addressing it takes proactive steps: diversifying training data across demographics, building in bias-detection tools such as fairness metrics with input from sociologists and ethicists, and monitoring continuously so problems are caught as they emerge. Any decision carrying serious ethical or legal weight should still stay in human hands.

Fairness has to be paired with privacy and transparency. Because medical data is legally protected and deeply personal, chatbots handling sensitive health metrics must use advanced encryption standards to guard against unauthorized access, particularly as the rapid expansion of medical big data introduces real privacy vulnerabilities alongside its clinical potential (Price & Cohen, 2019). Patients also have an explicit right to know whether they are speaking with an automated system or a licensed provider, and designers must disclose, in general terms, what data sources inform a response, since the complexity of advanced algorithms can otherwise obscure who is accountable for a given recommendation (Mittelstadt et al., 2016). Just as important, a chatbot must recognize its own limits: it should be restricted from issuing a definitive diagnosis or altering a prescription, since unauthorized medical advice can lead to dangerous self-management decisions, and it should route urgent symptoms straight to a human professional or emergency care whenever critical symptoms are detected (Blease et al., 2020). Upholding these standards keeps a healthcare chatbot a medical resource rather than a digital liability.

That handoff to a human matters because AI still cannot replicate clinical intuition. A chatbot can recognize patterns and generate a statistically likely answer, but it lacks the lived experience a physician brings to the table. Where AI holds patient care to statistics and proof, a doctor offers empathy, human connection, and judgment shaped by relating to the patient directly. Consider a patient whose clinical tests come back normal, but whose therapist notices signs of depression through body language that data alone would not catch, or a patient with chest pain and fatigue whom an experienced physician, recalling a similar case, correctly flags as an autoimmune condition rather than a cardiac event. A false or overconfident chatbot answer can cause a patient to panic, or just as dangerously, dismiss symptoms that need real attention. Used well, AI becomes a partner rather than a substitute: it can surface useful patterns for a clinician to review, freeing that clinician to focus on the judgment and empathy no algorithm can provide.

Figure 2. Four pillars researchers point to for a trustworthy healthcare chatbot: safety and privacy, fairness and inclusion, transparency, and human oversight. Credit: Original illustration, Project Diabot Research Team.

The Road Ahead

Looking forward, Diabot's developers see room to make the chatbot even more personalized and precise. One promising direction is the use of Personal Health Knowledge Graphs (PHKGs), which structure a patient's biomedical health data into a connected network, linking, for instance, continuous glucose monitor trends to metabolic genomics and pharmacological data drawn from resources like Harvard's PrimeKG (Rastogi & Zaki, 2020; Chandak et al., 2023). Built on knowledge graphs like these, every piece of medical advice Diabot generates could be traced back through a visible, physiological road map that both patients and clinicians can inspect, rather than trusting a black box. Reaching that future also depends on how well the development team works together: research on developer productivity shows that frameworks like Microsoft's SPACE, which weighs satisfaction, performance, and collaboration together rather than raw output, help engineering teams produce high-quality software without sacrificing team chemistry (Forsgren et al., 2021). Combined, richer data structures and healthier development practices point toward a next generation of Diabot that is more precise, more transparent, and more personalized.

Glossary of Terms

Algorithmic bias: systematic distortion in an AI system's outputs, caused by flawed data or design, that unfairly favors or harms particular groups.

Trustworthiness: the qualities, such as accuracy, reliability, transparency, and fairness, that a system must have to earn a user's trust.

Peer-reviewed research: studies that have been evaluated by independent experts before publication to catch errors and confirm reliability.

Hallucination (AI): when a model generates a confident-sounding response that is not actually supported by real evidence.

Supervised Fine-Tuning (SFT): the training step that turns raw Q&A data into a generalized knowledge base a chatbot can draw on.

Personal Health Knowledge Graph (PHKG): a structured, connected map linking a patient's health data, such as glucose trends, to broader medical and genetic information.

HAICEF: the Health Care AI Chatbot Evaluation Framework, a 271-question tool for assessing chatbot safety, trustworthiness, and effectiveness.

Algorithmic transparency: the practice of clearly disclosing what data and logic underlie an AI system's responses.

Human oversight: keeping trained professionals responsible for final decisions that AI can inform but should not make alone.

SPACE framework: a model for measuring developer productivity across satisfaction, performance, and collaboration rather than output speed alone.

References

Abd-Alrazaq, A., Safi, Z., Alajlani, M., Warren, J., Househ, M., & Denecke, K. (2020). Technical metrics used to evaluate health care chatbots: Scoping review. Journal of Medical Internet Research, 22(6), e18301. https://www.jmir.org/2020/6/e18301/

An, T. (2025, September 25). The complete guide to training LLMs with QA data: From zero to production. Medium. https://tao-hpu.medium.com/the-complete-guide-to-training-llms-with-qa-data-from-zero-to-production-b48a68f1cdb9

Asan, O., Bayrak, A. E., & Choudhury, A. (2020). Artificial intelligence and human trust in healthcare: Focus on clinicians. Journal of Medical Internet Research, 22(6), e15154. https://doi.org/10.2196/15154

Bias in AI. (n.d.). Chapman University. https://www.chapman.edu/ai/bias-in-ai.aspx

Bias in medical AI: Algorithmic fairness and ethics challenges. (2026, January 31). Journal of Young Investigators. https://www.jyi.org/2026-january-1/2026/1/8/bias-in-medical-ai-algorithmic-fairness-and-ethics-challenges

Blease, C., Locher, C., & Torous, J. (2020). Ethical challenges of digital mental health and chatbots. The Lancet Digital Health, 2(11), e559–e560. https://www.thelancet.com/journals/landig/article/PIIS2589-7500(20)30237-5/fulltext

Chandak, P., Huang, K., & Zitnik, M. (2023). Building a knowledge graph to enable precision medicine. Scientific Data, 10(1), 1–15. https://doi.org/10.1038/s41597-023-01960-3

Confronting the mirror: Reflecting on our biases through AI in health care. (2024, September 24). Harvard Medical School. https://learn.hms.harvard.edu/insights/all-insights/confronting-mirror-reflecting-our-biases-through-ai-health-care

Evidence based medicine: What it is and what it isn't. (1999). BMJ, 318(7179), 318. https://doi.org/10.1136/bmj.318.7179.318

Forsgren, N., Storey, M.-A., Maddila, C., Zimmermann, T., Houck, B., & Butler, J. (2021). The SPACE of developer productivity: There's more to it than you think. ACM Queue, 19(1), 20–48. https://doi.org/10.1145/3454122.3454124

The growing use of artificial intelligence in health care and implications for disparities. (2026, April 30). KFF. https://www.kff.org/racial-equity-and-health-policy/the-growing-use-of-artificial-intelligence-in-health-care-and-implications-for-disparities/

Hatem, R., Simmons, B., & Thornton, J. E. (2023). A call to address AI "hallucinations" and how healthcare professionals can mitigate their risks. Cureus, 15(9), e44720. https://doi.org/10.7759/cureus.44720

HCI News Medical Assisting Nursing Practical Nursing. (2025). The role of human judgment in healthcare: What AI can't do. HCI College. https://www.hci.edu/hci-news/33180-the-role-of-human-judgment-in-healthcare-what-ai-cant-do

Hua, Y., Xia, W., Bates, D., et al. (2025). Standardizing and scaffolding health care AI-chatbot evaluation: Systematic review. JMIR AI, 4, e69006. https://www.jmir.org/2025/1/e69006/

Kaltenbrunner, S. (2026). Human in control: Shared decision-making with clinical decision-support systems under the Artificial Intelligence Act. Computer Law & Security Review, 61, 106281. https://doi.org/10.1016/j.clsr.2026.106281

Marketing Team. (2025, January 28). Building a high-impact FAQ dataset for chatbot. Macgence AI. https://macgence.com/blog/faq-dataset-for-chatbot/

Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016). The ethics of algorithms: Mapping the debate. Big Data & Society, 3(2), 1–21. https://journals.sagepub.com/doi/10.1177/2053951716679679

National Academies of Sciences, Engineering, and Medicine. (2015). Improving diagnosis in health care. The National Academies Press. https://doi.org/10.17226/21794

Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447–453. https://doi.org/10.1126/science.aax2342

Price, W. N., & Cohen, I. G. (2019). Privacy in the age of medical big data. Nature Medicine, 25(1), 37–43. https://www.nature.com/articles/s41591-018-0272-7

Rastogi, N., & Zaki, M. J. (2020). Personal health knowledge graphs for patients. arXiv. https://arxiv.org/abs/2004.00071

Trust, trustworthiness, and the future of medical AI. (2025). Journal of Medical Internet Research, 27, e71236. https://doi.org/10.2196/71236

World Health Organization. (2021). Ethics and governance of artificial intelligence for health. World Health Organization. https://www.who.int/publications/i/item/9789240029200

Wu, Y., Zhang, J., Ge, P., et al. (2024). Application of chatbots to help patients self-manage diabetes: Systematic review and meta-analysis. Journal of Medical Internet Research, 26, e60380. https://www.jmir.org/2024/1/e60380/