Safety, Language and Literacy

This page provides context on how to incorporate safety, language and literacy instructions in your prompt. 

Safety and Accuracy of LLM-based Chatbots

There is huge potential for using LLMs to directly serve low-income populations, and to increase the equity of chatbots in global health. Just to name a few opportunities: there is opportunity to address the tremendous unmet need in mental health and psychosocial support, the potential to increase cognitive development of children by a full standard deviation by teaching caregiver’s responsive parenting techniques as part of Early Childhood Development, and to prevent infant mortality through increased uptake of exclusive breastfeeding and skin-to-skin care for pre-term and low birthweight babies.

Despite the potential of LLMs, they have limitations, such as producing nonsensical responses, struggling with complex questions, and exhibiting biases. There are also safety concerns with deploying LLMs without additional safeguards. For example, what if a chatbot user thought they were talking to a real human and asked highly sensitive or urgent questions about mental or sexual health, regardless of the focus area of the bot? In many cases, in such a scenario, we would want to ensure that a chatbot: 1) does not deviate from its purpose and 2) still empathetically responds to the user to re-direct them. 

Here are examples of instructions that Dimagi has incorporated into multiple chatbot prompts. 

These can be taken as suggestions to incorporate into your larger chatbot prompt- and it is always good to test and try and break your bot- for example, using Prompt Builder- even once you've included these. 

Language and Literacy of LLM-based Chatbots

When creating chatbots, it is useful to consider both:  1) the languages your users are most familiar with and 2) how proficient the LLM is in those languages. For example, Dimagi's internal tests have shown that OpenAI's LLMs such as GPT-4 appear to function very well in Hindi but are less conversant in Malawian Chichewa. 

Dimagi has developed and tested various tactics to improve the performance of LLMs (specifically GPT-4) in local languages such as Chichewa. One such tactic is to instruct the chatbot to speak simply - for example, "speak in Chichewa in the manner of a 5th grader." Other tactics include creating a prompt in the local language itself- i.e. writing than providing the chatbot with a full prompt in English, create the prompt in Chichewa instead.

We are continuing to test and identify which of these tactics might work. For now, we've included some example prompt text below for language and literacy.

We encourage users of Open Chat Studio to iterate - using Prompt Builder - to see which tactics result in improved performance- and us know when you do!