...
For example, we could say success is measured by the number of interactions a user has with a chatbot, i.e. the number of messages sent back and forth in a single session. Or we could say its the number of client scenarios the CHW works through. Or we could say its the change in how the CHW responds to scenarios as she works through the chatbot- for example, how different are her responses to the third scenario she tackles as compared to the first? If we were doing a study to assess the impact of a chatbot, we might want to conduct a pre- and post-survey to measure changes in knowledge, attitudes and behaviours. In the future, we could also measure success in terms of cost-effectiveness of the chatbot in achieving certain goals.
It's okay to not have all the answers - at Dimagi, we're continuing to work through these questions ourselves, and we're learning more with each new chatbot we make and deploy. But we recommend thinking about this a bit while you create your chatbot prompt.
...