Getting Started with Messaging
CommCare allows you to schedule SMS messages, emails, or IVR (voice calls) to cases and mobile workers registered in the system.
Common SMS Use Cases
Here are some example usage cases of SMS that can be used in a CommCare App -
Sending a reminder to a health worker when one of their beneficiaries is due for a delivery.
Sending counseling messages to beneficiaries (for example, reminding them to take their iron pills).
Sending a short survey to a beneficiary (for example, asking if they've attended a clinic)
Have a patient request information from the system. For example, if they are enrolled in a smoking cessation study, they may request support messages from the system.
Messaging allows you to set up -
Keywords (so users can send messages and data to the system on their own),
Reminders (scheduled conditional messages to a specific phone number), and
Broadcasts (mass messages or surveys to a group of numbers).
CommCare Messaging Vocabulary
SMS Medium & Pricing
SMS as a Medium
SMS, as a medium, is like email in that information is not encrypted before being sent. This means that you should not include personally identifiable or sensitive information in SMS, just as you would not in an email.
SMS is also not a medium that is 100% reliable. Messages can be dropped due to network issues and often there is no way for a sender to know if the recipient actually received the message. For this reason, SMS should not be used for situations such as reaching out for help in a life-threatening emergency, and other more reliable mediums should be used instead.
With all of this in mind, SMS does make a good medium for use cases such as generic SMS messaging campaigns, appointment reminders, or alerts.
How CommCare HQ Sends Messages
CommCareHQ uses an SMS Gateway to send and receive messages. SMS gateways are phone numbers provided by other companies (or cell phone companies) that connect to the cell phone network to send and receive messages. CommCareHQ is connected to multiple gateways to support different countries.
More information on Dimagi's existing gateways can be found Gateway Options for SMS Projects
Using an Android Gateway
If your country isn't supported by Dimagi's existing gateways you can use an Android-based phone to send and receive messages for your project. This will use the phone's SIM card and SMS balance. Dimagi has partnered with Telerivet to use their system to support Android gateways. The Android phone will connect to CommCareHQ using the internet to know when to send messages and to send incoming messages to CommCareHQ.
Android phones are typically used for low-volume projects since cell phone companies usually limit the number of messages that can be sent or received through a regular phone number.
Sending an SMS directly from a phone
A CommCare phone application can also generate an SMS from a form using the phone's default SMS application and allow the user to send this message directly to the recipient, without requiring a server. This approach gives full control to the mobile user and does not require data connectivity between the user and the server. It can, therefore, be useful in settings where the cell network only supports SMS messages. SMS costs are charged to the user's SIM card and CommCare cannot send these messages automatically, always requiring direct user interaction. Here is a brief description of how to set up a CommCare form to generate an SMS.
Short Code vs. Long Codes
Depending on the gateway used it can be a short code or a long code. Long codes are similar to regular phone numbers. Short codes are special phone numbers that have a specific name (ex. CommCare) or a shorter phone number and can be used to send a very high volume of messages. Short codes also allow for reverse billing.
Reverse Billing
When users send a message to your project's SMS gateway, they normally pay a per message cost - this is exactly like them sending a message to a friend. Since this cost can dissuade users from using your messaging system, some gateways are reverse billed. This means that users can send messages for free to the gateway, and you will pay for those messages instead of the user paying for them.
SMS Pricing
There are two types of costs associated with SMS - a monthly software plan cost for CommCareHQ and a per-message cost.
SMS Pricing Plan: To use SMS, you must be on the Standard or above software plan. The Standard plan allow for outgoing messages (messages from CommCareHQ to a user). To receive messages from users (example, users respond to surveys or send in information), you must be in the Advanced or above pricing plan.
Per Messages Costs:
If using a gateway integrated with CommCareHQ, your CommCare invoices will charge the per-SMS fee charged by the SMS gateway. The charge from the SMS Gateway can fluctuate and also depends on exchange rates as well as the destination number's country and network.
If using a Telerivet gateway, you may have to pay a monthly fee to Telerivet for use of their service based on your subscription, as well as pay for the cost of the phone, it's phone plan, and the cost of messages to and from the phone.
In all cases, your CommCare invoices will charge a USD 0.01 per SMS surcharge paid to Dimagi.
In all cases, large messages can be charged as multiple SMS. Typically, an SMS sent using only the Roman alphabet is charged as one SMS for each 160 characters (roughly), and an SMS sent using special characters is charged as one SMS for each 70 characters (roughly).
New In Country Gateways
For larger high-volume projects in countries we do not already support, Dimagi can support connecting a new gateway to CommCareHQ if your subscription allows it. SMS gateways can be purchased directly from a cell phone company in a country, or from an aggregator that resells phone numbers. Setting up a new gateway requires a significant amount of time and can have high costs. After identifying potential providers, please use the Aggregator Evaluation Worksheet to ensure that key questions are asked and answered during meetings with respective telcos/aggregators. Once you have selected an appropriate gateway provider based on project requirements (cost, setup time, reverse billing), the gateway can be connected to CommCareHQ with assistance from Dimagi's developer team. Please contact information@dimagi.com for more information.
SMS Types & Connectivity
When setting up your CommCare project, it’s important to determine the type of messaging you'll use and select the appropriate SMS connectivity option.
Setting Up Your Project for Messaging - When setting up your project, it's essential to determine the type of messaging you'll use and select the appropriate SMS connectivity.
Type of Messaging: Decide whether your project will use outgoing-only messaging or two-way messaging based on your communication needs:
Outgoing-Only Messaging
Users can only receive messages and cannot respond or send data to the system.
Available on the Standard Plan.
Two-Way Messaging
Users can send data back to the system by responding to surveys or sending keywords.
Available on the Advanced Plan.
For more details, refer to the CommCare Messaging Vocabulary section.
Choosing a Phone Number (SMS Connectivity)
CommCare requires a phone number, also known as a gateway, to send and receive messages. The choice depends on the project’s country, scale, and messaging requirements. Options include:
Dimagi-Owned Gateways
These are high-scale phone numbers or shortcodes that are only available in certain countries. Per-message fees are lower and some countries support toll-free incoming messages for users.
Telerivet Android Phone
You can use an Android phone with the Telerivet app to send and receive messages for your project. The phone connects to CommCareHQ and handles all messaging.
This option is ideal for smaller-scale projects or pilots. The project will have to directly pay costs for the phone and messages to and from the phone.
More details on SMS options are available here: Gateway Options for SMS Projects. Once you've chosen a gateway option, use Setup SMS Connection for your project.
Messaging in CommCare - A Beginner’s Guide
Set up an SMS Connection for a project
This tutorial will introduce you to using Messaging within your CommCare application. This will be based on a very simple application (registration of a pregnant woman and tracking of pregnant women). We'll add some basic messaging to the application including a welcome message to registered women, a message to remind women to visit the clinic if they have danger signs, and messages to health workers to remind them of delivery dates.
Reviewing CommCare and Case Management
Before starting this tutorial, please make sure you've completed the https://dimagi.atlassian.net/wiki/x/HTDKfw and https://dimagi.atlassian.net/wiki/x/0ibKfw. This contains important information about setting up a basic CommCare application.
Learning Objectives for Messaging Beginner Tutorial
In this tutorial you will learn the following:
Choosing the SMS connection for a project
Registering a case that has a phone number that can be used for Messaging
Adding a phone number to a mobile worker so that they can send and receive messages
Setup a Reminder that is sent to all cases
Setup a Conditional Reminder
Setup a Reminder That Sends Based on a Date in a Case
Viewing Messages in the Reminder Calendar and Message Log
Before starting this tutorial, set up a new project space, unless you've already done so. The next steps to follow are:
Choose an SMS Connection for the Project
All SMS projects require an SMS connection to send and receive messages from users. Dimagi provides several shared connections for use in projects. Details on Dimagi's connections are available on these pages:
This tutorial will use one of Dimagi's international SMS connections, Twilio. This connection supports sending messages to many countries but at a higher cost. If there is a specific connection that makes more sense in your country, feel free to use that instead.
Choose Twilio Connection
Go to the Messaging tab and choose the SMS Connectivity page.
From the list of gateways, find MOBILE_BACKEND_TWILIO and click on the Set as Default button. This will set up the project to use Twilio to send messages by default.
Register a Case For Messaging
This describes how to set up a form that will register a case for use with Messaging. You can currently only have one phone number per case. We need to save this phone number to a specially named case property called contact_phone_number.
Create and Set up a New Application
In your project create a new application called Pregnancy Messaging.
Update the first module to set the name to Pregnancy.
c. And then set the case type to pregnancy.
Create the Registration Form
This tutorial assumes you already know how to create a form with hidden values. Rename the Untitled Form to Registration and add questions so that it resembles the following. Some important things to note:
We name the phone number question contact_phone_number. This is the name of the special case property that Messaging will look for when creating a new case. When entering a phone number here, it must include the country code. For example, for a US phone number, you would enter 15551234567. For an Indian number, you would enter a number like 919560196285.
We add a hidden value called contact_phone_number_is_verified with a calculation of 1. This is also needed as a case property for Messaging to know how to send messages to a case.
The full definition of the form is listed below:
Question ID | Type | Label | Required | Validation | Calculate |
|---|---|---|---|---|---|
name | Text | Name | Yes |
|
|
edd | Date | Expected Date of Delivery | Yes | . > today() and (. - today() <= 275) |
|
contact_phone_number | Phone Number or Numerical ID | Phone Number |
|
|
|
contact_phone_number_is_verified | Hidden Value |
|
|
| 1 |
Save the form and configure the case management to create a new case. Save all the questions as case properties. It is important the contact_phone_number and contact_phone_number_is_verified case properties are named correctly.
This form is now set up to register a new case that is set up to send and receive SMS messages. We'll now configure an Update form that the mobile worker can use to identify high mothers with any risk symptoms.
Configure a Follow-Up Form
We'll now add another form to the application to allow health workers to follow up with pregnant women.
Add another form to the Pregnancy module using the + Form link.
Name the new form Follow-Up.
Add questions and a hidden value to the form that will evaluate the mother for any risk symptoms.
The following calculations and display logic are used in the form:
Question ID or Choice Value | Type | Label | Required | Display Logic | Calculation |
|---|---|---|---|---|---|
danger_signs | Multiple Answer | Does the mother have any of these risk symptoms? |
|
|
|
rupture_or_leaking | Choice | Rupture or leaking |
|
|
|
continued_bleeding | Choice | Continued Bleeding |
|
|
|
severe_headache | Choice | Severe Headache |
|
|
|
convulsions | Choice | Convulsions |
|
|
|
visited_clinic_already | Single Answer | Has already visited the clinic for symptoms? | Yes | /data/danger_signs != '' |
|
yes | Choice | Yes |
|
|
|
no | Choice | No |
|
|
|
clinic_visit_reminder | Label | Please check in with this mother weekly until she visits the clinic. |
| /data/send_clinic_visit_reminder = 'yes' |
|
send_clinic_visit_reminder | Hidden Value |
|
|
| if(/data/visited_clinic_already = "no", 'yes', 'no') |
Save the form and configure the case management for the form. This form will be setup to update or close cases. We'll only save send_clinic_visit_reminder to the case. When we setup the reminder, we'll use this case property to determine if a pregnant mother should receive a reminder to visit the clinic.
Our application is now set up with the information it needs to trigger sending reminders to cases. We now need to add phone number information to any mobile workers so that we can also send them messages.
Setup a Mobile Worker for Messaging
To set up a mobile worker for Messaging, we just need to add their phone number in the correct format. The phone number must be added with the country code. For example, for a US phone number, you would enter 15551234567. For an Indian number, you would enter a number like 919560196285. Register a new mobile worker (if needed) and set their phone number using the Add Number button.
Verifying a Phone Number
For some projects, you may need two-way messaging (or to allow the mobile worker to send messages to CommCareHQ). To do this, you need to verify your phone number which will associate that particular phone number with the mobile worker (and not allow it to be used by other mobile workers). This is not needed for this tutorial, but you may need it for another project.
To verify a phone number, click on the Verify button.
You should receive a message from CommCareHQ to your phone number. Reply with 123 to that message to verify the phone number. If you refresh the mobile worker page, it should now show you the phone number as Verified.
If you see an Already In Use message, this means that the phone number is already in use by somebody else (maybe a case or another mobile worker). Close any cases that are using the phone number or remove it from any mobile workers who are using it. You should then be able to click the Verify button.
Now that our application and mobile workers are set up, we can now configure the reminders to send messages to pregnant women and mobile workers.
9.Welcome Message Reminder
We'll now configure a reminder that will be sent to every registered mother. This message will inform them that they may be receiving messages and what they mean.
To set up a reminder, go to the Messaging tab, then in the left sidebar choose Reminders. Then click on the + Add Reminder button.
We'll now configure when the reminder will be sent. Give the reminder a name (ex.
Welcome Message) and scroll down the Start section.
Send for Case Type: This controls which case type will cause the reminder to send. Choose pregnancy from the dropdown list.
Send Reminder For: This can be used to choose which cases will cause the reminder to be sent. We want the welcome message to be sent to All Cases.
Day of Reminder and Time of Day: You can also control what day or date and what time the reminder will be sent. For this reminder, we want it to be sent Immediately.
The next step is to choose who will receive a reminder. There are a number of options (the case, the case's owner, or a specific mobile worker group), but for this reminder, we just need to choose Case.
We can now specify the message content to send. Choose the SMS send option and provide the message to send. For this reminder, we can set the message to "You'll receive reminders and tips over the course of your pregnancy on this number."
This reminder doesn't need to Repeat and we don't need any of the advanced options, so we can just go ahead and choose Create Reminder button.
We'll now follow a similar process to set up our other two reminders for the tutorial. Instead of setting up a reminder to all cases, we'll instead set up a recurring reminder to high-risk mothers until they've visited the clinic.
Conditional Reminders
We'll now set up a reminder that will only be sent to mothers who have been flagged by the CHW as needing to visit the clinic. This message will also be set to repeat on a weekly basis until the CHW visits the mother and indicates that she no longer needs to visit the clinic.
Add another reminder to the project (go to the Messaging tab, choose Reminders, and then click on the + Reminder button).
We'll now configure which pregnant mothers will receive this reminder. Give the reminder a name (ex. High Risk Clinic Visit) then scroll to the Start Section.
Send for Case Type: We want to choose pregnancy from the list as this message will be sent to pregnant mothers.
Send Reminder For: This will control which cases receive the reminder. We want to choose Only Cases in the Following State. Then set the reminder to send when the case property send_clinic_visit_reminder equals yes
Day Of Reminder and Time of Day. This controls what day and time of day the mother will begin to receive this reminder. For this example, let's assume that our clinic day occurs on Tuesdays. So we want to send the mother a reminder on Monday at 9 am. For Day of Reminder choose Specific Day of the Week and Monday. This option will send the message on the next upcoming Monday after the pregnant mother is flagged for a clinic visit. For Time of Day choose at a Specific Time and 9:00.
The next step is to choose who will receive the reminder. We want this reminder to go to the case (the pregnant woman), so choose Case.
We can now specify the message content to send. Choose the SMS send option and provide the message to send. For this reminder, we can set the message to "Please visit the clinic for a checkup. Clinic is on Tuesday mornings."
We will also configure to reminder to Repeat every week until the mother visits the clinic. Choose the Indefinitely option and specify that the reminder should repeat every 7 days.
Concept
Reminders will automatically stop sending if the condition used to start the reminder becomes false. In this case, if send_clinic_visit_reminder does not equal yes, the reminder will stop. You can also stop a reminder by using a stop condition, specified in Advanced Options.
Our reminder is now setup so we can go ahead and choose the Create Reminder button.
We'll now follow a similar process to set up our final reminder for the tutorial. This reminder will instead go to the health worker, indicating that the mother is due for delivery soon and she should be checked up on.
Date Based Reminder
The final reminder that we'll set up will go out to a different date for each client, based on their date of delivery. This reminder will also go to the health worker and remind them that a particular mother is due for delivery in the next few days.
Add another reminder to the project (go to the Messaging tab, choose Reminders and then click on the + Reminder button).
We'll now configure which pregnant mothers will receive this reminder. Give the reminder a name (ex. Delivery Reminder) then scroll to the Start Section.
Send for Case Type: We want to choose pregnancy from the list as this message will be sent for pregnant mothers.
Send Reminder For: This will control which cases receive the reminder. We want to choose All Cases since all pregnant mothers will have this message sent out to them.
Day Of Reminder and Time of Day. This controls what day and time of day the mother will begin to receive this reminder. We want to send this reminder 3 days before the expected delivery date at 9 am. For Day of Reminder choose Date in Case. Specify the case property as edd and before the date by 3 days. For Time of Day, choose Specific Time and enter 9:00.
The next step is to choose who will receive the reminder. We want this reminder to go to the health worker (the mobile worker), so choose Case Owner (this is the mobile worker who registered the case).
We can now specify the message content to send. Choose the SMS send option and provide the message to send. For this reminder, we can set the message to "Please visit {case.name}. She is due for delivery in 3 days". The special syntax {case.name} allows you to use case properties in the message. You can replace name with another case property if you want to have that in the message instead.
This reminder doesn't need to Repeat and we don't need any of the advanced options, so we can just go ahead and choose Create Reminder button.
Now that our reminders are all set up, we can use the application to register some test data and verify that the reminders are working correctly and are sent to each user.
Testing and Viewing Scheduled Reminders
Now that our reminders and application are set up, you can use a mobile worker to register a new pregnant mother and view her scheduled reminders on CommCareHQ.
Setup a Sample Case
Register a new mobile worker for your project and add a new phone number as described in Messaging Beginner Tutorial | Setup a Mobile Worker for Messagingarchived.
Install the application on a phone and register a new pregnant woman using the application. Make sure you enter a valid phone number (including country code) when registering the mother.
Fill out the follow-up form and indicate that the woman has risk symptoms and hasn't visited the clinic yet.
Viewing Cases in the Calendar
Go to the Messaging tab and click on Reminder Calendar in the left bar. You'll see the each upcoming message scheduled to go out to cases and mobile workers. For repeating reminders, only the next scheduled reminder will be displayed.
Viewing Messages in the Log
The message log can be used to view a history of all messages sent for the project. To view the message log, go to the Reports tab and choose Message Log from the left bar. You can choose a date range and filter to specific messages by message type. If you leave Message Type blank, it will show all messages. Click on the Apply button to view the messages in your chosen date range.
Basic Troubleshooting
If your messages are not in the message log, there is some basic troubleshooting that you can do.
Check the Error Log: Go to the Messaging tab and choose Reminders in Error on the left pane. This should show you a list of reminders that failed to send out. Review this list and address any issues notes.
Verify the Phone Number: You may have entered the phone number incorrectly - make sure you've included the country code when entering the number for the case or phone. You can also send a test message. Go to the Messaging tab and choose Compose SMS. Type the phone number here and a message and choose Send.
Upcoming Messages not in Calendar: If the upcoming messages are not in your reminder calendar, then you may have not configured your reminder correctly. Use the Case List report to make sure the registered cases have the right set of case properties (and match your reminders).
Other troubleshooting tips are available here: https://dimagi.atlassian.net/wiki/x/1S-Kfw.
Congratulations, you've completed the messaging beginner tutorial! See https://dimagi.atlassian.net/wiki/x/XCrKfw to read more about messaging and other functionality.
Do's and Don'ts of Sending an SMS to the Case
Send a welcome/confirmation message upon enrollment/registration
Why? Sending a message immediately upon registration is helpful because you can confirm that the phone number is correct and that the user is able to receive messages. It can also help in explaining how they are going to get messages.
How? Set up an offset-based reminder in which the reminder is triggered when a universal case property like name exists. Configure the offset to be 0 days, 00:01 time so that the message will be set 1 minute after registration the system, and set the repeat to 1 so that it only repeats one time
Trigger: (For each case with:) case type ____; case property name (exists)
Start: Start date (as soon as it triggers); Start Offset 0.
Send: Send (SMS); To (The Case); Frequency (Advanced); Schedule-type (Offset Based); Default Language __; Schedule- Days to Wait 0; Time to Wait 00:01; Language/Message __|Welcome! You are successfully registered!
Stop: Repeat the schedule (the following number of times:) 1
Other key things to keep in mind -
·Keep your messages short (<160 characters)! If you send long messages (greater than 160 characters) your message will be broken up into multiple SMS that your phone will have to join together. This can make the message appear disjointed or even in the wrong order! Unfortunately Dimagi/CommCare cannot guarantee that long messages will be sent in the right order, or even the that they will get sent at all. Also keep in mind that if you are doing a survey and have a label followed by a question, these will be combined into a single text message.
Allow your case to opt out if they want to. No one wants to be stuck receiving text messages you don't want. While hopefully you can do lots of testing and use SMS messaging best practices, some people may want to be able to remove themselves, or may need to be able to do so in accordance with IRB or other requirements. You can set up a simple keyword to do this. To do this, create a form that is a simple SMS survey and has an empty question/simple message and configure the form to deactivate any trigger or to close the case, whatever is appropriate for your project. Then set a keyword reminder like "stop" so that your clients can just text in "stop" in order to opt out.
Avoid Required Questions. In some cases users will not complete a survey. Required questions in a survey will not allow that survey to be saved in a partial state. Make sure that questions are not marked as required. For non-text questions, it is not possible to skip them through SMS anyway.