Getting Started with Messaging

Don't forget to visit the CommCare Messaging Home Page.

Overview

CommCare allows you to schedule SMS messages, emails or IVR (voice calls) to cases and mobile workers registered in the system. Here are some example usage cases: 

  • Sending a reminder to a health worker when one of their beneficiaries is due for a delivery.

  • Sending counselling messages to beneficiaries (for example, reminding them to take their iron pills).  

  • Sending a short survey to a beneficiary (for example, asking if they've attended a clinic)

  • Have a patient request information from the system.  For example, if they are enrolled in a smoking cessation study, they may request support messages from the system. 

Messaging allows you to setup keywords (so users can send messages and data to the system on their own), reminders (scheduled conditional messages to a specific phone number) and broadcasts (mass messages or surveys to group of numbers).  

The SMS Basics section covers basic information about how SMS works and is priced in CommCareHQ.  

If you have questions, check out CommCare Messaging Frequently Asked Questions.

SMS as a Medium

SMS, as a medium, is like email in that information is not encrypted before being sent. This means that you should not include personally identifiable or sensitive information in SMS, just as you would not in an email. 

SMS is also not a medium that is 100% reliable. Messages can be dropped due to network issues and often there is no way for a sender to know if the recipient actually received the message. For this reason, SMS should not be used for situations such as reaching out for help in a life-threatening emergency, and other more reliable mediums should be used instead.

With all of this in mind, SMS does make a good medium for use cases such as generic SMS messaging campaigns, appointment reminders, or alerts.

How CommCare HQ Sends Messages

CommCareHQ uses an SMS Gateway to send and receive messages.  SMS gateways are phone numbers provided by other companies (or cell phone companies) that connect to the cell phone network to send and receive messages.   CommCareHQ is connected to multiple gateways to support different countries.  

More information on Dimagi's existing gateways can be found https://dimagi.atlassian.net/wiki/x/rwfKfw.

89c3cc2c-29c6-49c9-9d5c-7425a027c3e8.png

Using an Android Gateway

If your country isn't supported by Dimagi's existing gateways you can use an Android-based phone to send and receive messages for your project.   This will use the phone's SIM card and SMS balance.   Dimagi has partnered with Telerivet to use their system to support Android gateways.  The Android phone will connect to CommCareHQ using the internet to know when to send messages and to send incoming messages to CommCareHQ.  

Android phones are typically used for low volume projects since cell phone companies usually limit the number of messages that can be sent or received through a regular phone number.

f4ae6000-22ef-45ba-aba5-76f503bc42bb.png

Sending an SMS directly from a phone

A CommCare phone application can also generate an SMS from a form using the phone's default SMS application and allow the user to send this message directly to the recipient, without requiring a server. This approach gives full control to the mobile user and does not require data connectivity between the user and the server. It can therefore be useful in settings where the cell network only supports SMS messages. SMS costs are charged to the user's SIM card and CommCare cannot send these messages automatically, always requiring direct user interaction. Here is a brief description of how to set up a CommCare form to generate an SMS.

Short Code vs. Long Codes

Depending on the gateway used, it can be a short code or a long code.  Long codes are similar to regular phone numbers.  Short codes are special phone numbers that have a specific name (ex. CommCare) or a shorter phone number and can be used to send a very high volume of messages.   Short codes also allow for reverse billing.  

Reverse Billing

When users send a message to your project's SMS gateway, they normally pay a per message cost - this is exactly like them sending a message to a friend.  Since this cost can dissuade users from using your messaging system, some gateways are reverse billed.  This means that users can send messages for free to the gateway, and you will pay for those messages instead of the user paying for them.

SMS Pricing

There are two types of costs associated with SMS - a monthly software plan cost for CommCareHQ and a per-message cost.  

  • SMS Pricing Plan: To use SMS, you must be on the Standard or above software plan.  The Standard plan allow for outgoing messages (messages from CommCareHQ to a user).  To receive messages from users (example, users respond to surveys or send in information), you must be in the Advanced or above pricing plan. 

  • Per Messages Costs:

    • If using a gateway integrated with CommCareHQ, your CommCare invoices will charge the per-SMS fee charged by the SMS gateway. The charge from the SMS Gateway can fluctuate and also depends on exchange rates as well as the destination number's country and network.

    • If using a Telerivet gateway, you may have to pay a monthly fee to Telerivet for use of their service based on your subscription, as well as pay for the cost of the phone, it's phone plan, and the cost of messages to and from the phone.

    • In all cases, your CommCare invoices will charge a USD 0.01 per SMS surcharge paid to Dimagi.

    • In all cases, large messages can be charged as multiple SMS. Typically, an SMS sent using only the Roman alphabet is charged as one SMS for each 160 characters (roughly), and an SMS sent using special characters is charged as one SMS for each 70 characters (roughly).

New In Country Gateways

For larger high-volume projects in countries we do not already support, Dimagi can support connecting a new gateway to CommCareHQ if your subscription allows it.  SMS gateways can be purchased directly from a cell phone company in a country, or from an aggregator that resells phone numbers. Setting up a new gateway requires a significant amount of time and can have high costs. After identifying potential providers, please use the Aggregator Evaluation Worksheet to ensure that key questions are asked and answered during meetings with respective telcos/aggregators. Once you have selected an appropriate gateway provider based on project requirements (cost, setup time, reverse billing), the gateway can be connected to CommCareHQ with assistance from Dimagi's developer team. Please contact information@dimagi.com for more information.  

Messaging Best Practices and Use Cases

Below are some suggestions for how to effectively use CommCare Messaging / SMS in different use cases.

Setting Up Your Project for Messaging

Type of Messaging

When starting your project, you need to decide between outgoing-only and two-way messaging.  Check out the CommCare Messaging Vocabulary section to learn more. 

  • Outgoing Only: Users can only receive messages and can't respond or send data to the system.  (Available on the Standard Plan)

  • Two Way Messaging: Users can also send data back to the system by responding to surveys or sending in keywords.  (Available on the Advanced Plan).

Choosing a Phone Number (SMS Connectivity)

To send and receive messages CommCare needs to use a phone number to send messages.  This is also called a gateway.  Depending on the country of your project, scale and type of messaging required, the following options are available:

  • Dimagi Owned Gateways : These are high scale phone numbers or shortcodes that are only available in certain countries.  Per-message fees are lower and some countries support toll-free incoming messages for users. 

  • Telerivet Android Phone: Send and receive messages using an Android phone by using the Telerivet app. This phone will connect to CommCareHQ and be used for messaging for the project.  This option works well for smaller scale pilots.   Project will directly pay costs for the phone and messages to and from the phone.  

More details on SMS options are available here: Gateway Options for SMS Projects.  Once you've chosen a gateway option, use https://dimagi.atlassian.net/wiki/x/fCnKfw for your project. 

CommCare Messaging Vocabulary

This section contains definitions and links to key concepts in CommCare Messaging. Please click below to see!

 

One way SMS

Refers to outgoing messages, sent towards devices or mobile phones. One example is an appointment reminder. Recipients do not need to be verified in HQ. These messages are recorded in the message log. This feature is available on the Standard plan.

Two way SMS

Refers to a system with both outgoing and incoming messages. In this situation, data can be collected over SMS. Two way messaging is available on Pro plan (at an additional cost, please contact sales) or higher.

Broadcast Messages

A simple messages scheduled to send to a group of users, either mobile workers or cases. Does not support case management.  

Case Groups

Case groups allow project managers to organize active cases together. Case groups can enable targeted messaging outreach, similar to mobile worker groups.

Chat

A feature that allows a CommCareHQ user communicate with a case in real time. The chat window displays all of the messages sent and received with that contact.

Contact

Contacts include both cases and mobile workers.

Contact Phone Number

Receives SMS messages, must include the country code. Must be verified to enable two way messaging.

Gateway

The portal that sends and receives messages for your project. Gateways are set-up with a local SIM card, and recipients will see that SIM as the contract phone number.

Gateway Testing

A process to test that the phone number is online, operational in the country, that messages arrive in a timely manner, and assess if maximum SMS volumes cause errors or delays in message delivery.

The process should verify that messages are correctly forwarded from HQ via API.

Keyword

A word or letter that activates an SMS configuration.

Message

The content of the SMS message, limit of 160 characters.

Message Log

A report that shows all of the messages sent and received on CommCareHQ.

Message History

A report that can be filtered by communication type, date and phone number. The report shows the delivery status of each message.

Recipient

A contact enrolled in HQ to receive SMS messages.

Reminder (single event)

A feature that allows you to schedule a single outgoing SMS message and may use case management.

Reminder (multi-event)

A feature that allows you to schedule a sequence of outgoing messages, sent at different intervals; may use case management.

Reminder Calendar

A feature that allows you to review upcoming messages that includes the date, recipient, and message type. Can be used for testing your SMS project. 

Structured Keyword

A keyword set-up that allows incoming SMS messages to submit multiple data points as a single message.

SMS Connectivity

The page on CommCare HQ that helps manage the SMS Gateway set-up.

SMS Export

A quick way to download the Messaging Log, with date filters.

Survey

A series of questions answered via incoming messages from a verified SMS user.

Telerivet

A software platform that can be used to set-up local SMS gateways and links to CommCareHQ.

Timing

A configurable part of SMS communication, that customize the sending of messages. This may include sending messages: immediately, at a specific time of day, a number of days before or after. or a time set in a case property.

Verification

A 2 step process that allows HQ to confirm the status of a contacts phone number, that enables the contact to submit information. This enables survey and keyword features.

 

SMS Basics

Sending an SMS to the Case

  • Send a welcome/confirmation message upon enrollment/registration

    • Why? Sending a message immediately upon registration is helpful because you can confirm that the phone number is correct and that the user is able to receive messages. It can also help in explaining how they are going to get messages.

    • How? Set up an offset-based reminder in which the reminder is triggered when a universal case property like name exists. Configure the offset to be 0 days, 00:01 time so that the message will be set 1 minute after registration the system, and set the repeat to 1 so that it only repeats one time.

      • Trigger: (For each case with:) case type ____; case property name (exists)

      • Start: Start date (as soon as it triggers); Start Offset 0

      • Send: Send (SMS); To (The Case); Frequency (Advanced); Schedule-type (Offset Based); Default Language __; Schedule- Days to Wait 0; Time to Wait 00:01; Language/Message __|Welcome! You are successfully registered!

      • Stop: Repeat the schedule (the following number of times:) 1

  • Keep your messages short (<160 characters)! If you send long messages (greater than 160 characters) your message will be broken up into multiple SMS that your phone will have to join together. This can make the message appear disjointed or even in the wrong order! Unfortunately Dimagi/CommCare cannot guarantee that long messages will be sent in the right order, or even the that they will get sent at all.  Also keep in mind that if you are doing a survey and have a label followed by a question, these will be combined into a single text message.

  • Allow your case to opt out if they want to. No one wants to be stuck receiving text messages you don't want. While hopefully you can do lots of testing and use SMS messaging best practices, some people may want to be able to remove themselves, or may need to be able to do so in accordance with IRB or other requirements. You can set up a simple keyword to do this. To do this, create a form that is a simple SMS survey and has an empty question/simple message and configure the form to deactivate any trigger or to close the case, whatever is appropriate for your project. Then set a keyword reminder like "stop" so that your clients can just text in "stop" in order to opt out.

  • Avoid Required Questions. In some cases users will not complete a survey.  Required questions in a survey will not allow that survey to be saved in a partial state. Make sure that questions are not marked as required.  For non-text questions, it is not possible to skip them through SMS anyway.

Messaging Beginner Tutorial

This tutorial will introduce you to using Messaging within your CommCare application.    This will be based on a very simple application (registration of a pregnant woman and tracking of pregnant women.).  We'll add some basic messaging to the application including a welcome message to registered women, a message to remind women to visit the clinic if she has danger signs and messages to health workers to remind them of delivery dates. 

Reviewing CommCare and Case Management

Before starting this tutorial, please make sure you've completed the https://dimagi.atlassian.net/wiki/x/HTDKfw and https://dimagi.atlassian.net/wiki/x/0ibKfw. This contains important information about setting up a basic CommCare application.

Learning Objectives for Messaging Beginnner Tutorial

In this tutorial you will learn the following:

  • Choosing the SMS connection for a project

  • Registering a case that has a phone number that can be used for Messaging

  • Adding a phone number to a mobile worker so that they can send and receive messages

  • Setup a Reminder that is sent to all cases

  • Setup a Conditional Reminder

  • Setup a Reminder That Sends Based on a Date in a Case

  • Viewing Messages in the Reminder Calendar and Message Log

Before starting this tutorial, setup a new project space, unless you've already done so.  

Choose an SMS Connection for Project

All SMS projects require an SMS connection to send and receive messages from users.   Dimagi provides a number of shared connections for use in projects.   Details on Dimagi's connections are available on these pages:

This tutorial will use one of Dimagi's international SMS connections, Twilio.  This connection supports sending messages to many countries, but at a higher cost.  If there is a specific connection that makes more sense in your country, feel free to use that instead. 

Choose Twilio Connection

Go the Messaging tab and choose the SMS Connectivity page. 

From the list of gateways, find MOBILE_BACKEND_TWILIO and click on the Set as Default button.  This will setup the project to use Twilio to send messages by default.  

Register a Case For Messaging

This describes how to setup a form that will register a case for use with Messaging.   You can currently only have one phone number per case.  We need to save this phone number to a specially named case property called contact_phone_number. 

Create and Setup a New Application

  1. In your project create a new application called Pregnancy Messaging.  

  1. Update the first module to set the name to Pregnancy. 

  1. And then set the case type to pregnancy.  

Create the Registration Form

This tutorial assumes you already know how to create a form with hidden values.  Rename the Untitled Form to Registration and add questions so that it resembles the following.  Some important things to note:

  • We name the phone number question contact_phone_number.  This is the name of the special case property that Messaging will look for when creating a new case.  When entering a phone number here, it must include the country code.  For example, for an US phone number you would enter 15551234567.  For an Indian number you would enter a number like 919560196285.

  • We add a hidden value called contact_phone_number_is_verified with a calculation of 1. This is also needed as a case property for Messaging to know to send messages to a case. 

The full definition of the form is listed below:

Question ID

Type

Label

Required

Validation

Calculate

Question ID

Type

Label

Required

Validation

Calculate

name

Text

Name

Yes

 

 

edd

Date

Expected Date of Delivery

Yes

. > today() and (. - today() <= 275)

 

contact_phone_number

Phone Number or Numerical ID

Phone Number

 

 

 

contact_phone_number_is_verified

Hidden Value

 

 

 

1

Save the form and configure the case management to create a new case.  Save all the questions as case properties.  Its important the contact_phone_number and contact_phone_number_is_verified case properties are named correctly.

This form is now setup to register a new case that is setup to send and receive SMS messages.  We'll now configure an Update form that the mobile worker can use to identify high mothers with any risk symptoms.  

Configure a Follow Up Form

We'll now add another form to the application to allow health workers to follow up with pregnant women.

  1. Add another form to the Pregnancy module using the + Form link.

  1. Name the new form Follow-Up.

  1. Add questions and a hidden value to the form that will evaluate the mother for any risk symptoms.

The following calculations and display logic is used in the form:

Question ID or Choice Value

Type

Label

Required

Display Logic

Calculation

Question ID or Choice Value

Type

Label

Required

Display Logic

Calculation

danger_signs

Multiple Answer

Does the mother have any of these risk symptoms?

 

 

 

rupture_or_leaking

Choice

Rupture or leaking

 

 

 

continued_bleeding

Choice

Continued Bleeding

 

 

 

severe_headache

Choice

Severe Headache

 

 

 

convulsions

Choice

Convulsions

 

 

 

visited_clinic_already

Single Answer

Has already visited clinic for symptoms?

Yes

/data/danger_signs != ''

 

yes

Choice

Yes

 

 

 

no

Choice

No

 

 

 

clinic_visit_reminder

Label

Please check in with this mother weekly until she visits the clinic.  

 

/data/send_clinic_visit_reminder = 'yes'

 

send_clinic_visit_reminder

Hidden Value

 

 

 

if(/data/visited_clinic_already = "no", 'yes', 'no')

  1. Save the form and configure the case management for the form.  This form will be setup to update or close cases.  We'll only save send_clinic_visit_reminder to the case.  When we setup the reminder, we'll use this case property to determine if a pregnant mother should receive a reminder to visit the clinic.  

Our application is now setup with the information it needs to trigger sending reminders to cases.  We now need to add phone number information to any mobile workers so that we can also send them messages. 

Setup a Mobile Worker for Messaging

To setup a mobile worker for Messaging, we just need to add their phone number in the correct format.  The phone number must be added with the country code. For example, for an US phone number you would enter 15551234567.  For an Indian number you would enter a number like 919560196285.    Register a new mobile worker (if needed) and set their phone number using the Add Number button.  

Verifying a Phone Number

For some projects, you may need two-way messaging (or to allow the mobile worker to send messages to CommCareHQ).   To do this, you need to verify your phone number which will associate that particular phone number with the mobile worker (and not allow it to be used by other mobile workers).   This is not needed for this tutorial, but you may need it for another project.  

  1. To verify a phone number, click on the Verify button.  

  1. You should receive a message from CommCareHQ to your phone number.  Reply with 123 to that message to verify the phone number.  If you refresh the mobile worker page, it should now show you the phone number as Verified. 

  1. If you see an Already In Use message, this means that the phone number is already in use by somebody else (maybe a case or another mobile worker).   Close any cases that are using the phone number or remove it from any mobile workers who are using it.  You should then be able to click the Verify button.  

    Now that our application and mobile workers are setup, we can now configure the reminders to send messages to the pregnant women and mobile workers.

Welcome Message Reminder

We'll now configure a reminder that will be sent to every registered mother.  This message will inform them that they maybe receiving messages and what they mean.  

  1. To setup a reminder, go to the Messaging tab, then in the left side bar choose Reminders.  Then click on the + Add Reminder button.

  1. We'll now configure when the reminder will be sent.  Give the reminder a name (ex.

Welcome Message) and scroll down the Start section.

  • Send for Case Type: This controls which case type will cause the reminder to send.  Choose pregnancy from the dropdown list.  

  • Send Reminder For: This can be used to choose which cases will cause the reminder to send.  We want the welcome message to be sent to All Cases

  • Day of Reminder and Time of Day: You can also control what day or date and what time the reminder will be sent.  For this reminder, we want it to be sent Immediately.

  1. The next step is to choose who will receive a reminder.  There are a number of options (the case, the case's owner, or a specific mobile worker group), but for this reminder we just need to choose Case

  1. We can now specify the message content to send.  Choose the SMS send option and provide the message to send.  For this reminder, we can set the message to "You'll receive reminders and tips over the course of your pregnancy on this number."

  1. This reminder doesn't need to Repeat and we don't need any of the advanced options, so we can just go ahead and choose Create Reminder button. 

We'll now follow a similar process to setup our other two reminders for the tutorial.   Instead of setting up a reminder to all cases, we'll instead setup a recurring reminder to high risk mothers until they've visited the clinic.  

Conditional Reminders

We'll now setup a reminder that will only be sent to mothers who have been flagged by the CHW as needing to visit the clinic. This message will also be setup to repeat on a weekly basis until the CHW visits the mother and indicates that she no longer needs to visit the clinic.

  1. Add another reminder to the project (go to the Messaging tab, then choose Reminders and then click on the + Reminder button).

  2. We'll now configure which pregnant mothers will receive this reminder.  Give the reminder a name (ex. High Risk Clinic Visit) then scroll to the Start Section. 

  • Send for Case Type: We want to choose pregnancy from the list as this message will be sent to pregnant mothers.

  • Send Reminder For: This will control which cases receive the reminder. We want to choose Only Cases in the Following State. Then set the reminder to send when the case property send_clinic_visit_reminder equals yes

  • Day Of Reminder and Time of Day. This controls what day and time of day that the mother will begin to receive this reminder. For this example, let's assume that our clinic day occurs on Tuesdays. So we want to send the mother a reminder on Monday at 9am. For Day of Reminder choose Specific Day of the Week and Monday. This option will send the message on the next upcoming Monday after the pregnant mother is flagged for a clinic visit.  For Time of Day choose at a Specific Time and 9:00.

  1. The next step is to choose who will receive the reminder. We want this reminder to go to the case (the pregnant woman), so choose Case.

  1. We can now specify the message content to send. Choose the SMS send option and provide the message to send. For this reminder, we can set the message to "Please visit the clinic for a checkup. Clinic is on Tuesday mornings."

  1. We will also configure to reminder to Repeat every week until the mother visits the clinic. Choose the Indefinitely option and specify that the reminder should repeat every 7 days.

Concept

Reminders will automatically stop sending if condition used to start the reminder becomes false. In this case, if send_clinic_visit_reminder does not equal yes, the reminder will stop. You can also stop a reminder by using a stop condition, specified in Advanced Options. 

  1. Our reminder is now setup so we can go ahead and choose the Create Reminder button.

We'll now follow a similar process to setup our final reminder for the tutorial.   This reminder will instead go to the health worker, indicating that the mother is due for delivery soon and she should be checked up on.  

Date Based Reminder

The final reminder that we'll setup will go out to a different date for each client, based on their date of delivery.  This reminder will also go to the health worker and remind them that a particular mother is due for delivery in the next few days.  

  1. Add another reminder to the project (go to the Messaging tab, then choose Reminders and then click on the + Reminder button).

  2. We'll now configure which pregnant mothers will receive this reminder.  Give the reminder a name (ex. Delivery Reminder) then scroll to the Start Section. 

  • Send for Case Type: We want to choose pregnancy from the list as this message will be sent for pregnant mothers.

  • Send Reminder For: This will control which cases receive the reminder. We want to choose All Cases since all pregnant mothers will have this message sent out for them.

  • Day Of Reminder and Time of Day. This controls what day and time of day that the mother will begin to receive this reminder. We want to send this reminder 3 days before the expected delivery date at 9am. For Day of Reminder choose Date in Case. Specify the case property as edd and before date by 3 days. For Time of Day, choose Specific Time and enter 9:00.

  1. The next step is to choose who will receive the reminder. We want this reminder to go to the health worker (the mobile worker), so choose Case Owner (this is the mobile worker who registered the case).

  1. We can now specify the message content to send. Choose the SMS send option and provide the message to send. For this reminder, we can set the message to "Please visit {case.name}. She is due for delivery in 3 days". The special syntax {case.name} allows you to use case properties in the message. You can replace name with another case property if you want to have that in the message instead.

  1. This reminder doesn't need to Repeat and we don't need any of the advanced options, so we can just go ahead and choose Create Reminder button.

Now that our reminders are all setup, we can use the application to register some test data and verify that the reminders are working correctly and are sent to each user.

Testing and Viewing Scheduled Reminders

Now that our reminders and application is setup, you can use a mobile worker to register a new pregnant mother and view her scheduled reminders on CommCareHQ.  

Setup a Sample Case

  1. Register a new mobile worker for your project and add a new phone number as described in Messaging Beginner Tutorial | Setup a Mobile Worker for Messagingarchived.

  2. Install the application on a phone and register a new pregnant woman using the application.   Make sure you enter a valid phone number (including country code) when registering the mother.  

  3. Fill out the Follow Up form and indicate that the woman has risk symptoms and hasn't visited the clinic yet.  

Viewing Cases in the Calendar

Go to the Messaging tab and click on Reminder Calendar in the left bar.  You'll see the each upcoming message scheduled to go out to cases and mobile workers. For repeating reminders, only the next scheduled reminder will be displayed.  

Viewing Messages in the Log

The message log can be used to view a history of all messages sent for the project.   To view the message log, go to the Reports tab and choose Message Log from the left bar.  You can choose a date range and filter to specif messages by message type.  If you leave Message Type blank, it will show all messages.   Click on the Apply button to view the messages in your chosen date range. 

Basic Troubleshooting

If your messages are not in the message log, there is some basic troubleshooting that you can do. 

  • Check the Error Log:  Go to the Messaging tab and choose Reminders in Error on the left pane.  This should show you a list of reminders that failed to send out.  Review this list and address any issues notes. 

  • Verify the Phone Number: You may have entered the phone number incorrectly - make sure you've included the country code when entering the number for the case or phone.  You can also send test message.  Go to the Messaging tab and choose Compose SMS.  Type the phone number here and a message and choose Send

  • Upcoming Messages not in Calendar: If the upcoming messages are not in your reminder calendar, then you may have not configured your reminder correctly.  Use the Case List report to make sure the registered cases have the right set of case properties (and match your reminders).  

Other troubleshooting tips are available here: https://dimagi.atlassian.net/wiki/x/1S-Kfw.

Congratulations, you've completed the messaging beginner tutorial!  See https://dimagi.atlassian.net/wiki/x/XCrKfw to read more about messaging and other functionality.