One of the best ways to ensure high data quality in your CommCare applications is by both #1 spending time up front thinking about the data that you want to collect and #2 designing applications that include data quality controls in them. Below are some tips for ways to design your application to control for data quality. Please check out the Application Development section as well.
What data do you want to collect?
To start, make sure you go into your application building with a clear list of key indicators that you want to collect. Make sure that the “key indicators” worksheet is reflected in the app worksheet. In other words make sure that there are questions or hidden values which will generate the data required for the indicator.
Figure out your indicators. In case helpful, you can download this key indicators worksheet to guide you through thinking about what you want to do with your data and which data you will need easily accessible in the exports. As part of this, you should also create demo data in Excel to map out exactly how you want yoru data to appear.
Build a mock-up dashboard off the demo data. You can use the Tutorial: Create an Excel Dashboard here.
Create your app. Build an app shell with the questions which generate the data that you need in the exports.
Test your exports. These data should look like the raw data from the beginning. If they do not, revisit the configuration of your application.
Putting data quality controls in place
Case Management
You can use Case Management to save anything you want to be able to edit to the case.
Advantages:
Disadvantages:
Erroneous form submissions will still be present in form exports
It may not be feasible to design an application that allows updating/correction of all fields
If an erroneous answer impacted display conditions or other case management logic, then this approach may not undo everything.
An app can contain "edit" forms that load in the previous values of case properties, and allow users to update them. This then updates the value of the case property, so that the case details/export contain the most recent values. Details on the workflow can be found here https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143955265/Define+Your+Workflow#How-can-your-user-edit-data? .
Automatically Close Cases
Create rules on CommCareHQ to automatically close cases that are no longer relevant. Read more here: Automatically Update Cases .
Before You Deploy the App
The steps below outline how you can take these two key points into account during app building.
What data should you NOT change in CommCare?
Using CommCare's application builder it is very easy to make changes and updates to an application after it is deployed. However, before making certain types of changes you should be aware that there can be unexpected and potentially confusing implications both for the application design, as well as for subsequent data analysis. This page is mean to highlight some of those potential issues.
In general we recommend against changing Question IDs, Choice Values, or Case Properties unless there is a compelling reason. If the motivation is that you want to be able to view the column header in a specific way we recommend just changing the name of the column in the Form Export configuration. You can read more below
Changing Question IDs
Click to Expand
One common change which may seem very minor is to change a Question ID. For example, imagine you have a Question ID called village_nme and you want to change it to be village_name
App Structure
When you change a Question ID, the form builder will recognize the change and will update any references to the Question ID (for example, in display or validation conditions).
However, if you change a Question ID that either loads a value from a case or saves to a case property you will need to update/fix the Case Management screen. The app builder will provide an error message. In the example above, the error message will say that the Case Management refers to a question called village_nme which could not be found. You can fix it by simply choosing village_name from the dropdown.
Data
When you change a question ID, CommCare will treat the two question IDs as two separate questions; that is, CommCare will not treat this as an update, rather, it will treat it as though the village_nme question was deleted and the village_name question was added. The consequence is that when you download the data you will have two columns: village_nme, which will have values for all old form submissions, and village_name, which will have values for all new form submissions. In the export configuration, the old question ID, village_nme will appear as a 'Deleted Question'.
Alternatives
If you are concerned that you won't be able to figure out what a question refers to (for example, you forgot to rename the question ID so it is question45) you can always use the Export Form Contents to figure out what the Label text is, and then use that in the form export configuration.
Changing Choice Values
Click to Expand
Updates to choice values have similar caveats to Question IDs but a few additional warnings.
Example:
What is your favorite color (favorite_color)? blue, rd, yellow
You decide to change rd to red
App Structure
While if you change a Question ID CommCare will alert you or automatically fix any references, this will not happen for choice values. You will need to manually go through and make sure that all display conditions, validation conditions, and calculations are updated with the new choice value.
In this example if there was a follow-up question which only displayed if favorite_color = rd, after changing rd to red the follow-up question would NOT show up correctly until the display condition was also fixed.
Data
When viewing exports with split multiple choice questions, each change will be treated as a new option in exports, so you will see a mixture of the old and new value names. If you split your multiple choice questions (Splitting multi-select data in exports) you will have an additional column for each new value name. In this example, after changing rd to red you would have four columns: blue, rd, red, and yellow.
Changing Case Type & Case Property names
Changing case properties can have particularly complex consequences. Do NOT change the case type after data collection has started.
Click to Expand
The Case Type is a case-sensitive term used to indicate the type of a set of cases. For example, an maternal and child health app might have two case types: mother and child.
If you change a case type (ex. mother -> Mother) all old cases will be of case type mother, and all new case types will be case type Mother. You will have to use a tool like the Case Importer tool in order to fix this issue and merge the cases together.
Example: You have a case property called diagnosis, but want it to have a more specific name so you want to change it to hiv_diagnosis
CommCare will treat these as separate case properties.
App Structure
When you change a case property name there are number of ways that the app configuration can silently fail. For example, imagine that diagnosis is updated in a registration form. You make the change in the case configuration page. This will not update the spelling of the case property anywhere else, and you will need to manually update it anywhere you refer to it, including:
Case List/Details
Form/Module Filters
In addition the change will happen retroactively. That is if you make the change then all of the old cases pre-update will have the case property with the old name. This can be fixed by using the Case Importer tool.
You can also use the Case Summary (in the App Builder) to make sure that your case property is loading/saving correctly.
Data
Again if you make a change without updating old cases you will have two sets of case properties - in this case diagnosis for the old cases and hiv_diagnosis in the new cases.
Alternatives
If a case property is spelled wrong we strongly recommend you just give it another name when doing the export.
CLICK TO EXPAND
While not an issue with the app design, it is important to understand that if your users update to a new application version they may not be able to access saved or incomplete forms that were created with the older app version. This is particularly an issue if the form structure has changed significantly.
In order to prevent any issues, users should be encouraged to submit any incomplete forms before updating, and should be aware that it is possible that it will no longer be possible to view old saved forms.