How does the DET work?
Example Use Case
Haik is a data analyst for a child immunization NGO specializing in tracking polio vaccinations in developing countries using CommCare. The increase of vaccination supplies and the training of competent healthcare workers to administer the vaccinations have significantly improved the child immunization program. More children are getting the vaccinations that they need. The program's success is leading to a considerable increase in the data collected. Haik needs to export data regularly to analyze and visualize it for the weekly reports. Manually exporting the data from CommCare HQ is not sustainable. Haik needs a way to export data in bulk in an automated manner. The CommCare Data Export Tool will help Haik. He can use the DET to pull data from one or more domains and combine data into one large data file, or database, for analysis. |
Using the DET
The data export tool does not change your data in CommCare. It simply exports the data into a raw format that you can use for data insights and analytics.
The DET requires you to download Python to your computer. You will use a command-line interpreter tool to run the commcare-export script from the Python script library.
The CommCare Export Tool calls the CommCare API to export the data. The configuration for what data is exported and how it is formatted are provided to the tool via an Excel configuration file.
The export can be in the format of a file (Excel, CSV, or JSON) or to a specified database.
Please note that the commcare-export tool offers no security features for dealing with PHI directly. If you intend to export PHI data from CommCare, please ensure the appropriate measures are in place to protect your data.
What is Python?
Python is a high-level, general-purpose programming language. It allows you to integrate systems more effectively.
It is used for:
Web development (server-side)
Software development
Scientific computing/ Data Science
System scripting
Python can connect to database systems and can be used to handle big data.
You need Python installed to use the commcare-export script. The script library is installed with Python. Installing the script allows the Data Export Tool to export your data from CommCare HQ.
What is a Query File?
A Query file is an Excel file (.xlsx) saved to the Python Scripts folder.
The Query file specifies which data objects should be requested from the API and which values should be stored when the API payload is received. All other values that are not specified are discarded and not stored in the results set.
One query file can contain many tabs and can combine case and form data queries.
It’s common to have a single query file with one tab for each case or form that you want to export.
A template query file is provided here for cases and forms.
You must provide the path to where the query file is saved in the commcare-export command.
E.g.,commcare-export --query /home/username/Documents/my-query-file.xlsx