Open

Understanding system architecture and data migration

Partners transitioning from another solution to CommCare may feel the urge to transfer all their historical data into CommCare. However, in most cases, they are primarily concerned with the most recent updated values of the entities they track. This facilitates a smooth transition of future data collection from their previous tool to CommCare. In these instances, Dimagi strongly advises preserving all historical data in secure environments such as cloud-based data warehouses and only importing the latest data into CommCare to ensure uninterrupted future workflows. Historical data can be integrated with BI tools along with data from CommCare for business intelligence purposes.

The reasoning behind this recommendation

The architecture of technology systems varies, leading to differences in how data is processed across platforms. On CommCare, for example, any data collected via CommCare apps in the field are considered “form submissions” - in other words, frontline workers (called mobile users/workers in our ecosystem) login to their CommCare apps and open individual forms and submit them after collecting data for clients or entities they are tracking. Once these forms are submitted, depending on what data model has been designed for that specific workflow, CommCare can create new cases, update existing ones or simply save these submissions on the server. The same underlying principles apply for web apps (CommCare apps running on the browser with no offline capabilities) submissions as well. In other words, data submitted on CommCare may have multiple dependencies. As partners transition to CommCare, they are usually interested in shifting their current workflows to the new system after a certain cutoff date where they use the last updated values for the entities they are tracking, upload them to CommCare and continue tracking them through CommCare for the remaining life cycle of their projects.

Open

High Level Data Flow when switching from another tool to CommCare

Here’s an example:

Partner A is running a community health program tracking pregnant mothers. They are currently using Kobo Toolbox. Steps to follow:

1. When it is time to switch to CommCare, Partner A will download all their data into a secure environment. These can be robust data warehousing solutions like Snowflake, SQL databases like MySQL, cloud storage like OneDrive or SharePoint.

2. Partner A will create their data model on their CommCare project space, ensuring case properties are created for all the indicators they wish to track for their clients or other entities via CommCare. This will include building app(s) on CommCare with the corresponding case type, case properties and workflows required for future data collection.

3. Partner A will then use CommCare’s built-in case importing feature via Excel to upload the last updated values from their previous database into CommCare. This will require them to format the excel file with the corresponding column headers for all the case properties they wish to populate in CommCare.

4. Once the data is successfully imported into CommCare, Partner A will proceed to use CommCare for future data collection.

More Detailed Instructions

Approaches to consider

1. CommCare offers robust documentation for each of these steps and partners can easily follow instructions to get the latest updated values of their data from a previous tool into CommCare through self-service.

2. All users are highly encouraged to complete Dimagi Academy courses and go through the CommCare Help Site and User Forum to familiarise themselves with the platform. 

3. CommCare Onboarding - Dimagi’s Customer Success team is well positioned to support new partners in setting up CommCare to receive data from a previous tool. Please contact us at sales@dimagi.com for more information and the best suited plan for this support.