Rebuilding Project Space Components Between CommCare Clouds

Rebuilding Project Space Components Between CommCare Clouds

In some cases, a project may be interested in moving core elements of one project space to another. Before undertaking such a large task, users are encouraged to check whether this process is a good fit for their requirements. Step by step instructions are provided for the elements that can be recreated in a new project space.

Overview

When signing up for CommCare, you can select the cloud location that best fits your organization's needs. You can see a full list of cloud locations here: https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/3101491209 Once created, a project space cannot be transferred from one hosting location to another.

CommCare does not support directly transferring a project space to a new cloud. However, programs may be able to achieve similar outcomes by manually recreating parts of the original project space in a new one on a different cloud location. This involves exporting and importing core components, such as applications, case types, and data structures as well as importing active case data.

While this process is technically feasible, it is complex and labor-intensive. Programs should be cautious and only pursue this approach if they have the technical capacity to manage the risks and ensure data integrity throughout moving components.

Please note: Form data cannot be migrated between project spaces.

There isn’t a one stop solution, but you can move components from one project space to another in CommCare. This page details how.

What Can I Move Between Project Spaces?

The table below gives a high-level overview of what CommCare components you can or acnnot move between project spaces. These are described in greater detail below.

Ability to Move

Component

Ability to Move

Component

Can’t Move

  • Forms

  • Historical SMS

Export / Import

  • Applications: Mobile apps and forms (XForms) used for data collection and case management.

  • Users, Locations, Groups : Mobile workers and web users, organization structure and hierarchy, user groups used for case sharing or reporting.

  • Lookup Tables: Static reference data used in forms.

  • Case Data: Case records (clients, entities). Requires mapping to new owner_id, user_id, location_id, and parent_id (if using parent/child cases).

Manual Recreation

  • Reports : Saved report configurations.

  • Data Exports : Saved export configurations for forms and cases.

  • Data Forwarding : API forwarding configurations (integrations).

  • Messaging : SMS and reminder rules.

  • Roles & Permissions : Roles assigned to web users.

  • Project Settings : Project settings like languages, timezones, feature toggles, branding

Is This Process Right For Me?

The first question to ask is “Is this process worth the effort?” If you can avoid this process or create a new project space, we strongly recommend doing so. It is usually much easier and more reliable to open up a new project space when you begin a new project. This is especially true for smaller or newer projects, where starting fresh is often the better option. If you are seriously considering moving components over to a new project space, look at the instructions below and try to estimate the amount of time and coordination it would take for your team to make this move. That may help you weigh the cost / benefit of whether this process is worth the effort.

If you do decide to move components--whether to change cloud locations or merge project spaces-- make sure you’re prepared for the effort involved. This process requires manual work, technical skill, and careful coordination. While it preserves live case data, it does not include historical form submissions. You’ll need to manually set up components, run multiple test migrations to ensure data accuracy, rebuild any form-based reports, reconfigure integrations, and clearly communicate with field teams so they can reinstall apps and follow the new timelines. All of this has to be done in careful sequence, so if you have many team members lending a hand, they will need to work together closely to ensure each step happens in the right order.

This process may work for you if…

This process may not work for you if..

This process may work for you if…

This process may not work for you if..

  • You want to move active case data, users, and locations.

  • You are comfortable using advanced Excel skills (lookups, formula-based ID mapping, bulk data cleaning).

  • You can manually update and redeploy apps in the new project space.

  • Your field teams can uninstall and reinstall their mobile apps relatively easily as part of the transition.

  • You need to move form submissions or SMS/IVR messages—these are not supported.

  • Your reporting or workflows rely heavily on historical form data.

  • You require a fully automated migration. Manual effort is needed to set up the new space, migrate data, and reconfigure apps.

  • You are lacking for time or technical skill to do this work.


Step-by-Step Instructions

This guide provides step-by-step instructions to help organizations easily transition individual components of their CommCare project space from one cloud location to another, on their own.

Getting Set Up

In the following instructions, we’ll be referring to both the “Source” project space and the “Destination” project space. The source project space is the project space that you are moving away from-- the old space you are trying to shut down. The destination proejct space is the new project space you are setting up, where you want your users to start working.

In order to get ready to move components from one project space to another, make sure you take the following steps:

  1. Sign into your account and open the source project space.

  2. Sign into your account and open the destination project space.

  3. Prepare a secure, local folder to save data files from CommCare during the process.

Below is a summary of what you’ll be able to transition.

First, Understand What You Can’t Transition

Forms (Raw Submissions)

Individual form submissions (XML) cannot be recreated in the new project space. Form history and metadata will remain on the original cloud location. If needed, you can optionally extract form data via the API or through form exports, but this is suitable only for archiving or reporting — it cannot be reloaded into the new project space.

Historical SMS

Historical SMS messages cannot be recreated in the new project space. If required, you can extract SMS history via the API for archival or reporting purposes only. SMS interactions will remain linked to the original cloud location and will not carry over to the new project space.

Elements You Can Import and Export Between Project Spaces

Applications

If you are copying an application between two project spaces on the same cloud, follow the instructions to https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143955454 .

If you are copying an application between two different cloud locations, follow the instructions below to copy applications from the source space to the destination space.

https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143955454/Copy+or+Delete+an+Application#Copy-an-Application-to-a-Different-Server


Users, Locations and Groups

Mobile users, locations and case sharing groups are interlinked in CommCare and therefore should be moved in tandem. This is divided into two sections - data preparation and data upload.

Data Preparation

1. Instructions for Mobile Users

  1. Use the Bulk Download Mobile Workers feature to download all mobile users from the source space. Repeat the same process in the destination space to get a blank template.

  1. Copy paste the exact names in the destination space file from the source space file downloaded above.

Ensure that the user_id column is blank in the destination space file as CommCare will automatically assign new IDs to these entities.

2. Instructions for Organization (Locations)

  1. Replicate the exact organization structure from the source space in the destination space. See https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2841739298/Setting+Up+Organization+Levels+Structure#Step-1%3A-Set-Your-Organization-Levels for step wise instructions. You will need to rebuild the entire structure (including all of the same settings) for every level in your organizational hierarchy.

  2. Download the organization structure from the source space and remove all location_ids from the Excel file. Remove rows for any locations that are no longer active. Save all changes to the Excel file.

Ensure that the location_id column is blank in the destination space file as CommCare will automatically assign new IDs to these entities.

3. Instructions for Case Sharing Groups

  1. Create case sharing groups in the destination space with the same values as in the source space. See https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143955669/Mobile+Worker+Groups#Create-Groups for step wise instructions.

Note that case sharing groups cannot be downloaded separately in CommCare. They are mapped in the mobile workers excel file.

Data Upload

  1. In the destination space, take the Excel file you just cleaned, and conduct a bulk upload. See https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2841739298/Setting+Up+Organization+Levels+Structure#Bulk-Upload-your-Organization-Structure for information on bulk uploads. This will create all of the locations in your new project space.

  2. Upload the excel file prepared for the destination space for mobile users in the destination space. See https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143955384/Create+and+Manage+CommCare+Mobile+Workers#Use-Bulk-Upload-to-create-multiple-mobile-workers-at-once

  3. Cross-check the mapping of mobile workers to their respective locations in the source space and replicate for all mobile workers in the destination space.


Lookup Tables

  1. Download existing lookup tables from the source space. See https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143955074/Lookup+Tables#Downloading-Table for step wise instructions.

  2. Duplicate the excel file and save as lookup table files for the destination space.

Ensure that the UID column is blank in the destination space file as CommCare will automatically assign new IDs to these entities.

  1. Upload lookup tables in the destination space. See https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143955074/Lookup+Tables#Upload-Table for step wise instructions.


Importing Case Data

Why Import Case Data?

When rebuilding project components in a new project space, the main reason to import case data is so that your mobile users can seamlessly continue working with their same set of cases. Once you have imported all of the case data correctly, this means that your mobile users can login to their new application in the destination project space, sync, and see all of the data that they had on their previous project space. They can continue collecting data as they would have before, with almost no interruption to their day.

General Process

When rebuilding or importing data into the destination project space, you need to export all of your case data from the source space, and you need to import it into your destination space. When you import it into the new system, you need to update the case’s owner to ensure it gets assigned to its new owner in the destination space. The case owner will either be a mobile user, a location, or a case-sharing group.

In addition, we recommend saving the old case id of the preivous case, and the old owner id. This allows you to easily look at data from your source space and destination space and link records to each other (particularly if you are putting both set of data into a database).

Methods for Exporting and Importing

There are several ways that you can go about export and import the data from your source space into your destination space. Depending on the amount of case types that you need to bring over from your source space, there are two different methods you can use for achieving this.

Simple method: The first method involves simple vlookups, and is appropriate if 1) you are migrating a small number of case types (for instance, 1-3 case types), and 2) if all cases are owned by the same type of entity (e.g. they are all owned by mobile users, or all owned by locations).

Advanced method: The second method involves creating a main mapping file, and saving several different files within one folder. By utilizing Excel formulas, you are then able to map the correct case owners from all of your location, mobile user, and case sharing files into your case data files, preparing them to import. This method is much more complex, but works well if you have either several different case types to import to the destination (4+), or if you have different entities that own cases (e.g. one case type might be owned by mobile users, while another is owned by locations). In this situation, using the more advanced mapping file will save you time in the long run.

Simple Method

The instructions below outline the process for exporting and importing Case Data. We run through two different scenarios:

  1. Cases owned by mobile workers or case sharing groups

  2. Cases owned by locations

For more information on exporting cases in CommCare, please see https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143956156.

Advanced Method

The process for exporting and importing Case Data is quite extensive. Please click the instructions below to access this process.

See the video below for stepwise instructions on the Advanced method of case data import and export.

For more information on exporting cases in CommCare, please see https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2143956156.