Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

This page walks through building an Excel dashboard with data from CommCare. Similar to how Excel helps people build spreadsheets, CommCare enables anyone to build a free customized mobile app - no IT background required. After collecting data with your CommCare app (don't yet have an account? Sign up for a free trial here.) you can download your data and load it in Excel to create dashboards. 

...

Excel Dashboards require a CommCare Software Plan

...

Multiexcerpt include macro
macro_uuid785c6088-0e28-4857-9b71-fdd34d840424
nameSoftware Plan - Standard
templateDataeJyLjgUAARUAuQ==
pageCommCare Help Site Design Guidance
addpanelfalse

Note: Mac users -  Excel dashboard integrations are primarily designed for PCs. Mac users should consider data extraction methods such as the data export tool, manual exports, or the daily saved export.

...

  1. Requires upfront work by an administrator just once, and then is easy to maintain going forward.

  2. Refreshes automatically from CommCareHQ when you reopen the report (or set time based refresh).

  3. Allows users to filter the dashboard for information relevant to them without detailed knowledge of how Excel tables work.

  4. Enables an administrator to quickly make new reports and send to end users.

The intended audience for this page is a CommCare administrator who is familiar with CommCareHQ and has some experience in Excel. The end result will contain the following:

...

Download the

...

Excel Dashboard

...

A graphical dashboard

...

Individual pages with data:

...

User definable filters:

...

...

Getting Started

Excel Version:  This tutorial was written for Microsoft Excel 2010. Most of the functionality described is also available in earlier versions of Excel though may be called different names.

...

Step 1: Create Custom Export in HQ for the project

1. Proceed to the Data section of CommCareHQ.

2. Select Excel Dashboard on the left-hand side.

...

3. See https://dimagi.atlassian.net/wiki/x/5SbKfw for details on creating the data export:

a. Name your

...

export.

b. Choose Automatically convert dates for Excel.

c. Be sure to rename your Display columns to how you want them to display in your reports.

4. Save the export.

...

Step 2: Copy the Web Address of your saved export

...

  1. In the main Exports section you will see your saved daily export.

  2. Click Update Data button to generate the first report. You may need to refresh the page to confirm that the update is complete.

  3. Select the Copy Dashboard Feed Link button. 

  4. Select the Copy Dashboard Feed Link button again to copy the URL. This will copy the URL of the export file so you can paste into Excel in the next step.

    1. Alternatively, you could simply highlight the URL to copy it.

...

...

Step 3: Create the Excel Connection to your export

The next step is to create the file that will hold the reports.

1. In Excel, Create a New File.

2. You will have a blank Excel file now.

3. Go to a blank tab in your workbook. Rename that sheet "Raw Data".

4. Select the top-left cell (A1).

5. Now we will pull in the data from the Excel file.

6. Choose the Data menu.

7. Choose From Web in the "Get External Data" section (NOTE: If you are getting issues when trying to import data to the web, you may want to use From Web (Legacy) instead of the standard From Web. Please see Alternative Web Data Access Method for Step 3, below).

...

8. Be sure you have Copy link address from step 2 above so it is on your clipboard.

9. The "New Web Query" window appears.  

10. In the "Address" bar at the top, Paste the link you address you copied in Step 2 above.  It should be something like "https://www.commcarehq.org/a/excelreportingdemo/reports/export/saved/download/96c41fd6fb9be6119ad32467254cbc9a/".

11. Click Go.

12. The "Windows Security" window appears. Type your CommCareHQ username and password. If you are using two factor authentication, review the custom instructions in the box below. If you do not use two factor authentication, proceed to step 13.  

...

Panel
panelIconIdatlassian-info

...

panelIcon:info:
bgColor#EAE6FF

Special Instructions for Users with Two-Factor Authentication

This box is only for accounts with two factor authentication enabled. If you do not use two factor authentication, proceed to (13) below. 

  1. Click the "Advanced" radio button. The URL supplied from step 10 should remain in the URL field.

  2. Under the "HTTP request header parameters (optional)" field, add "Authorization". NOTE: This is not a value found from the dropdown options.

  3. For the field next to this header, type "ApiKey <username>:<API_KEY>". For example, if your username is "JohnDoe@google.com", and your API Key is 12345, the field should look like: "ApiKey JohnDoe@google.com:12345"

...

dab23d4d-bc99-4b5c-b7dd-bf26793159f4.pngImage Added
  1. Note: If the wrong username or API_KEY are provided, a retry dialog will display like the following:

...

da0dd289-fc32-49de-9c27-66c7ec57d229.pngImage Added

With API Key verification, it is necessary to click back to fix the error and try again. Both the "Basic" and "Web API" options will not work.

13. The export will be displayed to you in the Web Query Window.

14. Click the second Yellow Arrow Image Modified above the table of data (not the very top one). This will select all the CommCare data in the export.

15. Click Import.

...

16. The "Import Data" window appears.

17. Click Properties.

18. The "External Data Range" window appears.

a. In the top of the window, name your export "Raw Data".

b. Choose Save query definition.

c. Choose Enable background refresh.

d. Choose Preserve Cell Formatting.

e. Choose Adjust column width.

f. Choose Overwrite existing cells with new data, clear unused cells.

g. Choose Fill down formulas in columns adjacent to data.

h. Click OK.

...

19. Click OK again.

20. The data is pasted into the empty worksheet.

...

Alternative Web Data Access Method for Step 3

...

The first chart we will create is the Line Graph of Expected Delivery Date.  This will plot the number of women expected to give birth in each month, broken down by Clinic.

1. Create a new tab called "Expected Delivery Date"

2. Select the top-left cell (A1) in this tab

3. In the Insert menu choose Pivot Chart from the PivotTable icon

...

4. In the window that pops up, click  "Table/Range" text box

5. Type "raw_data" into the box. This references the data range we created in the step above and is a lot easier than selecting all the columns each time.

...

6. Click Ok.

7. This creates a blank pivot table. (If this doesn't work it's likely because the range is referencing only to the first tab. Go back to step 4.2, then click "Formulas" at the top, next to "Name Manager" click on "Define Name", enter "raw_data" under "Name" and make sure "Scope" is set to "Workbook" as this will ensure the reference can be used in different tabs, click OK)

8. Click the pivot table on the left

9. In the Field list on the right, do the following:

a. Drag Expected Delivery Date to "Row Labels"

b. Drag District to "Column Labels"

c. Drag District to "Values"

Note: If you don't see the "Pivot Table Field List" on the right, click the Options menu at the top and choose Field List in the top right.

10. This will be the result:

...

11. We now have the data we want but it needs to be formatted:

a. You can drag the graphical chart to the side to get it out of the way

12. Next, we need to display the dates as months instead of days

a. Right-click on any of the dates in the pivot table and choose Group

b. Select Months in the list the appears and choose OK

c. This groups all the EDDs into monthly buckets

...

13. The graph defaults to a bar chart, but you can change it:

a. Right-click the graph and choose Change Chart Type.

b. Select the line graph you'd like.

14. Set the table to update automatically.

a. Highlight the pivot table.

b. Choose the Options menu and in the top-left type the name "EDD Graph."

c. Click Options below where you just typed.

d. On the Data Tab select Refresh data when opening the file.

...

15. Finally, rearrange the layout by inserting rows above table to move it down and drag the chart to the top of the page.

16. You can add titles and legends with the Layout menu.

17. You're final result should look like this:

...

Step 6: Create Additional Pivot Charts

  1. Repeat all the steps in Step 5 as many times as you'd like to create more charts.

  2. In this example, see the screen shot for how the "Patients Received Vaccines" chart was made.

  3. Be sure to set all table to update automatically (Step 15 in the section above).

...

Step 7: Create Filters

1. Excel uses "Slicers" to enable users to filter the reports in a user-friendly way.

2. For the first pivot table you created do the following:

a. Select the chart you created.

b. Choose Analyze menu at the top.

i. You may need to select Options menu instead.

c. Choose Insert Slicer from the menus.

...

d. Select the items you will want to filter your reports by and click OK.

e. Drag the filters and resize them to fit the layout as you like.

...

3. Now we want this filter to apply to all charts in our Worksheet, so we connect the filters to the other tables by:

a. Select one of the filters you just created.

b. Choose Options then PivotTable Connections.

...

c. Check all the boxes for the other tables you created.

d. Now changing this filter will change the filters on all the tables you created.

e. Repeat this step for all filters.

4. Copy and Paste the filters to other areas of your Excel sheet if you'd like them to appear in more than one place

Step 8: Build the Dashboard with Copy-Paste

...

One of the challenges with Excel reporting is ensuring that users you send the reports to do not accidentally corrupt the data or people get confused with too many options.  We can mitigate many of these issues with two methods:

  1. "Protect" all data.

    1. Right-click each tab and choose Protect Sheet.

    2. Scroll to the bottom of the list and check the box for Edit Objects and Use PivotTable Reports.

      1. This will allow users to change filters but not edit the data.

      2. Do NOT protect the "Raw Data" tab or the data will not update automatically.

  2. Hide the Raw Data tab (if desired).

    1. If you'd like to hide the raw data tab, you can right-click it and choose Hide.

    2. To unhide, right click on any sheet name, choose Unhide. It would show you the list of hidden sheets and you can pick the one to unhide.

  3. Choose PivotTable option in Step 4.

    1. Instead of putting the raw data on a tab you can directly make pivot tables when importing.

    2. This means the raw data tab will not exist but the pivot tables will.

...

  • In the Export Dashboard you will find a check box to select all or specific exports

  • On checking the box, you will notice a "Delete Selected Export" button becomes available on the left side of the page

  • Click on the "Delete Selected Export" button and you will be promoted to confirm the bulk delete action

  • Once Delete Export is clicked it will bulk delete the exports selected at once.

...

Common Errors & Troubleshooting Steps

Users can unexpectedly encounter errors when setting up the Excel integration, which can be easily resolved by following this troubleshooting guide.

  1. Excel Dashboard integration feed link is disabled and my data is not getting updated

    1. Re-enable the disabled feed.

    2. Try using a different browser.

  2. If you face the issue seen in the screenshot below, attempt using “Basic” instead of “Anonymous” or use the From Web (Legacy) option instead:

    Image Added

  3. If you have Two-Factor Authentication enabled and are experiencing issues with your credentials (API Key):

    1. Navigate to your CommCare project space → My Account Settings → API Keys → Check if the API key is active and not expired. If it is expired, you will need to generate a new one.

  4. If your credentials are causing an error on your device not allowing the connection to be made, try setting up the connection on another device.

    1. A common issue is the absence of a Microsoft account on the device, which can prevent the connection from being established.

    2. In this case, check with your internal IT team to see if your Microsoft account might be causing issues with the Excel integration.