CommCare supports a strong evidence base of peer-reviewed publications. Together, these studies demonstrate CommCare’s positive impact on strengthening frontline healthcare systems, frontline worker capabilities, and client results. This resource lives as an online, up-to-date repository, with new studies being shared and published frequently.
In addition to CommCare's evidence base that speaks to its impact on frontline programs, users, and clients, there are also a number of third-party technology evaluations of CommCare. Many of these include comparisons of CommCare against other digital systems in terms of features, pricing, reach, maturity, security, etc.
Summary
A designated Global Good, CommCare has been evaluated in comparison to other digital systems across a variety of factors - including its product capabilities and reach. This is in addition to more than tens of peer-reviewed publications RCTs in https://dimagi.atlassian.net/wiki/x/bzDKfw measuring CommCare's impact on frontline programs, frontline users, and client outcomes.
Capabilities: Various research organizations have evaluated CommCare against other digital systems for specific sector uses, with a focus on its feature set. CommCare has come out as a top 1 or 2 tool for agriculture (compared against 17 tools total), Ebola response (58 tools total), and COVID-19 (9 tools total).
Reach: Other researchers have conducted broader landscape assessments of entire industries to better understand the reach of tools. These found that CommCare is both the most popular mobile platform among frontline health workers in developing countries (out of 140 studied programs) as well as the most commonly used for nutrition-related digital interventions (16 out of 53 programs).
Maturity: An analysis found that CommCare is the 53rd most valuable open source repository on GitHub (out of 96 million). Another found that CommCare was one of two tools graded 100% for Product Maturity (out of 24 tools when this was written).
Cost Savings: Beyond its technical capabilities and reach, a subset of studies have looked at CommCare’s impact on program costs - some are which are covered in https://dimagi.atlassian.net/wiki/x/bzDKfw. Two separate studies found that launching CommCare would save the state of Colorado $15 million over 3 years, and between $830,000 and $1.7 million a year for a government program in Burkina Faso.
Featured Third-Party Technology Evaluations
Date | Evaluation Firm | Evaluation Focus | High-level Summary |
|---|---|---|---|
General Digital Public Good Repository | Digital Public Goods | Capabilities | The Digital Public Goods Alliance is a multi-stakeholder initiative with a mission to accelerate the attainment of the sustainable development goals in low- and middle-income countries by facilitating the discovery, development, use of, and investment in digital public goods. Digital Public Goods (DPGs) have been successfully reviewed and found to meet the DPG Standard receive a DPG icon. You can see CommCare's Digital Public Good profile here. |
September 2022 USAID Global Health Supply Chain Landscape Analysis | USAID | Capabilities | "Most technology solutions in global health are not designed with the community health worker (CHW) in mind. USAID Global Health Supply Chain Program-Procurement and Supply Management (GHSC-PSM) project conducted this landscape analysis, which is the first of its type. Along with the analysis is a supporting technical brief highlighting five mobile applications that government entities and local organizations can deploy for CHWs to capture supply chain data easily, report and request stock, and successfully provide services to their communities. Also included in the analysis and the technical brief are common and unique features of the mobile apps to enable countries and organizations to decide which application is most suited based on their programmatic and country needs." |
September 2021 Digital Impact Alliance Catalog of Digital Solutions | Digital Impact Alliance | Maturity Reach | The Catalog of Digital Solutions is an interactive online tool providing data and resources for donors, governments, procurers, and implementers to support their digital initiatives. CommCare received a 100/100 on Product Maturity, very active repository information and 178 out of 3226 projects listed on the Catalog use CommCare. |
June 2021 CommCare in Burkina Faso: How an Existing App at Scale was Adapted for COVID-19 | Bill & Melinda Gates Foundation - Exemplars in Global Health | Maturity Capabilities | In this case study, members of the Bill & Melinda Gates Foundation's Exemplars in Global Health team describe the implementation story of CommCare in Burkina Faso and its adaptation during the COVID-19 pandemic. At the end, there is an assessment of performance against the MAPS framework. This project was specifically selected by the Foundation as an "Exemplar" in Global Health. "The sustained success of IeDA can be attributed to a strong partnership between the government and TDH and many other organizations, such as Dimagi, which have assisted in the operations, research, and funding for the project....Even before the COVID-19 pandemic, CommCare was a rare example of a digital health tool with widespread use across sub-Saharan Africa. Burkina Faso’s decade-long experience with one of the largest digital health projects in sub-Saharan Africa demonstrates that a rapid response to an emerging health crisis is possible when a strong digital infrastructure exists." |
May 2021 Benchmarking of Mobile Data Collection Solutions [CartONG] | CartONG | Capabilities | "The evaluation of the eight final products is the result of the assessment of 39 key features. For each solution, a presentation of the tool is given, highlighting the strengths and weaknesses. Finally, the pros and cons table summarises the assessment while giving indication on the key features a reader can search for in a Mobile Data Collection solution." |
March 2021 Is Development Economics a Good Investment? Evidence on scaling rate and social returns from USAID’s innovation fund [USAID DIV] | Members of USAID's Development Innovation Ventures team | Cost Savings | "The social return for (Dimagi's intervention, ) the attendance monitoring innovation...returned over $24 per dollar invested by DIV." (Row 484) |
February 2021 Procuring Differently: How Colorado Used User Research and Active Vendor Management for COVID-19 Technology | Colorado Department of Public Health and Environment Georgetown University | Capabilities Cost Savings | "As part of an effective pandemic response, the State of Colorado needed a comprehensive digital solution for case investigation and contact tracing, which has been used by local public health agencies...A vendor (CommCare) was selected in just 19 days and $15 million under budget." "In the end, one vendor stood out as more user-friendly, with terminology and workflows that felt more familiar and natural to public health. This solution would also require less configuration and would allow for a simpler, faster, and more transparent contracting process, in addition to being the more affordable option. The process also found that over a three-year period, working with the selected vendor would save the state a minimum of $15 million. This would allow the State of Colorado to reinvest some of the savings to add new features and support implementation." |
December 2020 Using Community Health Workers and a Smartphone Application to Improve Diabetes Control in Rural Guatemala | University of Wisconsin School of Medicine, Stanford School of Medicine | Capabilities | "Earlier versions of the application used Enketo (Enketo LLC) web forms for the user interface and Ona (Ona Systems) for data storage and management...While both platforms allow for offline data collection and have branching logic capabilities, permitting the delivery of algorithmic clinical decision support, we transitioned to CommCare because it has more advanced capabilities for storing and modifying longitudinal data, includes robust database functions, and allows for application updates to be pushed to end-user devices." |
September 2020 Using Digital Tools to Strengthen Nutrition Service Delivery: An Overview | USAID | Reach | USAID Advancing Nutrition, the Agency’s flagship multi-sectoral nutrition project, published a report, Digital Tools for Nutrition Service Delivery, that explores how 53 digital solutions have supported or are supporting health and agriculture providers to deliver nutrition services in low- and middle-income countries. In addition to supporting the largest proportion of digital tools for nutrition, CommCare was the platform most commonly used for nutrition-related digital interventions. The report states "This wide use merits further investigation to see whether features of this platform are better designed for nutrition activities in particular." |
August 2020 The Global Health Innovation Index | USAID | Maturity | The Global Health Innovation Index was created to compare more than 150 early stage and scaling innovations across the entire USAID and Saving Lives at Birth (SL@B) global health innovation portfolio, irrespective of the health challenge they are addressing. The innovations included in this Index demonstrate strong early evidence or proven results across all four index criteria: health impact, demand and sustainability, organizational and/or partner capacity, progression to scale. CommCare was one of nine innovations selected. |
June 2020 Global Goods Maturity | Digital Square | Capabilities Maturity | CommCare is recognized as a Global Good by the World Health Organization and Digital Square. This table reflects the maturity model baseline assessments and the latest reassessment of Global Goods funded through Digital Square. |
2020 Software Architecture and Software Usability: A Comparison of Data Entry Form Designing Configurable Platforms | University of Malawi | Capabilities | "When creating a data entry form in DHIS2 one has to go through the maintenance app, to create a data element and then create a data set, after which they can proceed with designing a form...In CommCare, everything is done within the same window. Literature shows that presenting data and information on one page increases systems efficiency." |
2020 Digital Solutions for COVID-19 Response: An assessment of digital tools for rapid scale-up for case management and contact tracing | Johns Hopkins University | Capabilities | The Bill & Melinda Gates Foundation requested the Johns Hopkins University conduct a rapid assessment of digital platforms that have an established presence in several LMICs and either have or could be rapidly reconfigured to address COVID-19 related case management and contact tracing needs. Nine core digital platforms were identified for this assessment: CommCare, Community Health Toolkit (CHT), DHIS2 Tracker, Go.Data, ODK, OpenSRP, RapidPro, SORMAS, and WelTel. Based on the assessment, CommCare and DHIS2 Tracker stand out in the context of patient triage, referral for testing, and contact listing and follow-up. |
December 2019 Software Global Goods Valuation Framework | USAID | Capabilities | This evaluation framework put out by USAID describes the differences and value between CommCare, OpenMRS, and IHRIS. |
May 2019 Top 100 Most Valuable GitHub Repositories | GitHub, U°OS | Maturity | CommCare was assessed at the 53rd most valuable repository on GitHub by an independent analysis, out of a total of 96 million repositories. |
May 2019 The Global Goods Guidebook | Digital Square | Capabilities | CommCare is recognized as a Global Good by the World Health Organization and Digital Square. |
2019 Impact Tracker Tech Catalog | Kopernik, Plan International | Capabilities | The technology evaluator Kopernik assessed 17 digital data collection tools, and scored CommCare with the highest score. This 2019 update of the Impact Tracker Catalog catalog has been implemented by Kopernik in partnership with Plan International. |
November 2018 Economic impact study of the Integrated eDiagnosis Approach (IeDA) for the management of illness in under-five children at the primary health care level in Burkina Faso | London School of Hygiene & Tropical Medicine | Cost Savings | "An economic impact study projected that yearly savings from IeDA were between US$830,000 and US$1.7 million as a result of reduced training times for health workers and reduced paper consumption at PHC facilities." |
October 2018 Assessing the Concepts and Designs of 58 Mobile Apps for the Management of the 2014-2015 West Africa Ebola Outbreak: Systematic Review | Research associated with Helmholtz Centre for Infection Research (HZI) | Capabilities | A third-party study evaluated 58 technologies that were used as part of the 2014 - 2015 Ebola response efforts in West Africa. The study found that, of the 58 tools, only CommCare and one other tool supported all 7 technical characteristics and 4 key functionalities relevant to Ebola outbreak response. |
September 2018 Evaluation and Development of Android mHealth Application for Community Health Workers in Malawi: Comparison of CommCare and DHIS2 Tracker | University of Oslo (Master Thesis) | Capabilities | In this study, the author built the same digital health application in DHIS2 as well as in CommCare, and provided their evaluation of both tools across a variety of areas. |
December 2017 At-Scale Evaluation of Digital Data Collection Apps (DDCAs) in ACIAR projects: Mobile Acquired Data phase 2 (MAD 2) | Australian Centre for International Agricultural Research | Capabilities | ACIAR funded a study designed to determine the feasibility of using a single, off-the-shelf mobile data collection solution for future use. In addition to evaluating the technology features for the 17 apps in the study, the research team also analyzed how mobile data collection would impact that type of data collected, and how the various users involved in a project would utilize the data. |
2016 Mobile Technology in Support of Frontline Health Workers | Johns Hopkins University | Reach | In a study by Johns Hopkins University's Global mHealth initiative, CommCare was found to be the most popular mobile platform among frontline health workers in developing countries. The report summarized data from more than 140 frontline-supported mHealth projects. |
2016 Open Source Solutions for Data Management | UN Commission on LifeSaving Commodities | Capabilities | This report from the Coalition of Life Saving Commodities evaluated three open source supply chain systems, including CommCare, OpenLMIS, and DHIS2. |