Geospatial Features - GPS, Maps, and Distances

CommCare has several geospatial features that allow for calculating distances between geographic coordinates, sorting case lists by distance from you, and displaying case location data in a map view. 

76869276-76495580-683e-11ea-8db2-cf22ebf9a523.png

GPS

CommCare supports two options to capture GPS locations: automatically, using the Automatic GPS Capture feature, and manually, through the use of the GPS question.

Automatic GPS Capture

This feature will allow forms filled out on CommCare for Android to automatically capture a GPS location with no intervention from the mobile user.  

This feature includes the following functionality:

  1. Automatically capturing GPS from a form. This will include the location data as part of the form’s metadata

  2. If unsuccessful at getting a GPS location accurate with in 10 meters after 2 minutes, GPS location capture stops (to manage battery life). Below version 2.29 the cutoffs are 5 meters and 5 minutes, which caused issues with battery life.

  3. Configuration of automatic GPS capture on a per-app level (turn it on and off per application)

  4. Configuration of automatic GPS on a per-form level (if not set at the app level, can individually configure it per form)

  5. Ability to access the automatic GPS captured data in a form export

  6. Starting to capture GPS as soon as a form that has automatic GPS capture enabled is opened

When a form is set up to automatically capture a GPS location, it will do the following:

  • Upon opening the form, the phone will attempt to get a GPS location. Users will be able to fill out data in the form while this is occurring

  • Once the phone has captured a location of an acceptable accuracy (10m), the location will be recorded as part of the form’s metadata

  • If a GPS location is not captured to the acceptable level of accuracy within 2 minutes (the time threshold), the current location will be recorded as part of the form’s metadata.  This behaviour is to save battery life on the device.

  • If the form is completed prior to the time or accuracy threshold being met, the current GPS location is recorded as part of the form’s metadata.  If there is no location information, a blank will be recorded.

Things to Note

  • The phone's GPS must be on for this feature to work. If it is switched off, CommCare will prompt the user if they want to turn on GPS. This message will pop up each time they enter a form for which auto capture is on.

  • The accuracy reported with every location actually comes with a 68% probability, see details on Android's Accuracy here. So an accuracy of 4000m doesn’t mean the location is within 4000m, it means that there’s a 2/3rd chance that the correct location is within 4000m and a 1/3rd chance that it isn’t.

Setting up Automatic GPS Capture

  1. Make sure that the application is running the appropriate versions of CommCare. Both the client and the application content must be updated:

    1. The CommCare client version from the Play Store must be 2.14 or greater. See https://dimagi.atlassian.net/wiki/x/vQXKfw.

    2. Your application version on the settings page for your application must also be set to 2.14. More information found at https://dimagi.atlassian.net/wiki/x/bzPKfw.

  2. Turn on Auto GPS Capture on CommCareHQ

    1. For all forms (when GPS capture will be active in all forms in an application)

      1. Navigate to the application settings page

      2. Under Android settings check the box for "Auto Capture Location (all forms)" and save

    2. For one form (when GPS capture will be active in only some forms in an application)

      1. Navigate to the Settings tab of the specific form that needs to auto capture GPS location

      2. Under Advanced, click the check box "Auto Capture Location" and save

  3. Make a new version of the application and deploy it to a phone.  The phone will now attempt to automatically capture GPS locations for the forms you have selected.

Configuration at the app level

Configuration at form specific level

Configuration at the app level

Configuration at form specific level

 

956b5b84-285a-46b0-909a-5e7fa2c21bf3.png
Auto Capture Location - App Level

 

 

 

Saving Automatic GPS data to case

Automatic GPS data is saved to the form’s metadata by default while some use cases require the GPS data to be saved to the case, such as features like case list maps and distance sorting (a preview feature).To save GPS data to a case, first ensure Auto Capture Location is enabled on CommCare HQ so that the GPS data is available to reference from the form’s metadata. This value can be saved on to a case with following steps :

  1. Open the form in the CommCare HQ App Builder where you want to save the location.

  2. Add a hidden value question and configure the calculation expression to "/data/meta/location".

  3. Save the hidden value question to the case within the Case Management tab of the form settings.

 

Exporting Automatically Captured GPS Data

One can use the standard Export Forms routine to export GPS data. To do so, select Show Advanced Settings in the Export Settings page and then include the Location field. To further break down the location and include latitude, longitude, altitude and accuracy in separate columns of the export, tick the Expand Checkbox Questions box.

Manual GPS Capture

To enable the manual GPS capture the first step is to add a GPS question to the form. To do that follow the steps below:

  1. Go to CommCareHQ and navigate to the form where GPS coordinates need to be captured

    1. This should enable form builder

  2. In the middle section of the form builder, go to the top right corner and click Add Question, move the cursor over the Advanced submenu and select GPS

  1. To understand how to capture GPS coordinates using the GPS question, follow the steps described here

Understanding GPS Data

Deciphering GPS Data

GPS data in CommCare is captured in decimal degrees. It has 4 components, separated by a space character:

  1. Latitude: in decimal degrees

  2. Longitude: in decimal degrees

  3. Elevation: in metres, and

  4. Precision / Accuracy: in metres

For example, a data point collected in CommCare and viewed through form data may look something like 25.615311244889146 85.08323017699811 28.44 20.0, where:

  1. 25.615311244889146 specifies the latitude in decimal degrees

  2. 85.08323017699811 specifies the longitude in decimal degrees

  3. 28.44 specifies the elevation in metres, and

  4. 20.0 specifies the precision in metres.

Mapping the Data

GPS data in CommCare is captured in decimal degrees. It is easy to make a map using Google Earth.

Google Earth requires a kml file. You can generate a kml file using an Excel file and a free converter like this one.

  1. Export data from CommCareHQ into Excel

  2. If all of the coordinates are in one cell, select the GPS coordinates field and use the Excel "text to columns" feature to separate the GPS field into its 4 component fields (latitude, longitude, elevation, precision). Alternatively, use the following formulas to extract latitude and longitude:

Latitude = LEFT(H2,SEARCH(" ",H2,1))

Longitude  = MID(H2,SEARCH(" ",H2,1)+1,SEARCH(" ",H2,SEARCH(" ",H2,1)+1)-SEARCH(" ",H2,1))

In this example, H2 is the cell that contains the full GPS coordinates. Replace H2 with whatever cell contains the GPS coordinates

  1. Copy the latitude and longitude fields into a clean Excel file. The first number is latitude. Label the two columns in the top rows at latitude and longitude respectively.

    1. For a basic map, elevation and precision columns are not needed.

  2. Create additional columns with other information needed in the map (i.e. name, ID, etc.)

  3. Save the Excel file as a xls (Excel 97-2003 Workbook) or xlsx.

  4. Upload the file into the converter.

  5. Open the resulting kml file in Google Earth.

Extracting GPS Components

The location returned by a GPS question is a string composed of the latitude, longitude, altitude and accuracy, each separated by a space. 

If all of the coordinates are in one cell, the following expression will extract the latitude from the location: if(/data/location = '', '', selected-at(/data/location, 0))

Similarly, for longitude: if(/data/location = '', '', selected-at(/data/location, 1))

Calculating Distances Between GPS Coordinates

CommCare now supports distance calculation via the distance XPath function. For example, the following expression returns the distance between two locations in meters:

distance(/data/location1, /data/location2)

Since the distance function returns -1 if either argument is an empty string, a more robust way to display the distance is shown below:

if(/data/location1 = '', '', if(/data/location2 = '', '', distance(/data/location1, /data/location2)))

GPS Troubleshooting & Limitations

Limitations to GPS

GPS positions provided via android devices are generated using multiple different methods, resulting in highly variable performance. Performance depends on the device, the cell network, availability of GPS satellites and line of sight to these satellites (GPS signals).

GPS devices need 3 pieces of data in order to find the current coordinates:

  • Almanac data: very coarse satellite position information, it helps determine which satellites are available for positioning.

    • Good for 4 months and taking 12.5 minutes to download from a satellite; updated every 6 days

  • Ephemeris data: fine-grained satellite position information, crucial for determining the exact position of a satellite at any given time.

    • Good for 4 hours and taking 18-36 seconds to download from a satellite; updated every 2 hours

  • A lock on 3+ satellites 

A "cold” start happens when the device was off for more than 4 hours and new ephemeris data needs to be downloaded. A "warm" start happens when the ephemeris data is still valid and the device simply needs to get a lock on satellites. A device is "hot" if it has that lock. To accelerate the "time to first fix" (time from turning on the device to getting coordinates), a variety of techniques get used: (Assisted GNSS).

  • A-GPS or assisted GPS 

    • Mobile Station Based (MSB) - the phone downloads ephemeris data from the cell towers

    • Mobile Station Assisted (MSA) - the phone sends its captured GPS data to the cell tower, letting the tower perform the calculations in a later stage

  • The phone uses cell tower triangulation, totally ignoring the phone's GPS functionality - accuracy is usually reported as being more than 1000m. 

Some phones have incomplete GPS hardware, requiring a cell network to function - they depend on MSA as described above. The quality of the GPS antenna will affect how quickly a device will get a lock; so will trees, tall buildings or being inside concrete structures. Stand-alone devices usually have better GPS hardware and antennas.

Finally, new devices often support other navigation system such as GLONASS (Russia) and GALILEO (European Union), in addition to GPS. GLONASS and GALILEO are networks very similar to GPS and when combined, improve satellite locking and location accuracy. GLONASS particularly excels in high latitudes (very south or very north of the equator).

Speed and accuracy of getting a GPS lock is highly variable. For example, the Samsung Galaxy S10 has relatively good GPS hardware, including GLONASS, GALILEO and A-GPS support. However, it will still have issues when surrounded by concrete and getting a satellite lock will take longer when outside of cell network data coverage. It's not possible to "boost" the signal. However, it is possible to ensure that the GPS unit is at the very least "warm" and thereby reduce the acquisition time by making use of activity tracking apps such as the Google Fit app. Making sure that the app is running in the background ensures that the GPS is continuously accessed and therefore “warm".

GPS FAQs

Generally, what is "accurate" in GPS captures?
There are two accuracy thresholds. 1600m is "acceptable" and 10m is "good." 

How does GPS capture differ between using the GPS question in a form and the autocapture form/app setting (aside from how it shows up in form exports)?

Aside from the fact that for the GPS question, the user has to trigger the GPS capture process, whereas for autocapture that happens automatically, there is no difference, notably in things like the accuracy of the capture or how long CommCare tries for the capture (2 minutes in both cases). For GPS questions, once the device gets a location that's "acceptable" it gets displayed to the user, but the GPS will keep querying until it reaches a "good" value (as it gets more accurate values, it'll keep updating the screen that says "Your location is ABC, do you want to save?"). For auto GPS capture, "acceptable" doesn't matter, the GPS just keeps querying until it hits a "good" value (or 2 minutes pass). So even if it never hits a "good" or even "acceptable" value, after 2 minutes, it saves whatever best value it has, regardless of how rough it is, notably because the accuracy is also stored with the location. 

Does the CommCare version matter? I.e., is GPS capture via the question worse in v2.10.0 than in v2.26.0? Is that different for GPS via autocapture?

Accuracy shouldn't depend on CommCare. It will depend on location, hardware, and proximity to wifi and mobile networks.

What happens if you move around after submitting the form? Say, drive in a car for the remaining 1.5 minutes after submitting a 30 second form? Does it try to find the location you submitted the form at (doubt this is possible?)?

It uses the last location acquired during form entry. I.e., it doesn't update once you hit submit on the form.

Distance Calculations in CommCare

Distance calculations in CommCare

As of CommCare 2.26, the distance between two geographic coordinates (latitude and longitude pairs) can be calculated using the distance() function. This function takes in two geopoints and returns the distance, in meters between them, taking into account the curvature of the earth's surface.

Displaying and sorting by 'distance from current location' in the Case List 

If the case data contains location data, it is possible to display and sort the case list using distance from the current location. This is accessible via the 'Distance from current location' format of the case list display properties.

Calculating distance traveled by mobile worker

For advanced users: in theory, it is now possible to determine how far a mobile worker has traveled over the course of a given time period by storing a few additional parameters in the case data and using the 'distance()' function to aggregate that data. You would need to store the total distance traveled and the location of last form submission in the case data and update those values accordingly on each new form submission.

Maps

Maps in Case List

If the case list uses the 'Address' display property then the View on Map option will be available in the case list settings drop down menu. Selecting this will display the case list in a map view. While an internet connection is needed to download the map from Google Maps, it appears that those maps are cached and available, even without an internet connection, for some time afterwards.