Coverage Analysis on Android 14 and iOS 18

Posted on by

As part of the CATCH project, we have recently done a series of our so-called “Sampling Coverage” tests where we test the sampling of data in CARP Mobile Sensing. There are quite a long series of tests done, which are documented in this Issue #475. But the TL;DR version of this testing is:

  • Android 14 and iOS 18 will aggressively try to pause or kill your app, if not used by the user (hence “background sensing” is challenging)
  • If the app is used regularly by the user (e.g., by filling a survey hourly), data collection is improved
  • If location tracking is configured for “high” accuracy and a very low (zero) distance filter, the sampling of location data helps to keep the app and hence general data collection running. However, when the phone is sitting still – like during the night, the app and hence sampling is stopped.
  • If the app is connected to a wearable device (like a HR monitor) using BLE, this will keep the app running in the background and give a 100% sampling coverage.

Below are some details and data.

Android 14

Testing was done on a Samsung Galaxy A54 running Android 14 SDK level 34. The CARP Studies app was used for testing and coverage analysis was done on the data stored locally on the phone.

Test #1 – 31 hours with and without the Polar HR device

Below is the sampling coverage for the 1st test.

From this test we can observe that:

  • The app is kept alive while the HR is samples and the BLE device is connected (07-11:00 to 07-18:00). We see a sampling of 100% as measured by the heartbeat measure, for example.
  • Once the HR device is disconnected during the night, sampling is reduced to 13-17%
  • The next morning when surveys are filled in once per hour (08-09:00) sampling is increased but only to 50-60%
  • Once the app is no longer used for surveys (09-14:00), sampling is decreased to 20-30%

Note that:

  • The phone was not charged during this period (which is something that affects the background mode).
  • The many missing weather and airquality measurements are due to the phone not being on the Internet most of the time. The phone does not have a SIM card.

Test #2 – 20 hours with high location tracking

In the 2nd test, location tracking was increased to high precision, zero distance filter and one second sampling interval. The Polar HR device was not connected. This provided this result:

From this picture we can observe:

  • There is a good coverage during day time when the phone is used and moves around
  • Mobility features are being sampled and calculated on a regular basis
  • There is still no sampling during the night when the phone is sitting still
  • heartbeat is below 100% during the day, which indicates that the app is not running all the time.

iOS 18

Testing was done on a iPhone 12Pro iOS level 18. The CARP Studies app was again used for testing and coverage analysis was done on the data stored locally on the phone.

We observe:

  • The connected HR device keeps the app alive also during the first night.
  • The app is inactive during the second night since the HR device is disconnected.
  • Using the app to fill in surveys once per hour seems to keep the app running. But if there is no user interaction, no data is collected, such as during the 2nd night.
  • A few mobility features are collected even though location sampling was at a low accuracy (like test #1 above)