Skip to content

SDK Performance Measurements

General

Network Bandwidth

The bitdrift Capture SDK, in its default configuration with all features enabled, consumes:

  • ~156 bytes of download network bandwidth per minute.
  • ~250 bytes of upload network bandwidth per minute.

Info

Follow the steps from the Measure SDK network traffic guide to measure the SDK network bandwidth usage yourself.

Notes

  • The default configuration includes the Session Replay and Resource Reporting features, both enabled, with a capture interval set to 3s.
  • Upload network bandwidth consumption increases as you deploy workflows that instruct the SDK to flush logs collected in buffers. The SDK customer has full control over what and how often logs are uploaded. This configuration can be changed on the fly and remotely using the bitdrift Capture frontend.

Disk Storage

The amount of disk space the SDK uses for storing logs is controlled via bitdrift Remote Control Plane (service) and can be changed at any time. Any changes to that value result in updating configurations of all connected mobile clients.

By default, the SDK is configured to use 5 MiB (~5.25MB) for storing collected network logs.

Notes

  • All disk space used for storing logs is reserved up-front, meaning that the SDK, configured to use 5 MiB of disk storage, takes up all that space immediately after the first time it's configured on a given device.
  • The greater the amount of space used to store collected logs, the more logs are available for upload when the SDK flushes its buffer(s).

Memory

The SDK uses RAM to store logs while they await being flushed to persistent storage. The amount of memory the SDK uses for storing logs in memory is controlled via bitdrift Remote Control Plane (service) and can be changed at any time. Any changes to that value result in updating configurations for all connected mobile clients.

By default, the SDK is configured to use 2 MiB (~2.1 MB) of RAM for storing logs.

Notes

  • The total RAM consumed by the SDK is higher than the configured logs storage memory limit due to all of the code loaded by the SDK.
  • The lower the configured RAM amount, the higher the chances of logs being dropped before they are persisted to disk.

CPU

Currently we do not provide explicit per-log processing CPU cycles consumed or wall clock time as it is nearly impossible to provide accurate numbers given that workflows can be of various shapes and also have different levels of internal complexity. For example, doing a prefix or exact match has a smaller computational complexity than doing a regular expression match.

It is important to mention that all workflow processing happens on a background thread. The SDK has been architected to push as much work to a background thread as possible, even at the expense of dropping logs. Meaning that if the background thread can't keep up, logs will be dropped instead of blocking other threads. We think this is the appropriate compromise for most customers.

As a point of reference, existing customers utilize hundreds of concurrent workflows and our internal metrics show a p50 per-log processing wall clock time in the hundreds of microseconds range for a real-world mix of workflow types. Some performance numbers for log emission time (separate from workflow processing time) can be found below in the platform specific details section.

For a deep dive, the workflow execution code can find be found here. Benchmarks (runnable on desktop only currently and thus useful for comparative development) can be found here.

Platform Specific Details

iOS

iOS performance testing was conducted using an iPhone 15 Pro Max (256GB) device against SDK v0.11.3.

CPU/Wall Clock

SDK start (Logger.start(...) method call): ~18.665ms.

Emitting logs (Logger.log...(...) family of methods):

  • 50-characters-long message (9 built-in fields): ~2000ns.
  • 50-characters-long message and 5 custom fields (14 fields total): ~18,000ns.
  • 50-characters-long message and 10 custom fields (19 fields total): ~32,104ns.

Info

Follow the steps from the iOS Benchmarking app to time benchmark the Capture SDK yourself.

Notes

  • Each field is a key-value pair. The keys and values used in profiling tests were each 10 characters long.
  • The SDK adds nine built-in fields to each emitted log, such as network_type or foreground. The total number of fields is calculated as the sum of custom fields and built-in fields.

Android

Android performance testing was conducted using a Pixel 7A (128GB) device against SDK v0.15.1.

CPU/Wall Clock

SDK start (Logger.start(...) method call): ~7.248ms.

Emitting logs (Logger.log...(...) family of methods):

  • 50-characters-long message (9 built-in fields): ~51,743ns.
  • 50-characters-long message and 5 custom fields (14 fields total): ~83,706ns.
  • 50-characters-long message and 10 custom fields (19 fields total): ~113,163ns.

Info

The Android benchmarking test code can be found here.

Notes

  • Each field is a key-value pair. The keys and values used in profiling tests were each 10 characters long.
  • The SDK adds ten built-in fields to each emitted log, such as network_type or foreground. The total number of fields is calculated as the sum of custom fields and built-in fields.

Lyft A/B Experimentation

Separate iOS and Android experiments were run for the rollout of the SDK in the Lyft app. Each experiment:

  • Targeted 100% of the user base of a given app on a given platform i.e., izOS Rider app.
  • Had 2 variants: control and treatment with 50/50 split.

The experiments allowed us to confirm that the Capture SDK:

  • ✅ Caused no degradation to any of the top level business metrics used by Lyft.
  • ✅ Caused no increase in app crashes.
  • ✅ Caused no increase in app hangs1 on iOS or ANRs2 on Android. After the conclusion of the experiments the SDK was fully rolled out3 in the Lyft iOS and Android apps. The careful rollout exercise allowed us to discover small bottlenecks and optimization opportunities that we promptly addressed.

  1. Lyft’s app hangs definition comes from Bugsnag SDK and is defined as lack of app’s responsiveness for at least 2 seconds. ↩

  2. ANR stands for “Application Not Responding” ↩

  3. Lyft continued to maintain a long term holdout group (10% of users) with the SDK disabled ↩