RFC: Mobile Trackers v2.0

The recent releases of the mobile trackers version 1.x have seen a progressive reduction of the feature gap between iOS and Android and an increased reliability of the trackers. With version 2, we want to focus on easier implementation and improved long term extensibility. Furthermore, the architecture and functionality of the mobile trackers was initially very much inspired by web trackers. With the new version we want to characterize their mobile nature, by deprecating web related events and entities/contexts, and developing more meaningful data tracking for the mobile environment.

Our proposal for the new major version is focused on two main areas:

  • Revision of the public API
  • Mobile focused event tracking methods

Public API revision

Tracker configuration

Currently, the tracker configuration requires the creation of the Emitter component. Optionally, the Subject component can be configured as well. Then both need to be passed to the Tracker component. We want to simplify this process, letting the Tracker component build the Emitter using a basic configuration, and letting the user specify more complex configurations only when needed.

The builder pattern is largely adopted in the tracker for the configuration of the tracker and creation of events. It’s particularly useful in cases where the component has many optional parameters because it helps to reduce the number of different variants of component constructors. Unfortunately, it doesn’t solve the problem of the required arguments, as they need to be declared in the signature of the constructor in order to be checked at compile time. For this reason we will take care to use the builder pattern only for the optional parameters, keeping them in the constructor, favouring compile-time validation of them.

Below an example of configuration in the tracker v.1.x.

Emitter emitter =
  new Emitter.EmitterBuilder("collector.snowplow.com", this.getApplicationContext())
  .emitTimeout(5)
  .customPostPath("postPath")
  .byteLimitGet(60000)
  .byteLimitPost(60000)
  .build();

Subject subject = new Subject.SubjectBuilder()
  .context(this.getApplicationContext())
  .build();

Tracker.init(
  new Tracker.TrackerBuilder(emitter, namespace, appId, this.getApplicationContext())
  .base64(false)
  .screenviewEvents(true)
  .screenContext(true)
  ...
  .installTracking(true)
  .applicationContext(false)
  .build()
);

The new version 2 will adopt a similar approach using configuration objects and adopting builder patterns when needed.

The tracker configuration also needs to be very straightforward. The simplest setup should only need the strictly required parameters, setting the majority of the optional ones to default values.

// basic configuration (lot of settings are set to default)
Tracker.setup("collector.snowplow.com", HTTPS, POST, namespace, appID);

Similarly, we want to allow a high level of configurability offering plenty of optional parameters. The builder pattern will still be a possible way to configure the tracker as much as the direct assignment of the property parameters.

// using configuration objects (fine granular settings)
NetworkConfiguration network =
  new NetworkConfiguration("collector.snowplow.com", HTTPS, POST);
network.timeout = 5;
network.customPostPath = "postPath";

EmitterConfiguration emitter = new EmitterConfiguration();
emitter.byteLimitGet = 60000;
emitter.byteLimitPost = 60000;

TrackerConfiguration tracker =
  new TrackerConfiguration(namespace, appId);
tracker.base64 = false;
tracker.screenViewAutotracking = true;
tracker.installAutotracking = true;
...
tracker.applicationContext = true;
tracker.screenContext = true;

Tracker.setup(network, tracker, Arrays.asList(emitter));

The new configuration process completely decouples the configuration of the tracker and its components by their instantiation and consequently implementation.

The tracker setup can take a variable number of configurations. The required ones will be explicitly declared in the signature (as NonNull) forcing compile-time checking. All the optional ones will be grouped in a list.

public void setup(
    NetworkConfiguration network,
    TrackerConfiguration tracker,
    List<Configuration> configurations
);

This new approach leaves at the tracker the burden of creating the components it needs. The developer only has to create the various configuration objects, passing them directly to the tracker setup. It doesn’t need to create an internal component used by the tracker.

However, for very special cases we still allow the injection of external components as implemented in the iOS tracker (v.1.5) and Android tracker (v.1.6) for the NetworkConnection and EventStore components.

Future improvements can also define different configuration objects for different high level features. For example, a configuration for session management.

SessionConfiguration session = new SessionConfiguration();
session.foregroundTimeout = 600;
session.backgroundTimeout = 300;
...

Tracker.setup(this.getApplicationContext(), network, tracker, Arrays.asList(emitter, session))

This has some benefits:

  • It lets the tracker delegate the configuration of the features to the features themselves.
  • It allows a higher degree of granularity in the configuration of the added features.
  • It promotes separation between core and high level features.
  • It favoures future extensibility and easier open source contribution.

Event creation

The mobile trackers 1.x require the creation of the event object through a builder before it can be tracked.

Timing event = Timing.builder()
  .category("category")
  .variable("variable")
  .timing(1)
  .label("label")
  .build());

tracker.track(event);

The proposed solution adopts the same configuration concept explained above for the tracker configuration.

// constructor
Timing event = new Timing("category","variable",1,"label");
tracker.track(event);
// or static factory method
Timing event = Timing.build("category","variable",1,"label");

In this example the constructor (or a static factory method) is needed because all those arguments are required arguments. Also the created event object is just a configuration of the event, hence it doesn’t allow eventID and timestamp overriding.

This solution simplifies the development of custom events as they are essentially plain objects implementing a simple interface or extending SelfDescribing class.

The custom events are created by the developer based on the schema stored on Iglu, so they need to be simple to write and easy to maintain.

Revised public API

The public API must be the minimum interface that enables the power of all the features in the tracker. For this reason, the public API of the various components can be affected by some breaking changes.

The Tracker interface will be simplified:

public interface TrackerInterface {
  void track(final Event event);
  void pause();
  void resume();

  String getVersion();

  void setup(
      String uri,
      RequestSecurity protocol,
      HttpMethod method
      String namespace,
      String appId);
  void setup(
      NetworkConfiguration network,
      TrackerConfiguration tracker,
      List<Configuration> configurations);
  void getConfigurations(List<Configuration> configurations);
  List<Configuration> getConfigurations();
}

Note: to avoid too much disruption, the current tracker configuration process (adopted on v1.x) will be kept available but deprecated.

All the other specialized methods will be provided through different interfaces specific for each service: Session, Diagnostic, Global Contexts, etc…

Example of tracker configuration and use:

// Setup the tracker
Tracker.setup("collector.snowplow.com", HTTPS, POST, namespace, appID);
...
// Send an event
Tracker.track(Timing.build("category","variable",1,"label"));
...
// Access internal features (e.g. SessionInterface)
Tracker.session.forceNewSession();

Android API improvements

The current codebase has a lack of Nullability annotations in the method signature which can be really helpful for the instrumentation of the tracker on Kotlin or Java based apps.

iOS API improvements

The interoperability between Swift and Objective-C is much worse than between Kotlin and Java. We plan to make the Snowplow iOS tracker version 2.x much more Swift compatible:

  • Optional arguments in the API methods
  • Swift name specified for all the ObjC methods
  • Constants converted to Swift enumeration where possible

Where the Objective-C API can cause higher friction we will implement a Swift wrapper around the tracker library to improve some API interaction with Swift language. It would remove some of the issue faced by Swift developer working on Objective-C API:

  • ObjC can’t have generics on protocols - it forces a lot of casts in the code.
  • ObjC requires NSObject implementation when a Swift class implements an ObjC protocol.
  • The Swift wrapper could be useful to simplify the API using constructs unavailable in Objective-C.

A special note about the management of NSError and NSException in Swift. Much of the errors are managed as exceptions. The v2.x will avoid exception-throwing as much as possible, enforcing the use of NSError, taking advantage of the Swift-ObjC bridging which automatically converts ObjC errors in Swift exceptions.

Mobile focused event tracking

The mobile trackers have grown organically influenced by the design of the web tracker. We want to make them more mobile oriented, bringing forward some improvements already partially implemented, clearing out confusion due to contrasting concepts.

Web events and contexts

We will deprecate events and contexts unuseful in the mobile tracker (to remove in the version 3.x) such as page views and web related fields in subject (more details in the section below).

Also, the geolocation context is partially implemented and considering the further restrictions added to the mobile platforms it’s hard to use it as it has been designed. It will be probably deprecated in favour of a better solution more mobile oriented in one of the next 2.x versions.

Identifiers

The current implementation lets the setting of various user identifiers: user_id, user_ipaddress, domain_userid, network_userid. This is mostly a legacy of the web tracker configuration.

The version 2.x of the tracker will deprecate them in favour of a new set of user/tracker identifiers:

  • Default User ID: created by the tracker by default and it can be reset programmatically when the user logout or login. It will help to aggregate the events of the same user in the data model.
  • Installation ID: created by the tracker at first execution (it’s the session_userId that can be tracked even if the session context is turned off). The installation ID will be constant until the app deletion.
  • Instance ID: created at the app start. It identifies the instance of the tracker. It’s reset at each app restart.

The identifiers used currently in the versions 1.x will not be removed but just be deprecated. However, we encourage the adoption of the new identifiers.

Tracker as singleton

Trackers should work as singleton. If needed, it may be possible to create multiple instances of the tracker in the same app, but the automatic features would work only for one of the tracker instances (the default one). This is because multiple trackers can confuse auto tracking and other services. In general the multiple tracker instantiation is strongly discouraged.

Updated requirements

iOS:

  • Bump min supported version to iOS 9 (No changes for the other platforms).

Android:

  • Bump min supported version to sdk 16 (Android 4.1+).
  • Migrate to AndroidX API.
6 Likes

I would keep this functionality for the moment - perhaps exposed through a specific interface that allows overriding of any tracker protocol parameters. This is quite useful in a variety of scenarios including instances where an identifier / piece of information may not persist through the collector (ip addresses + useragent are two examples).

Given this RFC it would also be interesting to consider what versioning looks like in terms of tracker configuration - particularly if it moves more towards an object interface. It would be ideal if there was a method to configure the tracker via sending in a JSON configuration object - allowing for version control of the configuration objects within a tracker.

For example - consider a iOS application using the default foreground / session background timeout. At some stage these timeouts are changed.

How does an analyst determine what the previous and current timeouts were if performing calculations on a session? At the moment there is no tracker configuration information that persists with any event so it’s difficult to determine if perhaps an event has a 300 second timeout versus a 360 second timeout.

This isn’t isolated to the mobile tracker alone (all trackers have the same issue) but I think it’s worth considering how to attach configuration information in some way so it is possible to document the lineage of what tracker configuration generated what event.

2 Likes

Thanks a lot Mike for your very useful feedbacks.

I really like the idea of overriding tracker protocol parameters through a specific interface. It would clear out the confusion about various different parameters (some not mobile related) allowing their setting only when needed.

I’m totally on board on this. We recently introduced an experimental feature called Diagnostic with the purpose of reporting tracker errors in the pipeline in order to spot in advance the source of possible anomalies in the event flow. However, the initial idea wasn’t just to track errors but also to track configuration changes. This will be strategic when the tracker is more customisable. I’m glad that your suggestion confirms we are moving in the right direction. This is definitely something we will take in consideration after the version 2 release.

1 Like

Are we considering enabling configuration of the installation id? I would like to think that since its lacking on v1.x , existing integrations might have had their own implementations of install id. So being able to override that on v2 would be helpful to reduce possible additional stitching.

1 Like

You raised a very sensible point. Of the three IDs mentioned in the RFC, only the latter InstantiationID is new. The InstallationID is the current UserID present in Session (and not overridable on 1.x). My current opinion is that the IDs could take place in a separate context but the InstallationID wouldn’t be different of the UserID already present in Session, and present also when Session is not enabled.

About the ability to override it I think I wouldn’t allow this option because we should grant that the ID is unique and it follows the same rules for renewal, which are actually hard to keep if it’s customisable. You can add an extra ID using a custom context rather than overriding this ID if you need an additional identifier.

However, the additional stitching is something to consider. I think it should be addressed in the data modelling phase rather than in the tracker. A bit of redundancy in the tracker can be beneficial. Making the data modelling phase able to handle these cases easily may be a double win.

Thanks for clarifying. I checked the api again and I realized I overlooked session user id and was looking at the subject userid which is overridable. I do get the point of restricting to keep the spec true to its nature.

On another note, is it possible to allow view controllers to override the name/controller used in automatic screen view? Asking because it seems like the using the current automatic screen view doesn’t exactly makes sense when there’s a parent child controller relationship wherein the parent acts just as a container controller. I’m not sure if this is going to be a niche feature or something that’s within the scope of automatic screen view your team have in mind when it was created. Anyway just throwing it out there.

The automatic screen view tracking is one of the feature we definitely want to improve once the v.2.0 is released. It’s a badly documented feature but you can add a String variable called snowplowId in your ViewController and the value will be used as name in place of the ViewController name. Changing the value of snowplowId accordingly to what you show in the container should allow you to track the shown screen more meaningfully.

Here an example from our demo app.

1 Like