Issue with javascript tracker outqueue payload size

Hey all,

We are running the streaming pipelines + js tracker version 2.11.0, and are noticing issues with enableFormTracking on large forms.

In the situation we’ve been seeing (GET-based tracking in this case, not POST), large e=ue events are generated (as is the case with submit_form self-describing events) but fail to go outbound from the browser. These failing events are then retained in localstorage, via the tracker’s snowplowOutQueue_, but continue to fail when the retry kicks in - therefore stopping all subsequent events that build up in the queue.

Has anyone experienced this before? Is the only workaround setting maxLocalStorageQueueSize to something relatively small, presuming the queue purges chronologically when the threshold has been exceeded?

Note- I do know how to whitelist and blacklist form tracking, but that assumes forms have class consistency (and/or aren’t hashed), or don’t change very often. All of which I cannot rely on. My real concern is not losing the form submission event, but rather the failure to fire the (relatively low-value) event appears to be blocking all subsequent (high-value) events.

Hi @jakethomas
Setting maxLocalStorageQueueSize will drop new events, rather than purging the oldest events, so I don’t think this will work in your case (and is available from 2.12.0). The mode exists predominently to stop the JS Tracker from filling the Local Storage quota for a web app, if the web app also requires local storage to function correctly.

Is there a reason you are unable to switch to POST requests? This would likely be the easiest solution here. You could also switch to Beacon events and let the browser deal with the sending, these events are sent to the browser to deal with using the sendBeacon API and won’t end up in the localStorage queue.

You could also disable local storage, so the tracker doesn’t attempt to stored failed events. That obviously has the downside that failed events won’t be retried (or events that don’t get sent before a navigation event will be lost). You would set the stateStorageStrategy to cookie (or none with further downsides) on tracker initialisation to achieve this.

Thanks for the note @PaulBoocock- that does make sense and is much appreciated.

With respect to swapping things to POST: that is certainly the easiest solution on domains that we have quick/easy/direct access to. But we are also running the (GET) tracker on third-party/partner domains and this is an issue on those (if forms get added that are too large, etc). The development cycles for modifying the tracker on these domains are much slower, and we don’t have full control here. I definitely acknowledge the situation is a bit unique and probably not super common, but wanted to raise the concern.

1 Like