Monitoring over the afternoon we find that the our cloud provider infrastructure is not causing any further latency spikes. Last spike was roughly 13.00 UTC. During these spikes sensor event latency (from sensor firing to data-connector delivery) times was roughly as follows: Median time for events was 0.25 seconds. 75 percentile was 1-2 seconds 90 percentile was 6-8 seconds 99 percentile was up to 26 seconds
We will look into how we can organise our infrastructure to avoid latency spikes like this when avoidable.
Posted Oct 17, 2018 - 22:25 CEST
Monitoring
Our end to end latency is back to normal, but we will continue to monitor the situation.
Posted Oct 17, 2018 - 11:32 CEST
Investigating
Our data processing infrastructure is running behind which is causing delays in data received from realtime APIs and sensor state will not be fully up to date.
Posted Oct 17, 2018 - 10:49 CEST
This incident affected: Studio, Data ingestion, and API.