Complex Real World Application Systems behavioural profiling using a dynamical systems phase space approach and trending techniques

This research proposes a new approach to measuring system state to track behaviour for pre-emptive anomaly detection using known trending techniques and four whole-of-service metrics linked to underpinning service architecture inputs and outputs – workload, workload inter-arrival pattern, response time performance proxy and response time distribution pattern. The industry trend for systems monitoring is to buy tools to measure as many metrics as possible to detect anomalies. This leads to “alert fatigue” and inefficient detection. This has led to research in automated statistical learning algorithms to aggregate and simplify a burgeoning number of metrics. This paper proposes an alternative dynamical systems approach using “more global, fewer and nevertheless informative” metrics and NOT, “more precise, more detailed and infinitely many”. The four metrics are a phase space representation of system state for complex real-world online systems. Previous design science research (Gow et al.) has shown these metrics can be measured near real-time and used to detect anomalous behaviour with greater efficiency than standard industry monitoring approaches during core business hours. Using simulation this paper demonstrates that these metrics can create a longer-term behaviour model via standard trending techniques-DBSCAN clustering, Fourier Series curve fitting, and a geometrical view of nonlinear phase space metrics.