A Joint Matrix Factorization and Clustering Scheme for Irregular Time Series Data

Abstract

Key Performance Indicator (KPI) clustering plays an important role in Artificial Intelligence for IT Operations when facing large-scale KPIs. It can effectively reduce the overhead by dividing KPIs into several classes, then applying the same anomaly detection or prediction model to all KPIs in a class. However, KPI sampling strategies vary according to the various environments, which produces irregular KPIs. Few existing works have considered the clustering of KPIs with irregular sampling. Matrix factorization (MF) is widely applied in low-rank data recovery and can be used to align and fill the irregular KPIs. However, the clustering performance after recovering and filling by MF remains unknown. These two problems interact with each other and should be solved together. Therefore, we formulate the joint MF and clustering problem for irregular KPIs and design an iterative clustering scheme based on MF. The iterative clustering scheme comprises alignment and pre-filling, the loop of clustering and subclass filling by MF, which can work with two pre-filling methods and any type of clustering algorithms. Extensive experiments on two real-world datasets show that the iterative clustering scheme can obtain higher normalized mutual information (NMI) than non-iterative clustering, and moreover consumes less computational time than Dynamic Time Warping.

Keywords: Time series clusteringIrregular samplingMatrix factorizationIteration