TY - JOUR

T1 - Interpretable Feature Learning in Multivariate Big Data Analysis for Network Monitoring

AU - Camacho, Jose

AU - Wasielewska, Katarzyna

AU - Bro, Rasmus

AU - Kotz, David

N1 - Publisher Copyright: Authors

PY - 2024

Y1 - 2024

N2 - There is an increasing interest in the development of new data-driven models useful to assess the performance of communication networks. For many applications, like network monitoring and troubleshooting, a data model is of little use if it cannot be interpreted by a human operator. In this paper, we present an extension of the Multivariate Big Data Analysis (MBDA) methodology, a recently proposed interpretable data analysis tool. In this extension, we propose a solution to the automatic derivation of features, a cornerstone step for the application of MBDA when the amount of data is massive. The resulting network monitoring approach allows us to detect and diagnose disparate network anomalies, with a data-analysis workflow that combines the advantages of interpretable and interactive models with the power of parallel processing. We apply the extended MBDA to two case studies: UGR’16, a benchmark flow-based real-traffic dataset for anomaly detection, and Dartmouth’18, the longest and largest Wi-Fi trace known to date.

AB - There is an increasing interest in the development of new data-driven models useful to assess the performance of communication networks. For many applications, like network monitoring and troubleshooting, a data model is of little use if it cannot be interpreted by a human operator. In this paper, we present an extension of the Multivariate Big Data Analysis (MBDA) methodology, a recently proposed interpretable data analysis tool. In this extension, we propose a solution to the automatic derivation of features, a cornerstone step for the application of MBDA when the amount of data is massive. The resulting network monitoring approach allows us to detect and diagnose disparate network anomalies, with a data-analysis workflow that combines the advantages of interpretable and interactive models with the power of parallel processing. We apply the extended MBDA to two case studies: UGR’16, a benchmark flow-based real-traffic dataset for anomaly detection, and Dartmouth’18, the longest and largest Wi-Fi trace known to date.

KW - Analytical models

KW - Anomaly Detection

KW - Big Data

KW - Dartmouth Campus Wi-Fi

KW - Data models

KW - Data visualization

KW - Interpretable Machine Learning

KW - Monitoring

KW - Multivariate Big Data Analysis

KW - Network Monitoring

KW - Principal component analysis

KW - Representation learning

KW - UGR’16

U2 - 10.1109/TNSM.2024.3368501

DO - 10.1109/TNSM.2024.3368501

M3 - Journal article

AN - SCOPUS:85186994445

JO - IEEE Transactions on Network and Service Management

JF - IEEE Transactions on Network and Service Management

SN - 1932-4537

ER -