TY - JOUR

T1 - PARAFAC2 and local minima

AU - Yu, Huiwen

AU - Bro, Rasmus

N1 - Funding Information: Financial support by the program of China Scholarship Council as well as from the Danish Dairy Research Foundation is acknowledged. Dr. Dillen Augustijn is also acknowledged for his helpful comments in this research. Publisher Copyright: © 2021 Elsevier B.V.

PY - 2021

Y1 - 2021

N2 - PARAFAC2 is a useful algorithm for decomposing tensors that do not have low-rank variation such as e.g. PARAFAC requires. It has been applied in analyzing different types of multi-way data, such as GC-MS data. Since the optimization in fitting the loss function of PARAFAC2 is non-convex, the PARAFAC2 model suffers from local minima. In this paper, we investigated the local minima issue in the PARAFAC2 decomposition. We observed that the decomposed loading matrices corresponding to local minima and global minima are in general very different even though the loss function values can be very similar. We also observed that overfitting always led to higher fraction of local minima. For coping with local minima in PARAFAC2, we investigated the effect of non-negativity constraints on avoiding local minima, proposed some PARAFAC2 algorithms with optimized initializations, and implemented a simple local minima avoidance procedure in the general PARAFAC2 algorithm. These remedies are evaluated and illustrated by the use of different types of datasets such as simulated data, GC-MS data and EEG data. It is finally concluded that imposing non-negativity constraints on the PARAFAC2 model decreased the fraction of local minima significantly, using the proposed optimized initialization PARAFAC2 algorithm decreased the average fraction of local minima from the highest (84%) to the lowest (0%), and using the proposed local minima avoidance PARAFAC2 algorithm eliminated all the local minima in our case. In case of modeling data with high complexity, it will be promising to use all the remedies at the same time in order to avoid local minima PARAFAC2 models successfully. To the best of our knowledge, this is the first paper that investigate the local minima issue in the context of PARAFAC2 for irregular tensor.

AB - PARAFAC2 is a useful algorithm for decomposing tensors that do not have low-rank variation such as e.g. PARAFAC requires. It has been applied in analyzing different types of multi-way data, such as GC-MS data. Since the optimization in fitting the loss function of PARAFAC2 is non-convex, the PARAFAC2 model suffers from local minima. In this paper, we investigated the local minima issue in the PARAFAC2 decomposition. We observed that the decomposed loading matrices corresponding to local minima and global minima are in general very different even though the loss function values can be very similar. We also observed that overfitting always led to higher fraction of local minima. For coping with local minima in PARAFAC2, we investigated the effect of non-negativity constraints on avoiding local minima, proposed some PARAFAC2 algorithms with optimized initializations, and implemented a simple local minima avoidance procedure in the general PARAFAC2 algorithm. These remedies are evaluated and illustrated by the use of different types of datasets such as simulated data, GC-MS data and EEG data. It is finally concluded that imposing non-negativity constraints on the PARAFAC2 model decreased the fraction of local minima significantly, using the proposed optimized initialization PARAFAC2 algorithm decreased the average fraction of local minima from the highest (84%) to the lowest (0%), and using the proposed local minima avoidance PARAFAC2 algorithm eliminated all the local minima in our case. In case of modeling data with high complexity, it will be promising to use all the remedies at the same time in order to avoid local minima PARAFAC2 models successfully. To the best of our knowledge, this is the first paper that investigate the local minima issue in the context of PARAFAC2 for irregular tensor.

KW - Binomial confidence intervals

KW - Initialization

KW - Local minima

KW - Non-negativity constraints

KW - PARAFAC2

KW - Tensor

U2 - 10.1016/j.chemolab.2021.104446

DO - 10.1016/j.chemolab.2021.104446

M3 - Journal article

AN - SCOPUS:85118861499

VL - 219

JO - Chemometrics and Intelligent Laboratory Systems

JF - Chemometrics and Intelligent Laboratory Systems

SN - 0169-7439

M1 - 104446

ER -