Traditionally, chemometric models consists of parameters found by solving a least squares criterion. However, these models can suffer from overfitting, as well as being hard to interpret because of the large number of active parameters. This work proposes the use of a generalized L₁ norm penalty for constraining models to obey certain structural properties, including parameter sparsity and sparsity on pairwise differences between parameter estimates. The utility of this framework is used to modify principal component analysis, partial least squares, canonical correlation analysis, and multivariate analysis of variance type of models applied to synthetic and chemical data. This work argues that L₁ norm penalized models offers parsimony, robustness and predictive performance, and reveals a path for modifying unconstrained chemometric models through convex penalties.