This communication describes the combination of a feedforward neural network (NN) with one hidden neuron and partial least squares (PLS) regression. Through training of the neural network with an algorithm that is a combination of a modified simplex, PLS and certain numerical restrictions, one gains an NN solution that has several feasible properties: (i) as in PLS the solution is qualitatively interpretable; (ii) it works faster than or comparably with ordinary training algorithms for neural networks; (iii) it contains the linear solution as a limiting case. Another very important aspect of this training algorithm is the fact that outlier detection as in ordinary PLS is possible through loadings, scores and residuals. The algorithm is used on a simple non‐linear problem concerning fluorescence spectra of white sugar solutions.