Partial least squares regression (PLS-regression) is a statistical method that bears some relation to principal components regression; instead of finding hyperplanes of maximum variance between the response and independent variables, it finds a linear regression model by projecting the predicted variables and the observable variables to a new space. Because both the X and Y data are projected to new spaces, the PLS family of methods are known as bilinear factor models.

It is used to find the fundamental relations between two matrices (X and Y), i.e. a latent variable approach to modeling the covariance structures in these two spaces. A PLS model will try to find the multidimensional direction in the X space that explains the maximum multidimensional variance direction in the Y space. PLS-regression is particularly suited when the matrix of predictors has more variables than observations, and when there is multicollinearity among X values. By contrast, standard regression will fail in these cases.

PLS-regression is an important step in PLS path modeling, a multivariate data analysis technique that employs latent variables. This technique is often referred to as a form of variance-based or component-based structural equation modeling.^[1][2]

It was first introduced by the Swedish statistician Herman Wold. An alternative term for PLS (and more correct according to Wold^[3]) is projection to latent structures, but the term partial least squares is still dominant in many areas. It is widely applied in the field of chemometrics, in sensory evaluation, and more recently, in the analysis of functional brain imaging data(see Randy McIntosh).

Contents 1 Underlying model 2 Algorithms 2.1 PLS1 3 See also 4 Footnotes 5 References 6 External links

[edit] Underlying model

The general underlying model of multivariate PLS is

where X is an matrix of predictors, Y is an matrix of responses, T is a matrix (the score, component or factor matrix), P and Q are, respectively, and loading matrices, and matrices E and F are the error terms, assumed to be i.i.d. normal.

[edit] Algorithms

A number of variants of PLS exist for estimating the factor and loading matrices T,P and Q. Most of them construct estimates of the linear regression between X and Y as . Some PLS algorithms are only appropriate for the case where Y is a column vector, while others deal with the general case of a matrix Y. Algorithms also differ on whether they estimate the factor matrix T as an orthogonal, an orthonormal matrix or not.^{[4][5][6][7][8] [9]}. The final prediction will be the same for all these varieties of PLS, but the components will differ.

[edit] PLS1

PLS1 is a widely used algorithm appropriate for the vector Y case. It estimates T as an orthonormal matrix. In pseudocode it may be expressed as:

  1  function PLS1(X,y,l)  2    3  , the vector of all ones.  4  for k = 1 to l  5        6        7        8        9      if qk = 0 10          , break the for loop 11      if k < l 12           13           14           15  define W to be the matrix with columns W(1),W(2),...,W(l). Similarly define P,q 16   17   18  return B,B0 

This form of the algorithm does not require centering of the input X and Y, as this is performed implicitly by the algorithm. This algorithm features 'deflation' of the matrix X (subtraction of ), but deflation of the vector y is not performed, as it is not necessary (it can be proven that deflating y yields the same results as not deflating.). The user-supplied variable l is the limit on the number of latent factors in the regression; if it equals the rank of the matrix X, the algorithm will yield the least squares regression estimates for B and B₀

[edit] See also

• Feature extraction
• Data mining
• Machine learning
• Regression analysis

[edit] Footnotes

1. ^ Chin, W.W. (1998). Issues and opinion on structural equation modeling. MIS Quarterly, 22(1), vii-xvi.
2. ^ Fornell, C., & Bookstein, F.L. (1982). Two structural equation models: LISREL and PLS applied to consumer exit-voice theory. Journal of Marketing Research, 19(4), 440-452.
3. ^ Wold, S, Sjöström, M., Eriksson, L. (2001). PLS-regression: a basic tool of chemometrics. Chemometrics and Intelligent Laboratory Systems, 58, 109–130.
4. ^ Lindgren F, Geladi P, Wold S (1993) The kernel algorithm for PLS. J. Chemometrics 7:45-59
5. ^ de Jong, S. and ter Braak, C. J. F. (1994) Comments on the PLS kernel algorithm. J. Chemometrics 8:169-174.
6. ^ Dayal, B.S. & MacGregor, J.F. (1997) Improved PLS algorithms. J. Chemometrics 11:73-85.
7. ^ de Jong, S. (1993) SIMPLS: an alternative approach to partial least squares regression. Chemometrics and Intelligent Laboratory Systems, 18:251-263
8. ^ Rannar, S., Lindgren, F., Geladi, P. and Wold, S. (1994) A PLS Kernel Algorithm for Data Sets with Many Variables and Fewer Objects. Part 1: Theory and Algorithm. Journal of Chemometrics 8:111-125
9. ^ Abdi, H. (2009). Partial least square regression, projection on latent structure regression, PLS-Regression. Wiley Interdisciplinary Reviews: Computational Statistics: 2

[edit] References

• R. Kramer, Chemometric Techniques for Quantitative Analysis, (1998) Marcel-Dekker, ISBN: 0-8247-0198-4.
• Frank, Ildiko and Jerome Friedman (1993) (1993). A Statistical View of Some Chemometrics Regression Tools, Technometrics, 35(2), pp 109–148. 
• Haenlein, Michael and Andreas M. Kaplan (2004) (2004). A Beginner's Guide to Partial Least Squares Analysis, Understanding Statistics, 3(4), 283–297. 
• Henseler, Joerg and Georg Fassott (2005) (2005). Testing Moderating Effects in PLS Path Models. An Illustration of Available Procedures. 
• Lingjærde, Ole-Christian and Nils Christophersen (2000) (2000). Shrinkage Structure of Partial Least Squares, Scandinavian Journal of Statistics, 27(3), pp 459–473. 
• Tenenhaus Michel (1998). La Regression PLS: Theorie et Pratique. Paris: Technip.. 
• Rosipal, Roman and Nicole Kramer (2006) (2006). Overview and Recent Advances in Partial Least Squares, in Subspace, Latent Structure and Feature Selection Techniques, pp 34–51. 

[edit] External links

• An Interpretation of Partial Least Squares
• Connections of OLR, PLS, PCR
• Algorithm description
• PLS and regression tutorial
• PLS in Brain Imaging
• on-line PLS regression (PLSR) at Virtual Computational Chemistry Laboratory
• Uncertainty estimation for PLS
• A short introduction to PLS regression and its history