Abstract/Summary

A l1-norm penalized orthogonal forward regression (l1-POFR) algorithm is proposed based on the concept of leave-one-out mean square error (LOOMSE), by defining a new l1-norm penalized cost function in the constructed orthogonal space and associating each orthogonal basis with an individually tunable regularization parameter. Due to orthogonality, the LOOMSE can be analytically computed without actually splitting the data set, and moreover a closed form of the optimal regularization parameter is derived by greedily minimizing the LOOMSE incrementally. We also propose a simple formula for adaptively detecting and removing regressors to an inactive set so that the computational cost of the algorithm is significantly reduced. Examples are included to demonstrate the effectiveness of this new l1-POFR approach.