Accessibility navigation

Sparse modeling using orthogonal forward regression with PRESS statistic and regularization

Chen, S., Hong, X., Harris, C. J. and Sharkey, P. M. (2004) Sparse modeling using orthogonal forward regression with PRESS statistic and regularization. IEEE Transactions on Systems Man and Cybernetics Part B-Cybernetics, 34 (2). pp. 898-911. ISSN 1083-4419

Full text not archived in this repository.

To link to this article DOI: 10.1109/tsmcb.2003.817107


The paper introduces an efficient construction algorithm for obtaining sparse linear-in-the-weights regression models based on an approach of directly optimizing model generalization capability. This is achieved by utilizing the delete-1 cross validation concept and the associated leave-one-out test error also known as the predicted residual sums of squares (PRESS) statistic, without resorting to any other validation data set for model evaluation in the model construction process. Computational efficiency is ensured using an orthogonal forward regression, but the algorithm incrementally minimizes the PRESS statistic instead of the usual sum of the squared training errors. A local regularization method can naturally be incorporated into the model selection procedure to further enforce model sparsity. The proposed algorithm is fully automatic, and the user is not required to specify any criterion to terminate the model construction procedure. Comparisons with some of the existing state-of-art modeling methods are given, and several examples are included to demonstrate the ability of the proposed algorithm to effectively construct sparse models that generalize well.

Item Type:Article
Divisions:Faculty of Science > School of Systems Engineering
ID Code:15171
Uncontrolled Keywords:Bayesian learning, cross validation, orthogonal forward regression, predicted residual sums of squares (PRESS), statistic, regularization, sparse data modeling, BASIS FUNCTION NETWORKS, LEAST-SQUARES, SYSTEM-IDENTIFICATION, MODEL, SELECTION, ALGORITHM, VALIDATION

University Staff: Request a correction | Centaur Editors: Update this record

Page navigation