Regularized Linear Regression: A Study in Different Dimensional Frameworks

عون محمد بشير; مزود فتيحة

Article Details

Regularized Linear Regression: A Study in Different Dimensional Frameworks

Authors:

عون محمد بشير | مزود فتيحة

Publisher:

Ecole nationale supérieure en statistique et en économie appliquée

Journal:

Revue d'économie et de statistique appliquée Vol.22 Issue 1, pp.31-46

Document Type:

Article

Publication Date:

6/2025

Document Language:

English

Document ID (Org No.):

2001454905

pISSN:

2600-6642

Keywords:

Abstract:

 This study concentrates on the evaluation of various regression model selection methods with sample sizes exceeding the number of features in one hand, and the number of features Surpassing sample size case in other, using parameters from [1] for low dimensional settings and [2] for high dimensional setup , focusing on regularization techniques and true subset selection accuracy. The experiments used an orthogonal, low correlation and high correlation design matrix under various number of features, with noise vectors generated from a Gaussian distribution, and varied noise levels, and observation counts. The assessment covered Least Absolute Shrinkage and Selection Operator (LASSO), Ridge, Elastic-Net, Minimax Concave Penalty (MCP) and Smoothly Clipped Absolute Deviation (SCAD) as sparse regularization methods, adaptive methods and Step-Wise selection methods. Performance was evaluated based on the frequency of correct subset selections, with higher frequencies reflecting superior methods as mentioned in [3]. In low dimensional data, all methods generally performed well across all scenarios, while some regularization methods were sensitive to sample size and noise. For high dimensional generation, all methods reach the perfect rate of recovery except the Step-Wise selection that shows perturbations and limitation in this case. This study not only provides valuable insights for practitioners in statistical model selection but also calls for future research to explore the application of these methods in real- world phenomena in all fields of study, which matches the parameters settings mentioned in this paper.

Article Details

Regularized Linear Regression: A Study in Different Dimensional Frameworks

Authors:

عون محمد بشير | مزود فتيحة

Publisher:

Ecole nationale supérieure en statistique et en économie appliquée

Journal:

Revue d'économie et de statistique appliquée Vol.22 Issue 1, pp.31-46

Document Type:

Article

Publication Date:

6/2025

Document Language:

English

Document ID (Org No.):

2001454905

pISSN:

2600-6642

Statistical Model

High Dimension

Linear Regression

Sparse Regularization Methods

Model Selection

Issues

Article Details

Regularized Linear Regression: A Study in Different Dimensional Frameworks

Authors: عون محمد بشير | مزود فتيحة

Publisher: Ecole nationale supérieure en statistique et en économie appliquée

Journal: Revue d'économie et de statistique appliquée Vol.22 Issue 1, pp.31-46

Document Type: Article

Publication Date: 6/2025

Document Language: English

Document ID (Org No.): 2001454905

pISSN: 2600-6642

Statistical Model

High Dimension

Linear Regression

Sparse Regularization Methods

Model Selection

Issues

Authors:

عون محمد بشير | مزود فتيحة

Publisher:

Ecole nationale supérieure en statistique et en économie appliquée

Journal:

Revue d'économie et de statistique appliquée Vol.22 Issue 1, pp.31-46

Document Type:

Article

Publication Date:

6/2025

Document Language:

English

Document ID (Org No.):

2001454905

pISSN:

2600-6642