lasso

Univariate ranking is the trap everyone falls into: it retains redundant features and discards ones that only matter in combination. Here is the filter/wrapper/embedded breakdown plus mRMR and Boruta, and where each one falls short.

Lasso delivers sparsity, Ridge copes with correlated features, and each stumbles where the other excels. Elastic Net combines them for a specific, common failure. Here is when it earns its two hyperparameters.