Overview
Feature selection is the process of identifying a subset of the most informative variables, genes, proteins, metabolites, or spectral peaks, from a high-dimensional dataset. Biological experiments often measure tens of thousands of features from relatively few samples, creating a “large p, small n” problem that leads to overfitting and poor generalization. Feature selection mitigates this by removing irrelevant and redundant variables, improving model accuracy, reducing training time, and yielding more interpretable models that highlight the biological drivers of a phenotype.
Methods
Filter methods score each feature independently using univariate statistics such as t-tests, ANOVA, mutual information, or correlation with the target variable. They are computationally efficient but ignore feature interactions. Wrapper methods evaluate feature subsets by training and assessing a model on each candidate subset; recursive feature elimination (RFE) with a support vector machine is a popular example. These methods capture interactions but are computationally expensive. Embedded methods perform selection during model training, L1 regularization (Lasso) drives irrelevant coefficients to zero, and tree-based models provide feature importance scores as a byproduct. Domain knowledge from pathway databases can further guide selection toward biologically meaningful variables.
Practical Protocol
A practical feature selection pipeline for biomarker discovery starts with an omics dataset containing thousands of features (e.g., 15,000 genes) measured across a modest number of samples (e.g., 50 cases and 50 controls). The researcher first applies a filter method: a t-test is run for each gene comparing cases versus controls, and the p-values are adjusted for multiple testing using the Benjamini-Hochberg procedure. Genes with adjusted p-value below 0.05 and an absolute fold-change greater than 1.5 are retained, typically reducing the candidate list to a few hundred features. Next, an embedded method such as Lasso logistic regression is applied to the filtered set. L1 regularization is tuned via cross-validation over a range of lambda values, selecting the lambda that minimizes cross-validated error. The features with non-zero coefficients at the optimal lambda are selected, often yielding 10–30 genes. For final refinement, recursive feature elimination with a support vector machine (RFE-SVM) narrows the panel to a minimal diagnostic signature of 5–10 genes. In a real-world application, this three-stage approach was used to identify an 8-gene serum biomarker panel for early-stage pancreatic cancer, achieving 89% sensitivity and 92% specificity in an independent validation cohort. Another example comes from metabolomics, where feature selection of mass spectrometry peaks identified a 12-metabolite signature distinguishing responders from non-responders in a clinical trial for inflammatory bowel disease therapy, demonstrating clinical utility of feature selection beyond genomics.
Applications
Feature selection is essential in biomarker discovery, isolating a handful of diagnostic or prognostic markers from high-dimensional DNA microarrays and gene expression studies. In proteomics and mass spectrometry, it identifies discriminating spectral features for disease classification. The same principles apply to mass spectrometry metabolomics, where selection methods pinpoint metabolites that distinguish treatment groups, enabling the development of targeted clinical assays.
