Forward selection
Forward selection uses searching as a technique for selecting the best features. It is an iterative method in which we start with having no feature in the model.
The step forward feature selection procedure begins by evaluating all feature subsets that consist of only one input variable. It selects the “best” feature and afterwards, adds all the other features to it individually, and selects a second feature that creates the new best performing model.
The process repeats over and over, adding one feature at a time until it meets certain criteria. After adding additional features to the subset, if the machine learning model performance doesn’t improve by more than a specific threshold, then we can stop the search and select this feature subset.
References:
