C. Recursive Feature Elimination
It is a greedy optimization algorithm that aims to find the best-performing feature subset. It repeatedly creates models and keeps aside the best or the worst performing feature at each iteration. It constructs the next model with the left features until all the features are exhausted. It then ranks the features based on the order of their elimination.
It starts with ranking the feature with importance derived from an embedded method, such as those discussed earlier. Next, we remove the least important feature and build a new machine-learning algorithm. We then calculate a performance metric, such as ROC-AUC, MSE, or RMSE.
If the metric decreases by more than an arbitrarily set threshold, then the feature should be kept. Otherwise, we can remove that feature and repeat this process until a feature removal causes the performance metric to decrease past this threshold.
References:
