Unsupervised Feature Selection Using Local Intrinsic Dimensionality -- Extended Version

Introduction

High‑dimensional data pose a dual challenge: they increase computational cost and can degrade the performance of learning algorithms, especially when no class labels are available to guide dimensionality reduction. A recent preprint proposes LIDFS (Local Intrinsic Dimensionality Feature Selection), an unsupervised method that ranks features based on how locally discriminative they are, without requiring label information or prior knowledge of the number of classes.

How LIDFS Works

The core idea of LIDFS is to treat each feature as a point in a space where the other features are its neighbors. By transposing the data matrix, the algorithm estimates the local intrinsic dimensionality (LID) of each feature using a maximum‑likelihood estimator applied to distances to its k nearest neighboring features. Features with lower LID values are considered more locally discriminative and receive higher importance scores.

A refinement step follows the initial ranking: features that are close to each other in the LID space are grouped, and only a single representative is kept while redundant neighbors are demoted. This yields a complete, ordered list of features from which any desired subset can be extracted without re‑running the whole procedure.

Performance Highlights

LIDFS was evaluated on 23 high‑dimensional benchmark datasets, covering both supervised (classification accuracy, AUC) and unsupervised (clustering accuracy, normalized mutual information) downstream tasks. Five different classifiers and two clustering metrics were used, and results were reported for both modest and aggressive feature‑selection regimes.

Key findings include:

• Top ranking in supervised evaluations – LIDFS consistently achieved the highest accuracy across the five classifiers.
• Strong unsupervised performance – It placed first or tied for second in clustering‑based metrics, matching methods that do require class‑count information.
• Speed advantage – With a computational complexity of \(O(d^2 n)\) (where d is the number of features and n the number of instances), LIDFS runs faster than its closest competitor while delivering comparable or better results.
• Robustness to parameter choice – A sensitivity analysis showed that smaller neighborhood sizes (k) tend to produce the best outcomes, simplifying practical tuning.

Practical Takeaways

For practitioners facing high‑dimensional, unlabeled data, LIDFS provides a straightforward pipeline:

1. Transpose the data matrix so that features become points.
2. Compute pairwise distances between features and estimate LID for each using a chosen k (small values are recommended).
3. Rank features by their LID scores, applying the refinement step to prune locally redundant features.
4. *Select the top‑m features* required for your downstream model, confident that the ranking is stable and does not need to be recomputed.

Because the method does not depend on class labels or the number of latent classes, it is especially suited for exploratory analysis, preprocessing pipelines for clustering, or any scenario where labeling is costly or unavailable.

Conclusion

LIDFS bridges a gap in unsupervised feature selection by leveraging the theoretical link between local intrinsic dimensionality and distance indiscriminability. Its empirical results demonstrate that a simple, label‑free ranking can rival or surpass more complex supervised or semi‑supervised approaches, all while maintaining computational efficiency. Researchers and engineers working with high‑dimensional data should consider adding LIDFS to their toolbox as a fast, reliable way to reduce dimensionality without sacrificing downstream performance.