Mastering PCA & t-SNE: Ultimate Dimensionality Reduction Guide for Data Science Interviews (23 Q&A)👼🏿

Mastering PCA & t-SNE

The Ultimate Machine Learning Interview Guide for Dimensionality Reduction

Are you preparing for a data science interview? Understanding Dimensionality Reduction is crucial. This guide covers everything from Principal Component Analysis (PCA) to t-SNE(t-distributed Stochastic Neighbor Embedding), explaining the Curse of Dimensionality with 23 expert Q&As.

Quick Navigation (Indexing)

• Section 1: PCA Fundamentals & Purpose
• Section 2: t-SNE & Non-Linear Mapping
• Section 3: The Curse of Dimensionality
• Section 4: Expert Level & Implementation

Raw Data
(High Dimensions)

➔

PCA / t-SNE
(Feature Extraction)

➔

Reduced Data
(2D / 3D Visualization)

Lesson 1

PCA: Principal Component Analysis

Core Definition: PCA is a linear transformation technique that identifies the axes of maximum variance in data to reduce dimensions while preserving information.

1. What is PCA, and what is its primary purpose? PCA is a linear dimensionality reduction technique. Its primary purpose is to transform a large set of variables into a smaller one that still contains most of the information (variance) for data compression and visualization.

2. Explain Eigenvalues and Eigenvectors in the context of PCA? Eigenvectors represent the direction of the new axes (Principal Components), while Eigenvalues represent the magnitude of variance captured in that direction.

3. How does PCA reduce the Dimension of data? By projecting high-dimensional data points onto the top $k$ eigenvectors that have the highest eigenvalues, effectively ignoring directions with little variance.

4. Can we use PCA for Feature selection? No. PCA is for Feature Extraction. Selection keeps original variables; PCA creates new, combined variables.

Variance Explained (Visualized)

PC1 (90%) | PC2 (60%) | PC3 (30%) | PC4 (10%)

5. What is importance of variance explained in PCA? It tells us how much "information" is retained. It is calculated as $(\text{Eigenvalue}_i / \sum \text{Eigenvalues})$.

6. What are the Limitations of PCA? It assumes linearity, is sensitive to scaling, and the resulting components are hard to interpret.

7. In which situation might PCA not work well? When the data has non-linear relationships (e.g., Swiss roll shape).

8. How do we choose the number of Principal components? Using a Scree Plot to find the "elbow" or retaining components that reach 95% cumulative variance.

9. Can PCA be applied to non-numeric data? Only after converting it to numbers (e.g., TF-IDF for text or pixel intensities for images).

10. What is Relation between PCA and Covariance Matrix? PCA is the eigendecomposition of the Covariance Matrix.

Lesson 2

t-SNE: Visualizing Clusters

11. What is t-SNE and why is it used? t-SNE is a non-linear technique that preserves local structure, making it perfect for visualizing clusters in 2D.

12. Key differences between t-SNE and PCA? PCA is linear and fast (Global structure); t-SNE is non-linear and slow (Local structure).

Lesson 3

The Curse of Dimensionality

13. Explain Curse of Dimensionality and how to solve it? As dimensions increase, data becomes sparse and distance metrics fail. Solution: Use PCA, Feature Selection, or Regularization.

Lesson 4

Advanced Interview Concepts

14. Why is Scaling essential for PCA? PCA maximizes variance; unscaled data with large ranges will dominate the components unfairly.

15. What is the relation between PCA and SVD? Most software uses Singular Value Decomposition (SVD) to calculate PCA because it is more numerically stable.

16. What is Kernel PCA? An extension using the Kernel Trick to handle non-linear data.

17. What does "Perplexity" control in t-SNE? It balances the attention between local and global aspects of the data (similar to number of neighbors).

18. Why is t-SNE stochastic? It uses random initialization and gradient descent, leading to different results each run unless a seed is set.

19. Can you reconstruct data from t-SNE? No. Unlike PCA, t-SNE is a non-invertible probabilistic mapping.

20. What is the "Crowding Problem"? The difficulty of representing high-D distances in 2D. t-SNE solves this using the Student t-distribution.

21. What is PCA Whitening? Scaling components to have unit variance, decorrelating the data for better model training.

22. How does UMAP compare? UMAP is faster than t-SNE and preserves more global structure.

23. Does PCA reduce Overfitting? Yes, by removing noise and reducing the number of parameters the model needs to learn.

© 2025 Data Science Interview Masterclass. Follow for more technical guides!