Top 28 Decision Tree & Random Forest Interview Questions: The Ultimate Data Science Guide 👨‍💻

Ultimate Interview Guide:
Decision Tree & Random Forest

Preparing for a Data Science interview? This comprehensive guide covers the Top 28 Questions regarding Decision Trees and Random Forests. Master the concepts of entropy, overfitting, pruning, and ensemble learning to crack your next exam.

📚 Table of Contents (Quick Links)

• Part 1: Decision Tree Mastery
• 1. What is a Decision Tree?
• 5. Gini Index Explained
• 7. Entropy & Info Gain
• 11. Pruning Techniques
• 15. Overfitting Issues
• Part 2: Random Forest & Ensemble
• 17. What is Random Forest?
• 19. Bagging Approach
• 24. Bagging vs Boosting
• 28. Extra Trees vs Random Forest

1 Part I: Decision Tree Algorithm

01. What is a Decision Tree?

A Decision Tree is a non-parametric supervised learning method used for both classification and regression. It learns simple decision rules inferred from data features to predict the value of a target variable.

Analogy: It works like a flowchart of "If-Then-Else" statements.

Root Node (All Data)

↓

Decision Node
(Is Age > 30?)

↓

Leaf: Yes

Leaf: No

Fig: Simplified Structure of a Decision Tree

02. Pros and Cons of Decision Trees

✅ Pros:

• Interpretability: Easy to visualize and explain to stakeholders.
• No Scaling: Does not require feature normalization.
• Versatile: Handles both numeric and categorical data.

❌ Cons:

• Overfitting: Creates complex trees that don't generalize well.
• High Variance: Small data changes can result in a completely different tree.
• Bias: Biased towards dominant classes in imbalanced datasets.

03/04. How does it work & Find Best Split?

The algorithm uses a Greedy Approach (Recursive Binary Splitting):

1. Start at the Root Node with all data.
2. Evaluate every possible split on every feature.
3. Calculate a metric (Gini/Entropy) for each split.
4. Select the split that results in the highest homogeneity (purity).
5. Repeat recursively until a stopping criterion (leaf node) is met.

05-07. Splitting Metrics: Gini, Chi-Square, Entropy

1. Gini Index (CART Algorithm):
Measures impurity. Range [0, 0.5]. The goal is to minimize Gini.
Formula: $1 - \sum (p_i)^2$

2. Entropy & Information Gain (ID3/C4.5):
Measures randomness (0=pure, 1=impure). The goal is to maximize Information Gain (Parent Entropy - Child Entropy).

3. Chi-Square (CHAID):
Statistical test to find significance between parent and child nodes. Goal: maximize value.

08. What is Variance in Decision Tree?

Used for Regression Trees (continuous target). Since we can't use Gini/Entropy, the algorithm chooses splits that minimize the Variance (Sum of Squared Errors) within the child nodes.

09/10. Hyperparameters & Node Types

Key Hyperparameters:

• max_depth: Controls vertical growth (prevents overfitting).
• min_samples_split: Minimum samples to split a node.
• min_samples_leaf: Minimum samples required in a leaf.

Node Terminology:

• Root Node: Topmost node (entire population).
• Decision Node: Internal node that splits further.
• Leaf/Terminal Node: Final node (outcome), no further splits.

11/12. What is Pruning & How to Perform it?

Pruning is the technique of cutting down branches of the tree that add little power to classification to reduce complexity and prevent overfitting.

Types:

1. Pre-Pruning: Stop the tree early during construction (e.g., limiting max_depth).
2. Post-Pruning: Build the full tree, then remove insignificant branches (e.g., Cost Complexity Pruning).

13-16. Data Types & Overfitting

• Homogeneous: Pure node (Samples belong to same class).
• Heterogeneous: Impure node (Mixed classes).
• Greedy Algorithm: Makes the optimal local choice at each step without considering global optimality.

---

Why Overfit? Trees grow until every leaf is pure, learning "noise" in training data.
How to Avoid? Pruning, limit depth, use Random Forest.

2 Part II: Random Forest & Ensemble

17/18. What is Random Forest vs Decision Tree?

Random Forest is an ensemble method that builds multiple decision trees and merges their results (Voting for classification, Averaging for regression).

Feature	Decision Tree	Random Forest
Structure	Single Tree	Multiple Trees
Overfitting	High Risk	Low Risk (Robust)
Speed	Fast	Slower
Interpretability	High (White Box)	Low (Black Box)

19/20. Bagging & Random Sampling

Bagging (Bootstrap Aggregating):

1. Bootstrap: Create subsets of data using Random Sampling with Replacement (same row can be picked twice).
2. Feature Randomness: At each split, only a random subset of features is considered.
3. Aggregating: Combine results from all trees.

Tree 1
(Subset A)

Tree 2
(Subset B)

Tree 3
(Subset C)

➔

Voting / Average
Final Prediction

21. Setting Number of Trees

In Python (Scikit-Learn), we use the parameter n_estimators.

Example: n_estimators = 100 builds 100 trees inside the forest.

22. Which is better?

Random Forest is generally better for accuracy and robustness. However, Decision Tree is preferred if you need model explainability or have very strict latency/resource constraints.

23-26. Ensemble: Bagging vs Boosting

Ensemble Learning: Combining multiple weak models to create a strong model.

🆚 Bagging (Parallel)

Builds models independently at the same time.
Goal: Reduce Variance.
Example: Random Forest.

🆚 Boosting (Sequential)

Builds models one after another; each corrects the previous errors.
Goal: Reduce Bias & Variance.
Example: XGBoost, AdaBoost.

28. Extra Trees vs Random Forest

• Random Forest: Uses Bootstrapping and finds the optimal split within random features.
• Extra Trees (Extremely Randomized Trees): Uses the whole dataset (no bootstrapping) and selects a random split point. It is faster and can reduce variance further.

🚀 Ready to Ace the Interview?

Mastering these concepts covers 90% of tree-based algorithm questions. Keep practicing!