Chapter 24 of 25
Exam & Interview Prep
The questions that come up again and again
Motivation
You've now built an MLP from a single neuron all the way up through backpropagation, optimizers, and regularization. The last skill left to practice isn't a new concept — it's retrieval: being able to reach into everything you've learned and produce the right formula, the right diagnosis, or the right explanation, fast, under the pressure of an exam or an interview. This chapter is a condensed rehearsal of exactly that, pulling together the questions that come up again and again across viva boards and technical interviews alike.
Master Formula Sheet
Treat this as the sheet you'd want in front of you five minutes before walking into an exam — every core formula from the course, in one place.
| Topic | Formula |
|---|---|
| Neuron output | $z = \mathbf{w}^\top\mathbf{x} + b, \quad y=\varphi(z)$ |
| Layer forward pass | $\mathbf{z}^{[\ell]}=\mathbf{W}^{[\ell]}\mathbf{a}^{[\ell-1]}+\mathbf{b}^{[\ell]}, \quad \mathbf{a}^{[\ell]}=\varphi(\mathbf{z}^{[\ell]})$ |
| Sigmoid / derivative | $\sigma(z)=\frac{1}{1+e^{-z}}, \quad \sigma'=\sigma(1-\sigma)$ |
| Tanh derivative | $\tanh'(z) = 1-\tanh^2(z)$ |
| ReLU / derivative | $\max(0,z); \quad 1 \text{ if } z>0 \text{ else } 0$ |
| Softmax | $\text{softmax}(\mathbf{z})_k = e^{z_k}/\sum_j e^{z_j}$ |
| MSE | $\frac{1}{N}\sum(y_i-\hat{y}_i)^2$ |
| Binary cross-entropy | $-\frac{1}{N}\sum[y_i\log\hat{y}_i+(1-y_i)\log(1-\hat{y}_i)]$ |
| Output error (BCE+sigmoid or CCE+softmax) | $\boldsymbol\delta^{[L]}=\hat{\mathbf{y}}-\mathbf{y}$ |
| Backprop recursion | $\boldsymbol\delta^{[\ell]}=\big((\mathbf{W}^{[\ell+1]})^\top\boldsymbol\delta^{[\ell+1]}\big)\odot\varphi'(\mathbf{z}^{[\ell]})$ |
| Weight / bias gradient | $\partial L/\partial\mathbf{W}^{[\ell]}=\boldsymbol\delta^{[\ell]}(\mathbf{a}^{[\ell-1]})^\top, \quad \partial L/\partial\mathbf{b}^{[\ell]}=\boldsymbol\delta^{[\ell]}$ |
| Gradient descent update | $\theta\leftarrow\theta-\eta\nabla_\theta L(\theta)$ |
| Momentum | $\mathbf{v}_t=\beta\mathbf{v}_{t-1}+(1-\beta)\mathbf{g}_t; \quad \theta_t=\theta_{t-1}-\eta\mathbf{v}_t$ |
| Adam update | $\theta_t=\theta_{t-1}-\frac{\eta}{\sqrt{\hat{\mathbf{v}}_t}+\epsilon}\hat{\mathbf{m}}_t$ |
| L2 regularized loss | $L_{\text{data}}+\frac{\lambda}{2}\sum_\ell\|\mathbf{W}^{[\ell]}\|_2^2$ |
| Batch normalization | $\hat{z}=\frac{z-\mu_B}{\sqrt{\sigma_B^2+\epsilon}}, \quad y=\gamma\hat{z}+\beta$ |
| He initialization | $W\sim\mathcal{N}(0,\ 2/n_{in})$ |
| Xavier initialization | $W\sim\mathcal{N}(0,\ 2/(n_{in}+n_{out}))$ |
Don't just re-read it passively. Cover the right-hand column and try to reproduce each formula from the topic name alone. The ones you hesitate on are exactly the ones worth another pass through their original chapter.
Interview Corner: The Big Quiz
This is the heart of the chapter — the questions that come up again and again, across viva boards and technical interviews, spanning every major topic from the course.
Common Pitfalls
These are the specific traps that show up most often in exams and interviews — not because the underlying concept is hard, but because it's easy to state imprecisely under pressure.
δ^[ℓ] is a derivative with respect to the pre-activation z^[ℓ], not the weight itself. It's an intermediate quantity — you still need the outer product with the previous layer's activation, a^[ℓ-1], to get the actual weight gradient ∂L/∂W^[ℓ]. Stating δ as if it is the weight gradient is one of the most common half-credit answers on backpropagation questions.
"The perceptron is guaranteed to converge" is only half a sentence — the guarantee holds only if the data is linearly separable. Leaving that condition out is a very easy way to lose a mark on an otherwise correct answer.
The theorem says a network can exist that approximates a function — it says absolutely nothing about whether gradient descent will actually find that network, or how large it needs to be. Confusing existence with achievability is one of the most common conceptual slips at this level.
When one class dominates (say, 99.5% of transactions are legitimate), a model that always predicts "legitimate" scores 99.5% accuracy while being completely useless. Metrics like precision, recall, F1, or AUC — not raw accuracy — are what actually reveal performance on the minority class.
σ'(z) = σ(z)(1 − σ(z)) is meant to be evaluated using the cached activation a = σ(z), not by recomputing from the raw z. Plugging in z directly instead of the cached a is a common off-by-one-step implementation bug.
A NaN training loss has several distinct usual suspects: a learning rate too high (causing exploding gradients), an unclipped log(0) inside a cross-entropy loss, or unnormalized input features with extreme values. Diagnosing it well means checking all three, not jumping to the first one that comes to mind.
Numerical Practice
Working a few numbers by hand is the fastest way to expose a gap that reading alone hides. Try these before checking your work against earlier chapters:
- Given , , , compute the neuron's pre-activation and its sigmoid output.
- Perform one full forward pass through a network with layer sizes , using ReLU in the hidden layer and sigmoid at the output, with weights and biases of your choosing.
- Given and true label , compute the binary cross-entropy loss and the gradient , assuming a sigmoid output.
- Compute one step of Adam's parameter update (with , , ) given at , starting from .
- Count the number of trainable parameters in an MLP with architecture , including biases.
Case Studies for Extended Practice
Case Study 1 — Imbalanced Fraud Detection
A bank wants to build an MLP to detect fraud, where only 0.5% of transactions are fraudulent. Consider: an appropriate loss function and any modifications needed for the imbalance; appropriate evaluation metrics; and why accuracy alone would be misleading here.
Case Study 2 — Small Tabular Dataset
A hospital has only 500 patient records with 20 features and wants to predict a binary outcome. Consider: whether an MLP is necessarily the best choice on small tabular data; which regularization strategies matter most given the small dataset; and how to structure the train/validation/test split.
Case Study 3 — Slow, Unstable Training
An engineer reports that training loss oscillates wildly and occasionally spikes. Walk through the diagnostic order: learning rate first, weight initialization second, feature scaling third — and explain why that order makes sense.
Key Takeaways
- Mastery of this course looks like being able to state and derive every core formula from memory, not just recognize it when shown.
- It also means being able to explain why each technique exists — what specific failure it fixes — not just what it computes.
- Just as importantly, it means being able to diagnose a misbehaving network systematically: checking learning rate, initialization, and data scaling in a sensible order, rather than guessing.
- Every question in this chapter traces back to a specific earlier chapter — if any answer felt shaky, that's the signal for exactly where to go back and review.
Where This Shows Up in Practice
Interviewers rarely care whether you can recite a formula — they care whether you can explain what breaks without it, and what you'd check first when something goes wrong in a real training run. That diagnostic instinct, built up across every chapter of this course, is the actual skill being tested every time one of these questions comes up.