Senior interview questionAsked at LinkedIn · interview write-up
This from-scratch NumPy logistic regression doesn't train. Debug the sigmoid, its derivative and the gradient computation.
Model answer
import numpy as np
def sigmoid(z):
out = np.empty_like(z, dtype=float)
pos = z >= 0
out[pos] = 1 / (1 + np.exp(-z[pos]))
ez = np.exp(z[~pos])
out[~pos] = ez / (1 + ez)
return out
def train(X, y, lr=0.1, steps=1000):
# X (N, D), y (N,) of 0/1
w, b = np.zeros(X.shape[1]), 0.0
for _ in range(steps):
p = sigmoid(X @ w + b)
err = p - y # dLoss/dz for log loss
w -= lr * X.T @ err / len(y)
b -= lr * err.mean()
return w, b
The usual bugs: a sigmoid that overflows for large negative inputs; a derivative written as s(1 minus z) instead of s(1 minus s); and multiplying by that derivative again, when log loss already gives a logit gradient of exactly p minus y. Also check the sign of the update, dividing by N, and a y of shape (N, 1) silently broadcasting to (N, N). Confirm with a finite-difference gradient check.
Practise the rest
35 Training questions, 492 in all, each with a model answer. Free.
More from LinkedIn interview questions.
More Training interview questions
- Implement masked cross-entropy in NumPy that stays numerically stable, with label smoothing and a temperature.
- Build a pipeline that finds math documents in web crawls and scores their quality.
- Explain KL divergence for language models: which direction is which, what happens when supports don't match, and how it's used as a regulariser.
- When would you use mean squared error versus cross-entropy? And what is LoRA?
- Explain Adam and its bias correction, and why AdamW decouples weight decay.
- Explain policy types in RL and modern policy-gradient methods such as TRPO, PPO and GAE, then compare linear attention and grouped-query attention.