Viva preparation

The Level-2 viva asks you to explain the code, answer concept questions from any course, and live-code. For every model you use, know its 5 components: Data, Model, Loss, Optimisation, Evaluation.

The 5 components for each model we used

Logistic Regression (the baseline)


Data	numeric features, imputed (median) and standardised; tier as the class label
Model	for each tier k, a score z_k = w_k·x + b_k; probabilities = softmax(z)
Loss	cross-entropy (log-loss) + L2 penalty (`C` = inverse strength)
Optimisation	L-BFGS, a quasi-Newton method using gradients
Evaluation	macro F1 on operator-grouped folds

Why scaling matters here: L2 penalises all weights equally, and gradient steps behave badly when features have very different ranges.

Gradient-boosted trees (XGBoost; also LightGBM, CatBoost)


Data	raw numeric features, NaN allowed, categories as integer codes
Model	a sum of many small decision trees; each tree outputs a score per tier, summed then softmax
Loss	multiclass log-loss + regularisation (L2 on leaf values, minimum child weight, max depth)
Optimisation	gradient boosting: each new tree fits the gradient/Hessian of the loss of the current ensemble (second-order Taylor step). Splits are chosen greedily to maximise the gain formula; the learning rate shrinks each tree
Evaluation	early stopping on the validation fold's log-loss; macro F1 for the final score

XGBoost grows trees level-wise up to max_depth.
LightGBM grows leaf-wise (always splits the best leaf).
CatBoost grows symmetric (oblivious) trees (the same split at each depth) and uses ordered boosting to reduce target leakage.

XGBoost regression (ordinal view)

Same machinery, but the target is the tier as a number (1–4) and the loss is squared error. The output (e.g. 2.6) becomes a tier through cut-points tuned for macro F1.

Key concepts and short answers

Why macro F1 and not accuracy? Macro F1 averages per-class F1, so every tier counts equally. Accuracy would let a model coast on the biggest tier.

What is cross-validation, and why grouped? CV estimates performance on unseen data by rotating which part is held out. Our test set contains unseen operators, so validation must hold out whole operators (StratifiedGroupKFold). Otherwise CV is optimistic (0.758 vs 0.70).

What is target encoding, and how can it leak? Replace a category by the average target of that category. If a row's own label is included, the feature "contains the answer". We compute it out-of-fold: each row's value comes only from other folds.

What is data leakage? Any information in training features that wouldn't be available for the real test rows. Here: the operator's identity, and labels inside target encodings.

Why early stopping? Boosting keeps reducing training loss. We stop when the validation loss stops improving, which is the point where more trees start to overfit.

What do class weights / cut-points do? The model outputs probabilities. Macro F1 depends on where we draw the decision lines. We tune them on out-of-fold predictions, cross-fitted (tuned on 4 folds, scored on the 5th) so the tuning can't fit noise.

What is stacking? A second model that takes the first models' predictions as input. Ours also gets the average prediction of the operator's other listings, which helps because operators are consistent.

Bias vs variance here? Logistic Regression is high bias (too simple for "zone × size" interactions). Deep trees are high variance; regularisation, early stopping and averaging folds reduce it.

Likely live-coding tasks (practise these)

Load train.csv, print the class distribution and the missing-value share per column.
Make StratifiedGroupKFold folds by operator and show that no operator appears in two folds.
Write out-of-fold target encoding for zone by hand (loop over folds; mean tier of the other folds).
Train XGBoost with early stopping on one fold and print the macro F1.
Plot a confusion matrix and explain which tiers get confused.
Change a hyperparameter (max_depth, learning_rate) and explain the effect on train vs. validation scores.