v3 · the Colab run: results

This tab is the v3 notebook exactly as it ran on Colab (T4 GPU, full data, 5 operator-grouped folds), with every real output. The lessons above explain the method; these pages show what actually happened.

The numbers

For comparison, the first model scored 0.70 on the leaderboard (its random-fold CV said 0.758). Grouped CV has tracked the leaderboard closely, so expect the leaderboard score to land near 0.72.

What the run tells us

• XGBoost and CatBoost are close (0.715 vs 0.713). The blend weights came out about 55/45, so both contribute.
• Operator smoothing helps: the stacker adds about +0.6 points over the best single model. Averaging predictions across an operator's other listings works because operators are consistent.
• Class weights edge out cut-points here (0.7214 vs 0.7199). With the stacker's well-calibrated probabilities, adjusting the 4 probabilities directly works slightly better.
• Per-metro cut-points didn't help. More parameters to tune, no gain. That's a typical overfitting signal.
• This run used all feature blocks (v2 + H1 + H2). Earlier experiments showed that choice makes almost no difference (±0.001).

Run details

From the executed final notebook:

submission.csv           ← stack | A class weights | CV (test-weighted) 0.7214
submission_rank2.csv     ← opt_blend + xgb_reg (a=0.7) | B cut-points | CV (test-weighted) 0.7199
submission_rank3.csv     ← opt_blend + xgb_reg (a=0.5) | B cut-points | CV (test-weighted) 0.7198

Pages in this tab

1. Setup & data: loading, operator id, grouped folds
2. Features: every feature block, built once and checkpointed
3. Feature experiments: the ablation setting
4. Models: XGBoost, XGBoost regression, CatBoost on GPU, with training logs
5. Operator smoothing: the level-2 stacker
6. Scenarios: the full scenario table
7. Submission: which scenarios were written to which file