Cheat sheet + 20 flashcards covering supervised & unsupervised learning, neural networks, bias-variance, evaluation metrics, and feature engineering.
Supervised: labelled data → predict output (classification, regression)
Unsupervised: unlabelled data → find patterns (clustering, dimensionality reduction)
Reinforcement: agent + environment + rewards → learn policy
High Bias (underfitting) — model too simple, misses patterns, high error on both train & test.
High Variance (overfitting) — model memorises training noise, low train error but high test error.
Goal: find the sweet spot — complex enough to learn, simple enough to generalise.
Train (~70%): fit model weights
Validation (~15%): tune hyperparameters, select model
Test (~15%): final unbiased evaluation — never touch until done
Cross-validation: rotate train/val splits (k-fold) for more reliable estimates.
Iteratively nudge weights in the direction that reduces loss:
w = w − α · ∂L/∂w
α = learning rate (too high → diverge, too low → slow)
SGD: one sample per step · Mini-batch: subset · Batch: full dataset
Linear: ŷ = wX + b, minimises MSE, outputs continuous value.
Logistic: applies sigmoid σ(z) = 1/(1+e⁻ᶻ) to linear output, outputs probability [0,1]. Decision boundary at 0.5. Minimises binary cross-entropy.
Layers: Input → Hidden(s) → Output
Each neuron: a = activation(Wx + b)
Activation functions:
ReLU: max(0,x) — hidden layers, avoids vanishing gradient
Sigmoid: binary output
Softmax: multi-class output (sums to 1)
Computes gradient of loss w.r.t. every weight using the chain rule, working backwards from output to input.
Forward pass → compute loss → backward pass → update weights.
Repeat for every batch until loss converges.
1. Pick K random centroids
2. Assign each point to nearest centroid
3. Recompute centroids as cluster means
4. Repeat 2–3 until convergence
Elbow method: plot inertia vs K, pick bend point.
Decision Tree: split data on feature that maximises information gain (entropy) or minimises Gini impurity. Prone to overfitting.
Random Forest: ensemble of trees on random data/feature subsets (bagging). Majority vote. Reduces variance, more robust.
Accuracy: (TP+TN)/total — misleading on imbalanced data
Precision: TP/(TP+FP) — of predicted positive, how many correct
Recall: TP/(TP+FN) — of actual positive, how many found
F1: 2·P·R/(P+R) — harmonic mean, balances both
ROC-AUC: area under TPR vs FPR curve
L1 (Lasso): adds |w| to loss → sparse weights, feature selection
L2 (Ridge): adds w² to loss → shrinks weights, keeps all features
Dropout: randomly zero out neurons during training
Early stopping: halt when validation loss stops improving
Normalisation: scale to [0,1] — (x−min)/(max−min)
Standardisation: mean=0, std=1 — (x−μ)/σ
One-hot encoding: convert categorical to binary columns
Missing values: impute (mean/median) or drop rows/cols
Space flip · → next · ← prev · G got it · R review