Regression objectives and gradient updates
For predictions wx with no intercept, define J(w) = Σ(wxᵢ − yᵢ)²/(2n). Then J′(w) = Σ(wxᵢ − yᵢ)xᵢ/n, and gradient descent subtracts the learning rate times this derivative. A different loss normalization changes the gradient scale, so copy the objective before differentiating.
Check yourself: Are all residuals evaluated at the same current parameter?
Classification decisions and metrics
A probability estimate becomes a class decision only after choosing a threshold. With a stated positive class, precision is TP/(TP + FP), recall is TP/(TP + FN), and accuracy is (TP + TN)/N. Class imbalance can make accuracy high despite poor positive-class detection.
Check yourself: Are your denominators predicted positives or actual positives?
Validation, leakage, and regularization
Fit preprocessing only on training data within each validation split. Select hyperparameters using validation results; reserve untouched test data for final evaluation. Regularization trades training fit against complexity: stronger penalties can reduce variance but increase bias. Training loss alone cannot establish generalization.
Check yourself: Did validation or test information influence the fitted scaler or feature selection?
Decision trees and split criteria
A classification tree evaluates a split using child impurities weighted by their sample fractions. For class proportions pₖ, Gini impurity is 1 − Σpₖ²; a pure node has zero impurity. Compare the weighted child impurity with the parent under the same criterion. Deep trees can memorize training data, so choose depth or pruning settings using validation, not test results.
Check yourself: Did you weight each child impurity by its sample fraction?
Clustering and dimensionality reduction
k-means alternates nearest-centroid assignments with mean updates to reduce within-cluster squared distance; initialization affects the result. PCA finds orthogonal directions of greatest variance in centered data. Neither method uses target labels. Feature scaling changes distances and variance, so it changes what these methods emphasize.
Check yourself: Does the chosen scale reflect meaningful feature comparisons?