Calculus, Probability & Statistics for AI
How does a model "learn"? How does it measure uncertainty? This lesson explains the two mathematical engines of training: calculus (derivatives, gradient, gradient descent) and probability (Bayes, normal distribution, descriptive statistics) — in plain language with equations and diagrams.
Why Are Calculus and Probability the Foundation of AI Engineering?
AI needs two intertwined branches of math: calculus tells the model how to improve, and probability & statistics tell it how confident to be. When you press "Train" in PyTorch or TensorFlow, what happens behind the scenes is differential calculus (derivatives, gradient, gradient descent) on a cost function built from probability and statistics.
Limits
A limit answers: "What happens to a function as \(x\) approaches a certain value?" We don't always need \(x\) to reach that value — what matters is the behavior near it.
\[\lim_{x \to a} f(x) = L\]
Meaning: as \(x\) gets closer to \(a\), \(f(x)\) gets closer to \(L\).
Example: \(\lim_{x \to 2} (3x + 1) = 7\) — plug in \(x=2\) directly because the function is "smooth."
Why limits matter
- Derivatives are defined as a limit: instantaneous rate of change
- Continuity — the function doesn't "jump" suddenly
- Activation functions — ReLU at \(x=0\) uses left and right limits
\(\lim_{x \to 0} \dfrac{\sin x}{x} = 1\) — a classic result used in deriving activation functions and probability formulas.
At \(x=0\), ReLU \(\max(0,x)\) has right limit 0 and left limit 0 — so we define \(f'(0)=0\) in practice. softmax and sigmoid derivatives use limits to ensure numerical stability at large values.
Derivatives
A derivative measures the rate of change: if you increase the input a tiny bit, how much does the output change? Visually: the slope of the tangent line at a point on the curve.
\[f'(x) = \dfrac{df}{dx} = \lim_{h \to 0} \dfrac{f(x+h) - f(x)}{h}\]
Read as: "change in \(f\) divided by a tiny change in \(x\)"
Basic differentiation rules
| Function | Derivative |
|---|---|
| \(c\) (constant) | \(0\) |
| \(x^n\) | \(nx^{n-1}\) |
| \(e^x\) | \(e^x\) |
| \(\ln x\) | \(\dfrac{1}{x}\) |
| \(\sin x\) | \(\cos x\) |
If \(f(x) = x^2\), then \(f'(x) = 2x\). At \(x=3\): derivative = 6 — a tiny increase in \(x\) increases \(f\) by about 6 times as much.
Every weight in a network has a partial derivative of the loss. If \(\dfrac{\partial L}{\partial w} > 0\), decreasing \(w\) reduces loss. Libraries (autograd) compute these automatically — but understanding derivatives helps you interpret training behavior.
Gradient
When a function has multiple variables (like network weights), there isn't one derivative but a gradient: a vector collecting all partial derivatives, pointing in the direction of steepest ascent.
For \(f(x_1, x_2, \ldots, x_n)\):
\[\nabla f = \begin{bmatrix} \dfrac{\partial f}{\partial x_1} \\ \dfrac{\partial f}{\partial x_2} \\ \vdots \\ \dfrac{\partial f}{\partial x_n} \end{bmatrix}\]
The symbol \(\nabla\) (nabla) means "gradient."
Key properties
- The gradient is perpendicular to contour lines
- Gradient magnitude = maximum rate of change
- Negative gradient \(-\nabla f\) points toward steepest descent
\(f(x,y) = x^2 + y^2\). Gradient: \(\nabla f = [2x,\ 2y]^T\). At \((3,4)\): \(\nabla f = [6,8]^T\) — the function rises fastest in this direction.
Cost function \(J(\theta)\) has thousands or millions of parameters \(\theta\). Gradient \(\nabla_\theta J\) tells each weight: should I increase or decrease? This is the heart of backpropagation.
Partial Derivatives
When a function depends on multiple inputs, a partial derivative measures: what happens to the output if I change one variable only and hold the rest fixed?
\[\dfrac{\partial f}{\partial x_i} = \lim_{h \to 0} \dfrac{f(x_1,\ldots,x_i+h,\ldots,x_n) - f(x_1,\ldots,x_n)}{h}\]
The symbol \(\partial\) means "differentiate with respect to \(x_i\) while holding others constant."
Simple example
\(f(x,y) = 3x^2 + 2xy + y^2\)
\(\dfrac{\partial f}{\partial x} = 6x + 2y\) (treat \(y\) as constant)
\(\dfrac{\partial f}{\partial y} = 2x + 2y\) (treat \(x\) as constant)
At \((x,y)=(1,2)\): \(\dfrac{\partial f}{\partial x} = 6+4=10\), \(\dfrac{\partial f}{\partial y} = 2+4=6\). Gradient: \(\nabla f = [10, 6]^T\).
A network with 10 million weights = a function with 10 million variables. Each \(\dfrac{\partial L}{\partial w_{ij}}\) tells us the effect of one connection weight. Partial derivatives are the building blocks of the full gradient.
Chain Rule
Neural networks are composed functions: layer upon layer. The chain rule tells us how to differentiate a function inside a function — the foundation of backpropagation.
If \(y = f(g(x))\):
\[\dfrac{dy}{dx} = \dfrac{dy}{du} \cdot \dfrac{du}{dx} \quad \text{where } u = g(x)\]
For multiple layers: \(\dfrac{dy}{dx} = \dfrac{dy}{du_2} \cdot \dfrac{du_2}{du_1} \cdot \dfrac{du_1}{dx}\)
\(y = (2x+1)^3\). Let \(u = 2x+1\), so \(y = u^3\).
\(\dfrac{dy}{du} = 3u^2\), \(\dfrac{du}{dx} = 2\) → \(\dfrac{dy}{dx} = 3(2x+1)^2 \cdot 2 = 6(2x+1)^2\)
Backprop = chain rule on a computational graph. Loss flows backward: \(\dfrac{\partial L}{\partial w^{(l)}} = \dfrac{\partial L}{\partial a^{(l)}} \cdot \dfrac{\partial a^{(l)}}{\partial z^{(l)}} \cdot \dfrac{\partial z^{(l)}}{\partial w^{(l)}}\). Without the chain rule, there is no deep learning.
Optimization
Optimization finds the best parameter values — usually the minimum of a function (cost or loss). In machine learning: find weights that minimize model error.
\[\theta^* = \arg\min_\theta \, J(\theta)\]
\(\theta^*\) = best parameters, \(J(\theta)\) = cost function
Types of critical points
- Local minimum — lowest point in a neighborhood (may not be global best)
- Global minimum — lowest point on the entire surface
- Saddle point — zero derivative but not a valley
First and second order conditions
First order: \(\nabla J(\theta^*) = 0\) (gradient is zero at optimum)
Second order: Hessian matrix \(H\) positive definite → confirmed local minimum
Neural network loss surfaces are non-convex — thousands of local minima. So we use iterative methods (gradient descent, Adam) instead of closed-form solutions. Optimization in AI = practical search for "good enough."
Gradient Descent
Gradient descent is a simple, powerful algorithm: start from random weights, compute the gradient, walk opposite to the gradient (because we want to reduce loss), repeat.
\[\theta_{t+1} = \theta_t - \eta \, \nabla_\theta J(\theta_t)\]
\(\eta\) = learning rate — step size
Types of gradient descent
| Type | How gradient is computed | Note |
|---|---|---|
| Batch | All training data | Slow but stable |
| Stochastic (SGD) | One sample | Fast but noisy |
| Mini-batch | Batch (32–256) | Most common on GPU |
Learning rate \(\eta\)
- Too large — overshoots the valley, oscillates or diverges
- Too small — training extremely slow
- Scheduling — reduce \(\eta\) over time (cosine, step decay)
Minimize \(J(w) = w^2\). Gradient: \(\nabla J = 2w\). At \(w_0=5\), \(\eta=0.1\):
\(w_1 = 5 - 0.1 \times 10 = 4\), \(w_2 = 4 - 0.8 = 3.2\), … converges toward \(w^*=0\).
Adam, RMSprop, and Momentum are improvements on basic SGD — but all follow the same idea: follow the gradient (or its estimate) to update weights. SGD + mini-batch + GPU = speed of training massive models.
Probability
Probability measures how likely an event is — a number between 0 (impossible) and 1 (certain). In AI, models often output probabilities, not definitive answers.
Sample space \(\Omega\) — all possible outcomes
Event \(A\) — a subset of \(\Omega\)
\[0 \leq P(A) \leq 1, \quad P(\Omega) = 1, \quad P(\emptyset) = 0\]
Addition and multiplication rules
Addition rule (mutually exclusive): \(P(A \cup B) = P(A) + P(B)\)
Multiplication rule (independent): \(P(A \cap B) = P(A) \cdot P(B)\)
Complement: \(P(A^c) = 1 - P(A)\)
Fair die roll: \(P(\text{even}) = P(\{2,4,6\}) = \dfrac{3}{6} = 0.5\). A model predicting "even with probability 0.52" expresses uncertainty.
Softmax classification outputs probabilities summing to 1. Generation (LLMs) picks the next token from a probability distribution. Cross-entropy measures distance between predicted and true probabilities.
Random Variables
A random variable (RV) maps each outcome in the sample space to a number. Instead of asking "what happened?" we ask "what is the numerical value?"
Two main types
| Type | Values | AI example |
|---|---|---|
| Discrete | Countable integers | Classification: {cat, dog, bird} |
| Continuous | Any number in a range | Price prediction, temperature |
PMF (discrete): \(P(X = x)\) — probability of a specific value
PDF (continuous): \(f(x)\) where \(P(a \leq X \leq b) = \int_a^b f(x)\,dx\)
CDF: \(F(x) = P(X \leq x)\) — probability that \(X\) is at most \(x\)
Flip a coin twice. \(X\) = number of heads. \(X \in \{0,1,2\}\).
\(P(X=0)=\frac{1}{4}\), \(P(X=1)=\frac{1}{2}\), \(P(X=2)=\frac{1}{4}\). Mean \(E[X]=1\).
Classification outputs = discrete random variables. Regression = estimating a continuous variable. VAEs and GANs assume probability distributions over generated data.
Bayes' Theorem
Bayes' theorem converts "probability before seeing evidence" into "probability after seeing evidence" — the foundation of Naive Bayes classifiers, Kalman filters, and many probabilistic models.
\[P(A \mid B) = \dfrac{P(B \mid A) \cdot P(A)}{P(B)}\]
Prior \(P(A)\) — belief before evidence
Posterior \(P(A \mid B)\) — belief after evidence \(B\)
Likelihood \(P(B \mid A)\) — how well evidence fits the hypothesis
Rare disease: \(P(\text{disease})=0.01\). Test 99% accurate: \(P(+|\text{disease})=0.99\), \(P(-|\text{healthy})=0.99\).
Positive result: \(P(\text{disease}|+) = \dfrac{0.99 \times 0.01}{0.99 \times 0.01 + 0.01 \times 0.99} \approx 0.5\) — positive doesn't mean certain!
Naive Bayes assumes feature independence and computes \(P(\text{class}|\text{features})\) directly. Bayesian Neural Networks place distributions over weights. LLMs update "beliefs" token by token — a Bayesian idea in generation.
Conditional Probability
Conditional probability \(P(A \mid B)\) means: "What is the probability of \(A\) given that \(B\) has occurred?" We narrow the sample space to \(B\) only.
\[P(A \mid B) = \dfrac{P(A \cap B)}{P(B)} \quad \text{when } P(B) > 0\]
If \(A\) and \(B\) are independent: \(P(A \mid B) = P(A)\) — knowing \(B\) changes nothing
Law of total probability
\[P(A) = \sum_i P(A \mid B_i) \cdot P(B_i)\]
Where \(B_i\) partition the sample space — used to compute \(P(B)\) for Bayes
Box: 3 red balls, 2 blue. Draw two without replacement. \(P(\text{second red} \mid \text{first red}) = \dfrac{2}{4} = 0.5\) — because 2 red remain out of 4.
Language models estimate \(P(\text{word} \mid \text{context})\). Masked language modeling (BERT) predicts a masked word given the rest of the sentence. Conditional GANs generate images given a class label.
Mean
The mean (average) is the "center of mass" of data — sum of values divided by count. It summarizes a set of numbers into one number.
Sample: \(\bar{x} = \dfrac{1}{n}\sum_{i=1}^{n} x_i\)
Random variable: \(E[X] = \mu = \sum x \cdot P(X=x)\) (discrete) or \(E[X] = \int x f(x)\,dx\) (continuous)
Scores: 70, 85, 90, 75, 80. Mean = \(\dfrac{70+85+90+75+80}{5} = 80\).
Batch Normalization subtracts the batch mean. Mean loss over a batch = cost function. \(E[X]\) appears in MSE and bias-variance analysis.
Median
The median is the middle value after sorting data — half the values are smaller, half are larger. Less sensitive to outliers than the mean.
Sort values ascending. If \(n\) is odd: median = middle value. If \(n\) is even: average of the two middle values.
Example: [3, 7, 9, 15, 100] → median = 9 (mean = 26.8 — skewed by 100)
For model performance reports, median is sometimes clearer than mean (e.g., latency with outliers). Median pooling in CNNs takes the middle value in a window — robust to noise.
Mode
The mode is the most frequently occurring value in data. For discrete data: "the most common." For continuous data: it may be the interval where density is highest.
Colors: red, blue, red, green, red, blue → mode = red (3 times).
In imbalanced classification, the mode of classes may be the majority — a simple baseline. In generation, the mode of a distribution = most probable value (greedy decoding in LLMs).
Variance
Variance measures how spread out data is around the mean — how far values deviate from center. Large variance = scattered; small = clustered.
Sample: \(s^2 = \dfrac{1}{n-1}\sum_{i=1}^{n}(x_i - \bar{x})^2\)
Population: \(\sigma^2 = E[(X - \mu)^2] = E[X^2] - (E[X])^2\)
Data: 2, 4, 4, 4, 5, 5, 7, 9. Mean = 5. Squared deviations: 9,1,1,1,0,0,4,16 → sum=32. Sample variance = 32/7 ≈ 4.57.
Weight variance in regularization. Prediction variance in ensemble methods. VAEs minimize latent representation variance. Understanding variance helps diagnose overfitting.
Standard Deviation
Standard deviation is the square root of variance — same units as the original data, so easier to interpret than variance.
\[\sigma = \sqrt{\sigma^2} = \sqrt{E[(X-\mu)^2]}\]
Sample: \(s = \sqrt{s^2}\)
Input normalization: \((x - \mu) / \sigma\) puts features on the same scale. BatchNorm uses batch standard deviation. Weight initialization (Xavier, He) depends on desired activation standard deviation.
Normal Distribution
The normal (Gaussian) distribution is the symmetric "bell curve" — the most common in nature and statistics. It describes many phenomena arising from many random factors.
\[f(x) = \dfrac{1}{\sigma\sqrt{2\pi}} \exp\!\left(-\dfrac{(x-\mu)^2}{2\sigma^2}\right)\]
Notation: \(X \sim \mathcal{N}(\mu, \sigma^2)\) — mean \(\mu\), variance \(\sigma^2\)
68-95-99.7 rule
- About 68% within \(\mu \pm \sigma\)
- About 95% within \(\mu \pm 2\sigma\)
- About 99.7% within \(\mu \pm 3\sigma\)
Heights with \(\mu=170\) cm, \(\sigma=10\). Probability between 160–180 ≈ 68%. Above 190 cm (+\(2\sigma\)) ≈ 2.5% only.
Gaussian noise assumption in MSE. Gaussian weight initialization. VAEs assume latent \(\mathcal{N}(0,I)\). Diffusion models add Gaussian noise gradually then learn to remove it.
Loss Functions
A loss function measures how wrong the model is on one sample (or small batch). The smaller it is, the closer the prediction to truth.
Common loss functions
| Function | Formula | Use |
|---|---|---|
| MSE | \(\frac{1}{2}(y - \hat{y})^2\) | Regression |
| MAE | \(|y - \hat{y}|\) | Regression (robust) |
| Cross-Entropy | \(-\sum y_i \log \hat{y}_i\) | Classification |
| Binary CE | \(-[y\log\hat{y}+(1-y)\log(1-\hat{y})]\) | Binary classification |
| Hinge | \(\max(0, 1 - y\hat{y})\) | SVM |
Loss choice defines what the model is penalized for. MSE penalizes large errors quadratically. Cross-entropy suits probabilities. Focal loss handles class imbalance in detection.
Cost Functions
A cost function (or objective) aggregates losses over all training samples — usually the mean — and may add regularization penalties. Training = minimizing the cost function.
Empirical risk (mean loss):
\[J(\theta) = \dfrac{1}{m}\sum_{i=1}^{m} L\bigl(f(x^{(i)};\theta),\, y^{(i)}\bigr)\]
With L2 regularization:
\[J(\theta) = \dfrac{1}{m}\sum_{i=1}^{m} L^{(i)} + \dfrac{\lambda}{2}\|\theta\|_2^2\]
Loss vs. cost
3 samples, MSE: errors 0.5, 1.0, 0.3. Cost = \(\frac{1}{3}(0.25 + 1 + 0.09) \approx 0.447\). With \(\lambda=0.01\) and \(\|\theta\|^2=100\): add \(0.5\) to cost.
This lesson tied it all together: calculus gives us the gradient, the gradient drives gradient descent, probability describes model outputs and uncertainty, statistics summarizes data, and loss/cost functions define what we optimize. With linear algebra, you now have the full mathematical foundation for reading papers and understanding training code.