Convolutional Neural Networks (CNN)
Computer Vision & Working with Images
A detailed beginner guide explaining how machines “see” images: from pixels and color channels to convolution and pooling, famous architectures (ResNet, YOLO), transfer learning, object detection, segmentation, and hands-on projects — with numeric examples and everyday analogies.
Image Processing Basics
Before building a CNN, we need to understand how an image is stored on a computer. An image is not a “picture” in the visual sense — it is a table of numbers. A neural network does not “see”; it performs math on those numbers.
Pixels
A pixel (Picture Element) is the smallest unit of an image — a tiny square. Each pixel holds a numeric value representing color intensity or brightness.
Think of an image as a mosaic: each small tile = one pixel. Zoom in close on an old TV and you see RGB squares — those are pixels. Resolution = pixel count (e.g. 1920×1080 ≈ 2 million pixels).
A tiny 4×4 image = 16 pixels. Each value ranges from 0 (black) to 255 (white):
80, 120, 200, 255,
30, 60, 90, 110,
200, 180, 160, 140 ]
This 4×4 matrix is what the computer “sees.” A CNN slides filters over these numbers to detect patterns (edges, gradients).
Channels
A channel = a separate layer of data per pixel. In RGB, each pixel has 3 values: red, green, blue. Grayscale uses one channel only.
RGB
RGB = Red, Green, Blue. Any screen color is a mix of these three. Values 0–255 per channel:
White
R=255, G=255, B=255 — all channels on.
Black
R=0, G=0, B=0 — no light.
Pure red
R=255, G=0, B=0.
Purple
R=128, G=0, B=128 — red + blue.
A “light sky blue” pixel might be: R=135, G=206, B=235. A CNN learns that clusters of such values represent “sky,” “water,” or “grass.”
Grayscale
A grayscale image has one channel. Each pixel = brightness from 0 (black) to 255 (white). Convert RGB with:
Green is weighted more because the human eye is most sensitive to it. MNIST (handwritten digits) uses 28×28 grayscale — ideal for beginners.
| Type | Channels | Example | Tensor shape |
|---|---|---|---|
| Grayscale | 1 | MNIST, X-rays | (28, 28, 1) |
| RGB | 3 | CIFAR-10, selfies | (32, 32, 3) |
| RGBA | 4 | PNG with transparency | (H, W, 4) |
| Multi-spectral | 5+ | Satellite, medical | (H, W, N) |
Before feeding images to the network, divide by 255 to get 0–1 — or use mean/std (e.g. ImageNet stats). Without this, training is slow or fails to converge. Example: img = img / 255.0
CNN Architecture
Why not use a fully connected network for images? A 224×224×3 image = 150,528 inputs. One layer → 1000 neurons = 150 million weights! CNNs solve this with three ideas: weight sharing, local focus, and hierarchical features.
Convolution — step-by-step example
Convolution = a small window (filter) slides over the image, multiplies values, and sums them. Output = a feature map.
3×3 filter (detects vertical edges):
[-1, 0, 1]
[-1, 0, 1]
3×3 patch from the image (brightness values):
[10, 55, 95]
[12, 52, 88]
Sum: (-1×10)+(0×50)+(1×90)+(-1×10)+(0×55)+(1×95)+(-1×12)+(0×52)+(1×88) = 191
High value = “strong vertical edge here.” Repeat at every position → full feature map.
Slide a 3×3 lens over the image and ask: “Do I see a vertical line?” A vertical-edge filter gives high values at edges and low values elsewhere.
Filters
A filter = a weight matrix learned during training. Early layers: edges. Deep layers: eyes, wheels, faces.
Input 128×128×3 → output 128×128×32 — 32 maps, each searching for a different pattern.
Kernels
Kernel = filter (same meaning in most contexts). Common sizes: 3×3, 5×5, 7×7. Larger kernel = wider view but more parameters.
Feature maps
Each filter → one feature map. 64 filters → 64 maps. The hierarchy:
nn.Conv2d(3, 32, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
nn.Conv2d(32, 64, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
nn.Flatten(), nn.Linear(64*56*56, 10)
)
Pooling
Pooling shrinks the image and keeps the most important information — a “summary” of a region. It reduces computation and adds translation invariance: a cat on the left or right — the model still recognizes it.
Max pooling
A 2×2 window → take the maximum value. Most common in CNNs.
224×224 → MaxPool 2×2 → 112×112
Average pooling
Average all values in the window — smoother, preserves more information.
Global pooling
GAP (Global Average Pooling): average each entire feature map → one vector. 512 maps of 7×7 → 512 numbers. Replaces large Flatten layers — ResNet uses it heavily.
| Type | Idea | When to use |
|---|---|---|
| Max pooling | Maximum value | Default — edges, textures |
| Average pooling | Mean value | Smoothing |
| Global pooling | Mean of entire map | Before classification |
CNN Layers — in detail
Each layer has hyperparameters that control output shape. Understanding them = easier debugging and better architecture choices.
Padding
Add zeros (or other values) around image borders before convolution. Same padding keeps size: 224×224 stays 224×224.
5×5 image, 3×3 filter, no padding → 3×3 (shrinks!). With padding=1 → 5×5 (same size).
Stride
Stride = how many pixels the filter moves. Stride=1 (slow, high detail). Stride=2 (faster, downsampling like pooling).
Flatten
After Conv+Pool, a 3D tensor (e.g. 7×7×512) → 1D vector (25,088). Connects CNN to dense layers for classification.
Dense layers
At the end: Flatten → Dense(128) → ReLU → Dropout → Dense(num_classes) → Softmax. Dense = every neuron connected to every input.
model = keras.Sequential([
layers.Conv2D(32, 3, activation='relu', padding='same'),
layers.MaxPooling2D(2),
layers.Conv2D(64, 3, activation='relu'),
layers.Flatten(),
layers.Dense(128, activation='relu'),
layers.Dropout(0.5),
layers.Dense(10, activation='softmax')
])
Famous Architectures — from LeNet to EfficientNet
Each architecture is a “recipe” for stacking layers. You rarely build from scratch — use pre-built models from PyTorch (torchvision.models) or TensorFlow.
LeNet-5 (1998)
Yann LeCun — MNIST digits. Conv → Pool → Conv → FC. First commercially successful CNN (check reading).
AlexNet (2012)
ImageNet revolution — 8 layers, ReLU, Dropout, GPU. ~26% better error than SVM. Started the deep learning era.
VGG (2014)
Repeated 3×3 layers — VGG16 (16 layers). Simple but ~138M parameters.
GoogLeNet (2014)
Inception modules — multiple filter sizes in parallel. 22 layers but fewer params than VGG.
ResNet (2015)
Skip connections — solved vanishing gradient. ResNet-50/101/152. Industry standard today.
DenseNet (2017)
Every layer connects to all following layers — feature reuse, fewer parameters.
EfficientNet (2019)
Balances depth/width/resolution — high accuracy with fewer params. EfficientNet-B0 to B7.
GoogLeNet — Inception module
Inception idea: “I don’t know the best filter size — use them all!” In one layer: 1×1, 3×3, 5×5 conv + MaxPool — then concatenate. 1×1 conv reduces channels before large conv (bottleneck) — saves parameters.
ResNet — skip connections in detail
Instead of signal passing through 50 layers (and weakening), ResNet adds a shortcut: output = F(x) + x. If F(x)≈0, the layer is “skipped” — training deep networks (152+ layers) becomes feasible.
| Architecture | Year | ImageNet Top-5 | Parameters | Notes |
|---|---|---|---|---|
| LeNet-5 | 1998 | — | 60K | MNIST |
| AlexNet | 2012 | 15.3% | 60M | Revolution start |
| VGG-16 | 2014 | 7.3% | 138M | Simple design |
| ResNet-50 | 2015 | 3.6% | 25M | Most used |
| EfficientNet-B0 | 2019 | ~3.3% | 5M | High efficiency |
resnet = models.resnet50(weights='IMAGENET1K_V2')
efficient = models.efficientnet_b0(weights='IMAGENET1K_V1')
Transfer Learning
Training ResNet on ImageNet (1.2M images) takes weeks on GPU. Transfer learning: take a pre-trained model and adapt it to your task — high accuracy with little data.
Feature extraction
Freeze all pre-trained CNN layers — use them as a ready-made “feature extractor.” Add only new dense layers for classification.
- Load ImageNet-trained ResNet-50.
- Freeze all layers:
for p in model.parameters(): p.requires_grad = False - Replace final Dense with Dense(2) — cat/dog.
- Train only the last layer — 10 epochs may be enough!
- ~95%+ accuracy with only 500 images.
Fine tuning
After feature extraction, unfreeze the last 1–3 CNN blocks and train with a small learning rate (0.0001). The model adapts general features (edges, shapes) to your domain (cancer types, plant species).
- Your data is very different from ImageNet (X-rays, satellite).
- Feature extraction alone is not enough.
- You have enough data (1000+).
for param in model.layer4.parameters():
param.requires_grad = True
optimizer = optim.Adam(
filter(lambda p: p.requires_grad, model.parameters()),
lr=1e-4
)
Object Detection
Classification says: “This image contains a cat.” Detection says: “Cat at (x=120, y=80) size 200×150, and dog at (300, 200).” You need: what + where.
R-CNN (2014)
Region proposals → CNN per region → SVM. Accurate but very slow — seconds per image.
Fast R-CNN (2015)
One CNN on full image + ROI pooling. ~10× faster than R-CNN.
Faster R-CNN (2015)
RPN — Region Proposal Network learns proposals. End-to-end, ~5 FPS.
YOLO (2016+)
You Only Look Once — image → grid, each cell predicts boxes + classes. Real-time — 30–100+ FPS.
SSD (2016)
Single Shot Detector — multi-scale feature maps. Speed/accuracy balance between YOLO and Faster R-CNN.
YOLO — how it works simply
- 640×640 image → CNN backbone (e.g. CSPDarknet).
- Split into grid (e.g. 20×20).
- Each cell predicts: bounding box (x,y,w,h) + confidence + class probabilities.
- Non-Max Suppression: remove overlapping boxes — keep the best.
- Output: “person 0.92 at [100,50,200,400]”, “car 0.87 at [300,200,500,350]”
Measures overlap between predicted and ground-truth box. IoU > 0.5 = “correct detection.” mAP (mean Average Precision) = standard metric on benchmarks like COCO.
from ultralytics import YOLO
model = YOLO('yolov8n.pt')
results = model('street.jpg') # detect cars, pedestrians, signs
| Model | Speed | Accuracy | Use case |
|---|---|---|---|
| Faster R-CNN | Slow (~5 FPS) | High | Research, critical accuracy |
| SSD | Medium (~20 FPS) | Good | Mobile, embedded |
| YOLOv8 | Fast (30–100 FPS) | Good–high | Real-time, CCTV, robotics |
Segmentation
Detection draws a rectangle around an object. Segmentation labels every pixel: “this pixel is cat,” “this is background,” “this is road.” Critical for medicine, self-driving, photo editing.
FCN (2015)
Fully Convolutional Network — first end-to-end segmentation. Replaces FC with conv — output heatmap near full image size.
U-Net (2015)
Encoder-decoder + skip connections in a U shape. Excellent for medical images — works with little data.
Mask R-CNN (2017)
Faster R-CNN + branch for per-object pixel masks. Instance segmentation — separate mask per cat.
DeepLab (2018)
Atrous (dilated) conv — larger receptive field without downsampling. Precise semantic segmentation.
Semantic vs instance segmentation
U-Net — why the U shape?
Input: 256×256 brain MRI. U-Net → 256×256 mask — each pixel: 0=background, 1=tumor. Dice coefficient measures overlap with doctor’s mask — >0.85 is excellent.
Hands-on Projects — from idea to model
The best way to learn: a real project. Four popular domains with practical steps for beginners.
Image classification
- Data: 60,000 images 32×32 — airplane, car, cat...
- Preprocessing: normalize, augmentation (flip, crop).
- Model: 3 Conv blocks + MaxPool + Dense — or ResNet-18 transfer learning.
- Training: Adam, lr=0.001, 50 epochs, early stopping.
- Expected: simple CNN ~75%, ResNet transfer ~93%+.
train_set = datasets.CIFAR10('./data', train=True, transform=transforms.ToTensor())
model = models.resnet18(weights='IMAGENET1K_V1')
model.fc = nn.Linear(512, 10) # 10 classes
Face recognition
Pipeline: face detection (MTCNN or Haar) → face embedding (FaceNet, ArcFace) → cosine similarity comparison.
1. Detect face in camera. 2. CNN (e.g. FaceNet) maps face → 128-number vector (embedding). 3. Enrollment: store your embedding. 4. Unlock: compare new embedding vs stored — if similarity > threshold → unlock.
Folders faces/person1/, faces/person2/ — 20 photos each. Use facenet-pytorch or deepface. ~95%+ with 20 photos/person.
Medical imaging
X-rays, MRI, CT — CNN + transfer learning. Challenges: little data, class imbalance, critical accuracy.
| Task | Dataset | Model | Metric |
|---|---|---|---|
| Pneumonia from X-ray | ChestX-ray14 | DenseNet-121 + TL | AUC, Sensitivity |
| Tumor segmentation MRI | BraTS | 3D U-Net | Dice score |
| Diabetic retinopathy | EyePACS | EfficientNet | Quadratic Kappa |
OCR — text in images
Optical Character Recognition: image with text → digital text. Modern pipeline: detection (where is text?) + recognition (what does it say?).
- YOLO or EAST — detect plate region.
- Crop + perspective correction.
- CNN + CTC or Transformer (CRNN) — read characters.
- Output: “ABC 1234”
Ready tools: Tesseract (classic), EasyOCR, PaddleOCR — CNN + RNN/Transformer inside.
reader = easyocr.Reader(['en', 'ar'])
result = reader.readtext('document.jpg')
for bbox, text, conf in result:
print(f'{text} ({conf:.2f})')
- Days 1–2: MNIST with simple CNN — understand Conv+Pool.
- Days 3–5: CIFAR-10 + transfer learning (ResNet).
- Days 6–8: Your project: 2–3 classes from phone photos.
- Days 9–10: YOLO — detect objects in video.
- Days 11–12: EasyOCR — read Arabic/English text.
- Days 13–14: GitHub README + demo — document your work!