Convolutional Neural Networks Guide: Filters, Pooling, and Image Recognition

Images are not merely long lists of numbers. They are spatial arrangements in which nearby pixels form edges, edges form textures and parts, and parts combine into recognizable objects. A convolutional neural network, or CNN, is designed around that structure. Instead of learning a separate relationship between every pixel and every hidden unit, it repeatedly scans small learnable filters across an image. This simple idea gives CNNs an efficient way to detect visual patterns wherever they appear. In this guide, we will build the idea carefully, connect the equations to practical design choices, and see how convolution, pooling, receptive fields, classifier heads, regularization, and transfer learning work together in an image-recognition system.

Why fully connected networks struggle with images

A fully connected, or dense, layer treats its input as a vector. If a color image has height $H$, width $W$, and $C$ channels, flattening it produces $HWC$ input values. A dense layer with $M$ outputs then needs

$$HWC \times M + M$$

trainable parameters, including the biases. For a modest $224 \times 224$ RGB image and 1,000 hidden units, that is

$$224 \times 224 \times 3 \times 1000 + 1000 = 150{,}529{,}000$$

parameters in only the first layer. Storing those weights is expensive, training them requires a large amount of data and computation, and the resulting model can memorize the training set rather than learn reusable visual concepts.

Parameter count is only part of the problem. Flattening destroys explicit spatial organization. Pixels that were adjacent in the image may be far apart in the vector, while the dense layer has no built-in reason to treat nearby pixels differently from distant ones. It must learn from data that a horizontal edge is made from a particular local arrangement of light and dark pixels.

A dense layer also lacks a natural mechanism for reusing a detector across locations. Suppose it learns to recognize a vertical edge near the left side of an image. A separate set of weights is responsible for recognizing the same edge near the right side. The model can eventually learn both cases, but doing so wastes parameters and examples. Real visual patterns are often useful wherever they occur: a corner remains a corner in the top-left, center, or bottom-right.

CNNs address these weaknesses with two complementary ideas:

• Local connectivity: a unit observes a small spatial neighborhood rather than the whole image.
• Parameter sharing: the same detector is applied at every valid position.

These assumptions are examples of an inductive bias: the architecture encodes a useful expectation about the data before learning begins. The assumption is not that location never matters. Rather, it says that local patterns recur across locations and should be detected consistently. Later layers can still learn spatial arrangements of those patterns.

Local connectivity and parameter sharing

Imagine a $3 \times 3$ window moving across an image. At each position, the nine pixel values in the window are combined with nine learned weights. A dense layer would use different weights for every output location. A convolutional layer reuses the same nine weights everywhere. Those weights form a kernel, also called a filter.

For a color image, the filter spans all input channels. A $3 \times 3$ filter applied to RGB data therefore has shape $3 \times 3 \times 3$. It can learn a pattern involving red, green, and blue together. If the layer produces 64 output channels, it contains 64 different filters, each with its own weights and bias. The parameter count is

$$(3 \times 3 \times 3 + 1) \times 64 = 1{,}792.$$

This count does not depend on image width or height. The same layer can, in principle, operate on a $32 \times 32$ image or a $320 \times 320$ image because its parameters describe local detectors, not absolute pixel positions.

Local connectivity reduces complexity, while parameter sharing makes the learned representation approximately translation equivariant. Equivariance means that if an object shifts in the input, its feature response shifts correspondingly in the output. It is not the same as translation invariance, where the final prediction remains exactly unchanged. Convolution preserves where a feature was found; pooling and the final aggregation can make the prediction less sensitive to small shifts.

The convolution operation

Despite the name, deep-learning libraries usually implement cross-correlation rather than the mathematically flipped form of convolution. Because the kernel values are learned, this distinction does not limit the model: a learned flipped kernel could represent the equivalent operation.

For a single-channel input $X$ and kernel $K$ of height $K_h$ and width $K_w$, the output at location $(i,j)$ can be written as

$$Y_{i,j} = b + \sum_{u=0}^{K_h-1}\sum_{v=0}^{K_w-1} K_{u,v}X_{i+u,j+v},$$

where $b$ is a learned bias. Each output value is a weighted summary of one local input patch.

With multiple input channels, every filter includes one two-dimensional kernel slice per channel:

$$Y_{i,j,f} = b_f + \sum_{c=0}^{C_{\text{in}}-1} \sum_{u=0}^{K_h-1} \sum_{v=0}^{K_w-1} K_{u,v,c,f}X_{i+u,j+v,c}.$$

The index $f$ identifies an output filter. Applying all filters across all positions creates an output tensor called a feature map or activation volume. Its channels do not usually correspond to colors. They are learned feature dimensions. One early channel might respond strongly to diagonal contrast, another to green-to-brown transitions, and another to tiny repeating textures.

The filters are not hand-coded. During training, backpropagation calculates how each filter weight affected the loss, and an optimizer adjusts the weights. The network discovers useful filters for the objective and dataset. Visualizing first-layer filters often reveals edge and color detectors, but deeper filters are harder to interpret because they respond to combinations distributed across many channels.

A small numerical convolution

Consider this input and kernel:

$$X = \begin{bmatrix} 1 & 2 & 0 \\ 3 & 1 & 2 \\ 0 & 1 & 3 \end{bmatrix}, \qquad K = \begin{bmatrix} 1 & 0 \\ -1 & 1 \end{bmatrix}.$$

With stride 1, no padding, and no bias, the top-left output is

$$(1)(1) + (2)(0) + (3)(-1) + (1)(1) = -1.$$

Moving one column right gives

$$(2)(1) + (0)(0) + (1)(-1) + (2)(1) = 3.$$

Repeating the operation produces a $2 \times 2$ feature map. The sign and magnitude at each location indicate how strongly the local patch matches the filter's learned pattern.

Here is the same basic idea in NumPy:

import numpy as np

image = np.array([
    [1, 2, 0],
    [3, 1, 2],
    [0, 1, 3],
], dtype=float)

kernel = np.array([
    [1, 0],
    [-1, 1],
], dtype=float)

output = np.empty((2, 2))
for row in range(2):
    for col in range(2):
        patch = image[row:row + 2, col:col + 2]
        output[row, col] = np.sum(patch * kernel)

print(output)
# [[-1.  3.]
#  [ 4.  1.]]

Production frameworks perform this operation in optimized parallel code and automatically differentiate through it, but the essential calculation remains a sliding dot product.

Padding, stride, dilation, and output size

Four quantities determine the spatial geometry of a convolution: input size, kernel size, padding, stride, and dilation. Understanding them prevents shape errors and helps you control information loss.

Padding

Without padding, a filter can be centered only where its entire window lies inside the image. As a result, the feature map becomes smaller after each layer, and border pixels participate in fewer outputs than central pixels.

Padding adds values around the input boundary, usually zeros. If $P=1$, one row or column is added on each side. For an odd-sized kernel with stride 1 and dilation 1, “same” padding commonly keeps output height and width equal to input height and width. For a $3 \times 3$ kernel, that requires one padded cell per side.

Zero padding is convenient but introduces an artificial boundary. Alternatives such as reflection or replication padding can be useful for restoration and generation tasks where edge behavior matters. For ordinary classification, zero padding is a reliable default.

Stride

The stride $S$ is the distance between consecutive filter positions. A stride of 1 examines neighboring positions. A stride of 2 skips every other position and roughly halves each spatial dimension. Strided convolution therefore performs feature extraction and downsampling in one learned operation.

Larger strides reduce memory and computation, but they can discard small details. An early stride of 2 may be reasonable for high-resolution photographs, while it can erase important strokes in tiny digit images.

Dilation

Dilation $D$ spaces kernel elements apart. A dilated $3 \times 3$ kernel with dilation 2 samples a $5 \times 5$ region while still using nine weights. Its effective kernel size is

$$K_{\text{eff}} = D(K-1)+1.$$

Dilated convolution expands the receptive field without an equivalent increase in parameters or immediate reduction in resolution. It is especially useful in semantic segmentation, audio models, and contexts where dense output is required. Very large dilation values can create “gridding” effects because the kernel samples disconnected positions.

The output-size formula

For one spatial dimension, the general output size is

$$O = \left\lfloor \frac{I + 2P - D(K-1) - 1}{S} \right\rfloor + 1,$$

where:

• $I$ is the input size,
• $K$ is the kernel size,
• $P$ is padding on each side,
• $S$ is stride,
• $D$ is dilation,
• and $\lfloor\cdot\rfloor$ rounds down.

Apply the formula independently to height and width when their settings differ.

For a numeric example, take a $32 \times 32$ image, a $5 \times 5$ kernel, padding $P=2$, stride $S=2$, and dilation $D=1$:

$$O = \left\lfloor \frac{32 + 2(2) - 1(5-1) - 1}{2} \right\rfloor + 1 = \left\lfloor\frac{31}{2}\right\rfloor + 1 = 16.$$

The output is $16 \times 16$ per filter. If the layer has 48 filters, its complete output shape is $16 \times 16 \times 48$ in channels-last notation.

Now change only dilation to 2. The effective kernel becomes $9 \times 9$, and

$$O = \left\lfloor \frac{32 + 4 - 2(4) - 1}{2} \right\rfloor + 1 = \left\lfloor\frac{27}{2}\right\rfloor + 1 = 14.$$

Dilation expanded the sampled region and reduced the output because the original padding was no longer sufficient for a “same” result.

Activation after convolution

A convolution by itself is a linear operation. Stacking several linear convolutions without nonlinear activations is still equivalent to one linear transformation, no matter how many layers are used. To model complex decision boundaries, networks normally apply an activation function after each convolution.

The rectified linear unit is a common choice:

$$\operatorname{ReLU}(z)=\max(0,z).$$

ReLU is computationally simple and preserves positive responses without saturating them. Variants such as Leaky ReLU, GELU, and SiLU allow different behavior around zero and can improve optimization in some architectures. The right activation is an empirical design choice, but omitting nonlinearities accidentally is almost always a serious error.

Many convolutional blocks also include normalization. A traditional pattern is convolution, batch normalization, then activation. Batch normalization stabilizes channel statistics during training and often allows faster optimization. Modern architectures may use layer normalization or group normalization, especially when batches are small.

An activation value should not be interpreted as a guaranteed symbolic statement such as “this neuron is a wheel detector.” Features are often distributed: a concept may be represented by a pattern across several channels and positions. Nevertheless, activations are useful measurements of which learned local templates match the current input.

Pooling and translation tolerance

A pooling layer summarizes a local neighborhood independently in each channel. It has no learned kernel weights in its standard form.

Max pooling keeps the largest value:

$$y = \max(x_1,x_2,\ldots,x_n).$$

For a $2 \times 2$ window, it asks whether a strong response exists anywhere in that small region. If an edge detector moves by one pixel but remains inside the same window, the maximum may stay similar. This gives the next layers some tolerance to small translations and deformations.

Average pooling computes the mean:

$$y = \frac{1}{n}\sum_{i=1}^{n}x_i.$$

It preserves the average strength of evidence rather than only the strongest response. Average pooling is smoother, but it can weaken a sparse, highly informative activation. Max pooling was historically common within CNN feature extractors; average pooling is now especially common as a final global aggregation.

Pooling reduces spatial dimensions, memory use, and computation. It also increases the effective receptive field of subsequent units. However, translation tolerance is not perfect translation invariance. A shift crossing pooling-window boundaries can change the result, and downsampling can produce aliasing if high-frequency patterns are not adequately filtered.

Modern networks sometimes replace max pooling with stride-2 convolutions. A strided convolution learns what information to retain, whereas pooling uses a fixed rule. Neither choice is universally superior. The important principle is to downsample gradually enough that task-relevant detail survives.

Stacking convolutional blocks and receptive fields

Early layers observe small pixel neighborhoods. Later layers consume earlier feature maps, so their units indirectly depend on increasingly large regions of the original image. That region is a unit's receptive field.

Two consecutive $3 \times 3$ stride-1 convolutions have an effective receptive field of $5 \times 5$, not $6 \times 6$. The first layer sees three pixels across. Moving across three adjacent first-layer outputs adds one new input pixel at each side, producing five. Three such convolutions reach $7 \times 7$.

This stacking is attractive because three $3 \times 3$ layers provide a $7 \times 7$ receptive field while introducing three nonlinear transformations. For equal channel widths, they can also use fewer parameters than one $7 \times 7$ convolution.

A precise receptive-field calculation tracks both receptive-field size $r_l$ and the spacing, or jump, $j_l$ between neighboring features:

$$j_l = j_{l-1}S_l,$$ $$r_l = r_{l-1} + \left(K_{\text{eff},l}-1\right)j_{l-1}.$$

Starting with $r_0=1$ and $j_0=1$, a $3 \times 3$ convolution gives $r_1=3$. A $2 \times 2$ stride-2 pool then gives $r_2=4$ and $j_2=2$. Another $3 \times 3$ convolution gives $r_3=8$ because each kernel step now jumps two original pixels.

Receptive field explains the visual hierarchy learned by CNNs. Early layers can represent edges because their view is small. Middle layers combine edges into textures and motifs. Deep layers combine motifs across broader regions into object parts and scene-level evidence.

The theoretical receptive field tells us which pixels can affect a unit. The effective receptive field in a trained model is often smaller and concentrated near the center because not all computational paths contribute equally. Deep architecture, skip connections, dilation, and training all influence how broadly information is actually used.

From feature maps to a classifier

After several convolutional blocks, the network has a compact tensor of high-level features. A classification head must turn that tensor into class scores called logits.

The older approach is to flatten the feature maps into one long vector and feed it to one or more dense layers. If the final tensor has shape $7 \times 7 \times 512$, flattening creates 25,088 values. Connecting that vector to 4,096 units requires more than 102 million weights. This creates substantial capacity but also substantial overfitting risk.

The more economical alternative is global average pooling (GAP). It averages each channel over all spatial positions:

$$g_c = \frac{1}{HW}\sum_{i=1}^{H}\sum_{j=1}^{W}X_{i,j,c}.$$

A $7 \times 7 \times 512$ tensor becomes a vector of only 512 channel summaries. A final dense layer maps that vector to class logits. GAP dramatically reduces parameters and allows more flexible input resolutions, provided the feature extractor itself supports them.

Flattening preserves exact spatial arrangement for the head, which may help when absolute location matters. GAP encourages each channel to represent evidence that can be aggregated across location. For ordinary object classification, GAP is usually a strong default. Detection and segmentation should not collapse spatial dimensions this way because they need localized outputs.

For mutually exclusive classes, logits commonly pass through softmax:

$$p_k = \frac{e^{z_k}}{\sum_j e^{z_j}}.$$

For multi-label tasks, each class usually receives an independent sigmoid probability. Confusing these two problem types leads to the wrong loss and misleading outputs.

Classic CNN ideas and what they taught us

CNN history is useful because each influential architecture made a design lesson visible.

LeNet: a compact visual hierarchy

LeNet-5, associated with handwritten digit recognition, combined convolution, subsampling, nonlinearities, and dense classification. By modern standards it is small, but its structure demonstrated that local feature extraction could be learned end to end. It established the recognizable rhythm of convolution followed by spatial reduction, with deeper features feeding a classifier.

The lasting lesson is architectural alignment with data. A modest network can perform well when the input is simple, the resolution is small, and the inductive bias is appropriate. More depth is not automatically better.

AlexNet: scale, ReLU, and practical deep learning

AlexNet's ImageNet success in 2012 showed that a large CNN trained with GPUs could substantially outperform previous image-classification systems. It popularized ReLU activations, dropout in the dense head, overlapping max pooling, and aggressive data augmentation. Its early convolution used a large kernel and stride, partly reflecting the computational constraints of the time.

AlexNet also demonstrated that architecture, data, and hardware advance together. The concept of convolution was not new, but enough labeled data, efficient GPU computation, and effective training practices made a much deeper model practical.

VGG: depth through repeated small filters

VGG networks used a simple, regular design: stacks of $3 \times 3$ convolutions, occasional $2 \times 2$ pooling, and increasing channel counts as resolution decreased. Repeating small filters made the architecture easy to understand and showed that depth could improve representation quality.

VGG is computationally heavy and its dense head contains many parameters, so it is rarely the most efficient modern choice. Its enduring lesson is that multiple small convolutions can replace one large convolution while adding nonlinear depth.

ResNet: learning residual changes

As networks grew deeper, optimization became difficult. Surprisingly, adding layers could increase training error, even when the deeper network should theoretically imitate the shallower one. ResNet introduced skip connections:

$$y = F(x) + x.$$

Instead of forcing a block to learn a complete transformation from $x$ to $y$, it learns a residual correction $F(x)$. If no change is needed, setting the residual near zero preserves the input. The identity path also gives gradients a direct route backward through the network.

Skip connections do not merely “prevent vanishing gradients,” and they do not guarantee every deep model will train. Their deeper intuition is that layers can refine an existing representation instead of rebuilding it. This makes very deep networks easier to optimize and inspired residual ideas across vision, language, and generative modeling.

When dimensions change, the skip path cannot be added directly. A $1 \times 1$ convolution, possibly with stride, can project the input to matching spatial and channel dimensions. A $1 \times 1$ convolution also mixes channel information cheaply without examining a wider spatial neighborhood.

Data augmentation and overfitting

Vision models can overfit even when they contain far fewer parameters than a dense alternative. A model may memorize backgrounds, camera artifacts, watermarks, or subject identities rather than the intended category. High training accuracy with much lower validation accuracy is a classic warning sign.

Data augmentation generates varied training examples by applying label-preserving transformations. Common choices include:

• random horizontal flips for objects whose identity is unchanged by reflection;
• small crops, translations, rotations, and zooms;
• brightness, contrast, saturation, and hue variation;
• random erasing or cutout, which hides a patch;
• MixUp, which blends pairs of images and labels;
• CutMix, which pastes a region from one image into another.

Augmentation encodes assumptions and must match the task. Horizontal flipping is inappropriate when distinguishing left from right. Vertical flipping may be harmful for street scenes. Large rotations can turn a $6$ into something resembling a $9$. Color jitter is risky if color defines the class. The best augmentation policy reflects variations expected at deployment.

Apply random augmentation only to the training set. Validation and test sets should use deterministic preprocessing so metrics are comparable. Normalization must also be consistent. If pretrained weights expect a particular channel order, resolution, and scaling rule, follow that rule for both training and inference.

Other regularization tools include weight decay, dropout, label smoothing, early stopping, and reducing model capacity. Dropout is often used in classifier heads, though heavy dropout throughout convolutional blocks is not always helpful. Weight decay and augmentation are frequently stronger first choices.

Data splitting deserves equal attention. If near-duplicate frames from one video appear in both training and validation sets, the validation score can be falsely optimistic. For medical images, split by patient; for product photos, consider splitting by product instance; for temporal data, respect time. A trustworthy split tests generalization to genuinely unseen cases.

Transfer learning

Training a vision model from random initialization can require a large labeled dataset. Transfer learning begins with a feature extractor pretrained on another dataset, then adapts it to the target task. Early and middle visual features such as edges, textures, and shapes often transfer well.

A typical workflow is:

1. Load a pretrained backbone without its original classifier.
2. Apply the exact preprocessing expected by that backbone.
3. Add a new pooling layer and task-specific classifier.
4. Freeze the backbone and train the new head.
5. Optionally unfreeze some or all backbone layers.
6. Fine-tune with a smaller learning rate.

Freezing keeps pretrained weights fixed, making initial training fast and reducing overfitting on small datasets. Fine-tuning can adapt higher-level features to the new domain. Unfreezing everything immediately with a large learning rate may destroy useful representations, a problem called catastrophic forgetting.

Transfer works best when source and target domains share useful visual structure. A natural-image model can still help on many specialized tasks, but the advantage may shrink for satellite bands, microscopy, or other unusual modalities. Self-supervised pretraining and domain-specific pretraining can bridge that gap.

Be careful with batch normalization during fine-tuning. Its running statistics and trainable parameters can behave poorly with tiny batches. Depending on the framework and dataset, keeping normalization layers in inference mode or using a lower learning rate can be safer.

A practical architecture-building workflow

Start with the simplest model capable of learning the task. For small RGB images, a useful baseline might contain two or three stages. Each stage uses one or two $3 \times 3$ convolutions with “same” padding and ReLU, followed by a moderate downsampling step. Increase channels as spatial size decreases—for example, 32, then 64, then 128. Finish with global average pooling and a small classifier.

Track the tensor shape after every stage. For an input of $64 \times 64 \times 3$, three stride-2 reductions produce spatial sizes $32$, $16$, and $8$. If the final stage has 128 channels, GAP produces 128 values regardless of the remaining width and height.

Also track parameters and receptive field. A model with a tiny receptive field may classify textures but fail to capture object structure. A model that shrinks resolution too early may miss fine detail. A huge head may dominate the parameter budget without improving the feature extractor.

Train the baseline long enough to diagnose it:

• If training and validation accuracy are both poor, consider underfitting, optimization issues, incorrect labels, or inadequate receptive field.
• If training is strong and validation is weak, improve augmentation, regularization, splitting, or data quality.
• If loss does not decrease, inspect normalization, learning rate, output/loss compatibility, and gradients.
• If validation is unstable, check sample count, class imbalance, and whether evaluation preprocessing is deterministic.

Do not select architecture solely by accuracy. Consider precision and recall per class, calibration, inference latency, memory, robustness, and error severity. Inspect misclassified images; aggregate metrics cannot reveal a shortcut learned from backgrounds or a systematic failure on one subgroup.

Common CNN mistakes

Using the wrong input shape

Frameworks disagree about image layout. TensorFlow/Keras usually uses channels-last tensors shaped (batch, height, width, channels), while PyTorch usually uses (batch, channels, height, width). Passing a tensor in the wrong order can trigger a shape error or, worse, produce a model that runs while treating width as channels.

Always print one batch shape and verify it against the model declaration. Confirm grayscale images include a channel dimension such as $H \times W \times 1$.

Skipping normalization

Raw pixels often range from 0 to 255. Scaling to $[0,1]$, standardizing by dataset statistics, or applying a pretrained model's prescribed preprocessing usually improves optimization. The critical requirement is consistency between training, validation, and production.

Never estimate normalization statistics using test data. That leaks information from evaluation into training. Compute them on the training split only.

Pooling too aggressively

Repeated downsampling can reduce a feature map to almost nothing. Four halvings turn a $28 \times 28$ input into approximately $1 \times 1$ or $2 \times 2$, depending on rounding and padding. Tiny objects or narrow strokes may disappear much earlier.

Write down spatial sizes before building the model. For low-resolution inputs, use fewer pooling stages or delay the first reduction.

Building an oversized dense head

Flattening large feature maps into a wide dense layer can create most of the model's parameters in one step. This increases memory use and overfitting. Try global average pooling and compare validation performance before assuming a large head is necessary.

Mismatching outputs, labels, and loss

Single-label multiclass classification typically uses one logit per class and categorical cross-entropy. Binary classification can use one sigmoid logit or two softmax logits, but the labels and loss must agree. Multi-label classification needs independent sigmoid outputs. Also check whether the chosen loss expects raw logits or already-normalized probabilities.

Ignoring train and evaluation modes

Dropout and batch normalization behave differently during training and inference. Forgetting to switch modes can make evaluation noisy or update statistics using validation data. High-level training APIs usually manage modes, but custom loops require explicit care.

Practice with the Solver360 architecture designer

Equations become easier when you can manipulate an architecture and immediately observe its shapes. Open the free interactive CNN Calculator and create a small network from the input upward.

Begin with a $32 \times 32 \times 3$ input and a convolution using 16 filters, a $3 \times 3$ kernel, stride 1, and padding 1. Predict the output before reading the result: spatial dimensions should remain $32 \times 32$, while depth becomes 16. Then change stride to 2 and confirm that the spatial output becomes $16 \times 16$.

Next, add a $2 \times 2$ max-pooling layer with stride 2. Observe both the shape reduction and the growing receptive field. Add another convolution with 32 filters. Notice that increasing filter count changes channel depth but does not by itself change width or height.

Try a deliberately problematic design: remove padding from several $5 \times 5$ convolutions and add pooling after each one. Watch how quickly spatial dimensions collapse. Then repair the network using smaller kernels, same padding, and less frequent downsampling.

Finally, compare flattening with global average pooling. Calculate the input size of the dense classifier in each case and estimate its parameter count. This experiment makes a central design principle concrete: a small shape decision near the head can change the total parameter count by orders of magnitude.

Frequently asked questions

Are CNN filters manually designed or learned?

In a modern CNN, filters are learned from data. They normally start as small random values or values chosen by an initialization scheme. During the forward pass, each filter produces feature maps. The loss measures how far predictions are from targets, and backpropagation computes a gradient for every filter coefficient. The optimizer then updates those coefficients.

Early filters often resemble familiar edge or color detectors because such patterns are broadly useful and statistically common. This does not mean the network was instructed to find edges. Deeper filters are conditioned on earlier learned channels and may not have a simple visual description. Engineers still choose filter size, number of channels, stride, dilation, and connectivity, but training learns the coefficients.

How many filters should a convolutional layer use?

There is no formula that gives the universally correct channel count. More filters increase representational capacity and computational cost. A common pattern is to increase channels when reducing spatial resolution: 32 channels at a large resolution, then 64, 128, and so on. This shifts capacity from “where” detail toward richer feature types as the network goes deeper.

Treat those numbers as starting points, not laws. Small datasets, mobile deployment, or simple images may need fewer channels. Fine-grained classification may benefit from more. Compare models using a validation set and measure memory and latency, not just training accuracy. Channel count often has a roughly quadratic effect inside blocks because both input and output channel dimensions grow.

Does convolution make a network translation invariant?

Convolution is more accurately described as translation equivariant: shifting the input tends to shift the feature map. A final classifier can become tolerant to shifts through pooling, augmentation, broad receptive fields, and spatial aggregation, but exact invariance is not guaranteed. Padding boundaries, strides, aliasing, and object interactions all affect responses.

That distinction matters. A detector should preserve location; a classifier may want to ignore location; a segmentation network must output location. CNNs support all three because spatial information remains available until the architecture deliberately aggregates or downsamples it.

When should I use max pooling instead of a strided convolution?

Max pooling is parameter-free and keeps the strongest local activation, making it a simple, effective baseline. A strided convolution learns which channel combinations and spatial patterns to preserve during downsampling. It is more flexible but adds parameters and can overfit.

Use the choice made by a trusted pretrained architecture when transferring weights. For a new model, compare both if efficiency and quality matter. More important than the specific operation is whether downsampling is too early or severe. Anti-aliased pooling or a smoothing step can improve shift stability when aliasing is a concern.

Why are small kernels such as $3 \times 3$ so common?

A $3 \times 3$ kernel captures immediate horizontal, vertical, and diagonal relationships while remaining cheap. Stacking small kernels grows the receptive field and inserts nonlinear activations between stages. Two $3 \times 3$ layers reach a $5 \times 5$ receptive field, while three reach $7 \times 7$ under stride 1.

Large kernels are not obsolete. Modern architectures sometimes use $5 \times 5$, $7 \times 7$, or even much larger depthwise kernels efficiently. The tradeoff depends on parameterization and hardware. Small kernels remain a dependable default because they balance local detail, depth, parameter count, and optimized implementation.

What is the difference between a feature map and a filter?

A filter is a set of trainable weights. It defines the local pattern being tested. A feature map is the result of applying that filter across an input for one example. One filter generally produces one output channel, although grouped and depthwise convolutions modify how input and output channels connect.

Filters belong to the model and persist across examples. Feature maps depend on the current input and are recomputed on every forward pass. During training, intermediate feature maps may be retained temporarily because backpropagation needs them, which is why activation memory can exceed parameter memory.

Why does validation accuracy stop improving while training accuracy rises?

That pattern usually indicates overfitting, but the underlying cause still requires investigation. The training set may be too small, augmentations may be weak, the model may be too large, labels may contain noise, or train and validation distributions may differ. Leakage and duplicate images can also distort the picture.

Inspect validation loss and per-class metrics, not only accuracy. Review mistakes manually. Then try stronger task-appropriate augmentation, weight decay, early stopping, a smaller head, transfer learning, or better data. If the validation set is tiny, its metric may simply have high variance, so repeated splits or cross-validation can help.

Can a CNN accept images of different sizes?

Convolution and pooling themselves can operate on varying spatial dimensions. A flatten-then-dense head usually fixes the expected size because the dense layer requires a fixed number of inputs. Global average pooling removes that restriction by producing one value per channel regardless of height and width.

In practice, batches still need compatible tensor sizes, so loaders commonly resize, crop, pad, or group similar resolutions. The task may also require a minimum resolution because repeated downsampling and receptive-field assumptions break on very small images. Pretrained models frequently have a recommended input size even if their operations technically accept others.

How do depthwise separable convolutions reduce computation?

A standard convolution jointly mixes spatial and channel information with a kernel shaped $K_h \times K_w \times C_{\text{in}} \times C_{\text{out}}$. A depthwise convolution first applies one spatial filter per input channel. A $1 \times 1$ pointwise convolution then mixes channels. This factorization is much cheaper when channel counts are large.

MobileNet and related efficient architectures use this idea extensively. The tradeoff is that factorization constrains the operation, so accuracy and real hardware speed should be measured rather than inferred from parameter count alone. Some devices execute standard dense convolutions exceptionally efficiently.

Next steps: CNNs and vision transformers

CNNs build locality and parameter sharing directly into the architecture. This makes them data-efficient, computationally practical, and naturally suited to images. Vision transformers, by contrast, usually divide an image into patches and use attention to model relationships among patch representations. Attention can connect distant regions directly, whereas a conventional CNN grows global context gradually through depth and downsampling.

The boundary is not absolute. Modern CNNs borrow transformer-inspired training and design ideas, while vision transformers use convolution-like patch embeddings, hierarchical stages, or local attention. Both can achieve excellent image recognition. CNNs remain strong when data or compute is limited, low-latency deployment matters, and locality is a helpful prior. Transformers often shine with large-scale pretraining and tasks that benefit from flexible global interactions.

A productive next step is not to choose a winner in the abstract. Build and understand a compact CNN first. Track its shapes, parameter counts, receptive fields, training curves, and mistakes. Then compare it with a small vision transformer under the same dataset, augmentation, and evaluation protocol. That controlled comparison reveals more than headline benchmark numbers.

The essential CNN lesson is simple but powerful: exploit spatial structure. Local filters reuse knowledge across an image; nonlinear stacks turn pixels into hierarchical features; controlled downsampling trades detail for context; and a carefully chosen head converts those features into predictions. Once these mechanics are clear, architectures from LeNet to ResNet stop looking like collections of mysterious layers and become understandable choices about information, geometry, and optimization.