K-Means Clustering Explained: Centroids, Elbow Method, and When It Fails

Clustering begins with a deceptively simple question: which observations resemble one another? K-means answers by representing each group with a center, called a centroid, and repeatedly improving those centers until nearby observations gather around them. The method is fast, intuitive, and widely useful, but its apparent simplicity can hide strong assumptions. To use it responsibly, we need to understand not only how it works, but also what its objective rewards, how preprocessing changes the result, how to choose the number of clusters, and how to recognize data that k-means was never designed to handle.

From Supervised Learning to Unsupervised Learning

In supervised learning, every training example includes both input features and a target. A house-price model might receive floor area, age, and location as inputs, together with the known sale price. A classifier might receive an image and a label such as “cat” or “dog.” The target provides a direct teaching signal: predictions can be compared with correct answers, and parameters can be adjusted to reduce an explicit error.

Unsupervised learning removes that target. We observe feature vectors $x_1,x_2,\ldots,x_n$, but no supplied class, price, or outcome tells us what structure to recover. The task is therefore less constrained. We might seek compact representations, unusual observations, latent factors, probability densities, or groups of similar cases. Clustering is one important unsupervised task, but “unsupervised” does not mean “objective” or “free of human choices.” The selected features, distance metric, scaling method, algorithm, and hyperparameters all encode judgments about what similarity should mean.

K-means performs partitioning clustering. Given a requested number $k$, it divides $n$ observations into $k$ non-overlapping clusters. Each observation belongs to exactly one cluster, and each cluster is summarized by the arithmetic mean of its members. Unlike a supervised classifier, k-means does not discover pre-existing class names. Cluster 0 and cluster 1 have no inherent semantics, and their numeric identifiers can switch between runs without changing the solution.

This distinction matters in practice. If customers are clustered from purchase behavior, k-means may produce useful segments, but it has not proved that there are “true customer types.” It has found a partition that fits a mathematical criterion. Humans must inspect, interpret, validate, and decide whether that partition serves a legitimate purpose.

The K-Means Objective: Within-Cluster Sum of Squares

Suppose each observation $x_i$ is a vector in $\mathbb{R}^d$. We want $k$ clusters $C_1,\ldots,C_k$, with centroid $\mu_j$ for cluster $C_j$. K-means minimizes the within-cluster sum of squares, often abbreviated WCSS, SSE, or inertia:

$$J = \sum_{j=1}^{k}\sum_{x_i \in C_j}\lVert x_i-\mu_j\rVert_2^2.$$

The inner term is the squared Euclidean distance from an observation to its assigned centroid. The objective adds these squared distances across every observation. A small value means that observations are, collectively, close to their cluster centers.

For a fixed set of observations assigned to one cluster, the arithmetic mean is the point that minimizes the sum of squared Euclidean distances. That fact explains the “means” in k-means. If cluster $C_j$ contains $n_j$ observations, its optimal centroid is

$$\mu_j = \frac{1}{n_j}\sum_{x_i\in C_j}x_i.$$

Squaring distance has important consequences. First, large deviations receive disproportionate weight. An observation 10 units from a centroid contributes $100$, whereas one 2 units away contributes only $4$. Outliers can therefore pull a centroid strongly. Second, the objective naturally favors compact clusters centered around means. It does not directly reward separation between clusters, density connectivity, or semantic usefulness. Third, inertia cannot be compared naively across different feature scalings because changing units changes distances and their squares.

The global optimization problem is computationally difficult in general. The familiar iterative algorithm, commonly called Lloyd’s algorithm, finds a local optimum rather than guaranteeing the best possible partition. Different initial centroids can lead to different final answers. This is why initialization and repeated runs matter.

The Algorithm: Initialize, Assign, Update, Repeat

K-means alternates between two operations that each improve, or at least do not worsen, the objective.

1. Initialize the centroids

Choose $k$ starting points $\mu_1,\ldots,\mu_k$. They may be sampled from the observations or selected with a smarter seeding procedure such as k-means++.

2. Assign each observation

For each $x_i$, find the nearest centroid under squared Euclidean distance:

$$c_i = \arg\min_{j\in\{1,\ldots,k\}}\lVert x_i-\mu_j\rVert_2^2.$$

This assignment partitions the feature space into Voronoi cells. Every location is associated with its nearest centroid, so boundaries between clusters are linear hyperplanes.

3. Update each centroid

Recompute each centroid as the mean of all observations assigned to it:

$$\mu_j \leftarrow \frac{1}{|C_j|}\sum_{x_i\in C_j}x_i.$$

The update moves the representative point to the location that minimizes squared error for that fixed membership.

4. Repeat until convergence

Assignment and update continue until assignments stop changing, centroid movement falls below a tolerance, or a maximum iteration count is reached. Because there are finitely many possible assignments and each ordinary step cannot increase WCSS, the process converges. However, convergence means reaching a stable local solution, not necessarily the globally best one.

For dense data, one iteration has a rough computational cost of $O(nkd)$: each of $n$ observations is compared with $k$ centroids in $d$ dimensions. This favorable scaling helps explain the method’s popularity.

A Worked Two-Dimensional Example

Consider six points:

$$A=(1,1),\quad B=(1,2),\quad C=(2,1),$$ $$D=(7,7),\quad E=(8,7),\quad F=(7,8).$$

Let $k=2$, and suppose the initial centroids are

$$\mu_1^{(0)}=(1,1),\qquad \mu_2^{(0)}=(8,7).$$

During assignment, compare each point’s squared distance to both centroids. For $B=(1,2)$,

$$\lVert B-\mu_1^{(0)}\rVert^2=(1-1)^2+(2-1)^2=1,$$

while

$$\lVert B-\mu_2^{(0)}\rVert^2=(1-8)^2+(2-7)^2=74.$$

Therefore $B$ joins cluster 1. For $D=(7,7)$, the squared distances are

$$\lVert D-\mu_1^{(0)}\rVert^2=6^2+6^2=72$$

and

$$\lVert D-\mu_2^{(0)}\rVert^2=(-1)^2+0^2=1,$$

so $D$ joins cluster 2. The other comparisons produce

$$C_1=\{A,B,C\},\qquad C_2=\{D,E,F\}.$$

Now update the centroids. For the first cluster,

$$\mu_1^{(1)} =\left(\frac{1+1+2}{3},\frac{1+2+1}{3}\right) =\left(\frac{4}{3},\frac{4}{3}\right).$$

For the second,

$$\mu_2^{(1)} =\left(\frac{7+8+7}{3},\frac{7+7+8}{3}\right) =\left(\frac{22}{3},\frac{22}{3}\right).$$

Repeating assignment with these updated centroids leaves every point in the same cluster, so the algorithm has converged. We can calculate WCSS. In cluster 1, the squared distances to $(4/3,4/3)$ are $2/9$, $5/9$, and $5/9$, totaling $4/3$. By symmetry, cluster 2 also contributes $4/3$. Total WCSS is therefore

$$J=\frac{8}{3}\approx2.667.$$

This example separates cleanly, but it also illustrates a subtle point: k-means assigns based on the current centroids, not on pairwise friendship among points. With noisier data or unfortunate initial seeds, early assignments may guide the algorithm toward another local optimum.

Initialization: Random Seeds and K-Means++

A basic initialization chooses $k$ observations at random as initial centroids. It is inexpensive, but it can choose several seeds from the same natural region while leaving another region unrepresented. The neglected region’s points may initially attach to a distant centroid, and subsequent updates can converge to a poor partition. Random initialization also makes results vary across runs.

One defense is to run k-means multiple times with different seeds and keep the result with the lowest inertia. In scikit-learn this behavior is controlled by n_init. Repetition does not prove global optimality, but it substantially reduces the chance of accepting an obviously poor local solution.

K-means++ improves the starting positions. It selects the first centroid randomly, then chooses each subsequent centroid with probability proportional to the observation’s squared distance from its nearest already-selected centroid. Points far from existing seeds are more likely to be chosen. This spreads the initial centroids across the data while retaining randomness.

The intuition is valuable: if a region is already represented, another point from that region adds less initial coverage than a distant point. K-means++ usually converges faster and reaches better objectives than naive random seeding. It is the standard default in many libraries, but it is not magic. Outliers may receive high selection probability, high-dimensional distances can be uninformative, and multiple starts remain prudent.

Reproducibility also deserves attention. Fixing a random seed makes experiments repeatable, which is useful for debugging and reporting. Yet a robust conclusion should not depend entirely on one lucky seed. Examine stability across multiple initializations, bootstrap samples, or nearby values of $k$.

Feature Scaling Is Mandatory

K-means treats Euclidean distance as the definition of similarity. Consequently, a feature’s numeric scale directly controls its influence. Imagine customer data with annual income measured in dollars, ranging from $20{,}000$ to $200{,}000$, and satisfaction measured from 1 to 10. A $1{,}000$ income difference contributes $1{,}000{,}000$ to squared distance, while a five-point satisfaction difference contributes only $25$. Without scaling, income overwhelms satisfaction even if satisfaction is more meaningful.

Standardization is a common remedy:

$$z_{ij}=\frac{x_{ij}-\bar{x}_j}{s_j},$$

where $\bar{x}_j$ and $s_j$ are the training-set mean and standard deviation of feature $j$. This gives each standardized feature roughly zero mean and unit variance. Min-max scaling, robust scaling based on medians and interquartile ranges, or domain-specific transformations may be more suitable in some settings.

Scaling is not a clerical step; it defines geometry. Giving every feature unit variance asserts that one standard deviation of change is comparably important across dimensions. That assumption should be examined rather than applied blindly. Domain expertise may justify weights, transformations, or exclusion of variables.

Fit the scaler on training data only when clustering is part of a production pipeline. Reusing statistics from the full dataset can leak information from future or held-out observations. Apply the fitted scaler to new observations before asking the model for a cluster assignment.

Categorical variables create another problem. Integer-encoding colors as red $=1$, green $=2$, and blue $=3$ invents distances and ordering that do not exist. One-hot encoding may be usable, but mixed numerical and categorical data often calls for another distance measure or an algorithm such as k-prototypes. Plain k-means is designed for continuous numeric vectors whose means and Euclidean distances are meaningful.

Choosing the Number of Clusters

K-means requires $k$ before fitting. No single diagnostic can always reveal the correct value because “correct” depends on data geometry and the intended use.

The elbow method

Fit models over a range of values, such as $k=1$ through $k=10$, and plot inertia against $k$. Inertia never increases as $k$ grows: more centroids can fit the data at least as closely. In the extreme, if $k=n$, every point can become its own cluster and inertia is zero.

The elbow method looks for a bend where additional clusters produce diminishing improvement. A sharp decrease from $k=1$ to $k=3$ followed by modest decreases might suggest $k=3$. The method is a heuristic, not a statistical proof. Many real curves are smooth, several bends may look plausible, and the selected range can influence perception.

Silhouette analysis

For observation $i$, let $a(i)$ be its average distance to other observations in its own cluster. Let $b(i)$ be the smallest average distance from $i$ to observations in any other cluster. The silhouette coefficient is

$$s(i)=\frac{b(i)-a(i)}{\max\{a(i),b(i)\}}.$$

Values approach $1$ when an observation is much closer to its own cluster than to alternatives. Values near $0$ indicate a boundary case, and negative values suggest that another cluster may fit better. Averaging $s(i)$ gives a model-level summary.

Silhouette scores provide more information than inertia alone because they consider both cohesion and separation. However, they also tend to favor compact, well-separated groups and can undervalue legitimate elongated or density-based structures. Computing all pairwise distances can also be costly on very large datasets, so sampling may be necessary.

Domain knowledge and stability

Operational constraints can dominate geometric scores. A support team may be able to design only four service strategies, making twenty mathematically distinct segments unusable. A scientific theory may predict three regimes. Conversely, a neat $k=3$ solution has little value if clusters cannot be interpreted or acted upon.

Compare candidate values using several criteria: inertia shape, silhouette distribution, cluster sizes, stability across resamples, interpretability, and downstream usefulness. Avoid selecting $k$ solely because it maximizes one score. Cluster solutions should be treated as hypotheses about structure, then challenged.

A Small Scikit-Learn Workflow

The following example standardizes features, fits several candidates, and records both inertia and silhouette score:

from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X = [
    [1.0, 1.0], [1.0, 2.0], [2.0, 1.0],
    [7.0, 7.0], [8.0, 7.0], [7.0, 8.0],
]

for k in range(2, 5):
    model = make_pipeline(
        StandardScaler(),
        KMeans(n_clusters=k, init="k-means++", n_init=20, random_state=42),
    )
    labels = model.fit_predict(X)
    kmeans = model.named_steps["kmeans"]
    print(k, kmeans.inertia_, silhouette_score(
        model.named_steps["standardscaler"].transform(X), labels
    ))

For serious analysis, inspect the individual silhouette values, not only their average. Also invert the scaling when presenting centroids so stakeholders can understand them in original units. A standardized centroid of $0.8$ is less interpretable than an annual income of $74{,}000$.

Evaluation Without Labels Is Hard

In supervised learning, held-out labels provide a relatively direct measure of predictive error. Clustering has no such universal answer key. Internal metrics evaluate properties of the same feature geometry used to create the clusters, so they can reward an algorithm for satisfying its own assumptions.

Inertia measures compactness but always favors larger $k$. Silhouette combines compactness and separation but favors certain shapes. The Davies–Bouldin index compares within-cluster scatter with between-centroid separation; lower values are preferred, yet it too assumes centroid-like groups. The Calinski–Harabasz index compares between-cluster dispersion with within-cluster dispersion and often prefers well-separated convex groups.

External evaluation becomes possible when reference labels exist, using measures such as adjusted Rand index or normalized mutual information. But reference labels change the nature of the exercise. They may represent a different concept than the structure sought by clustering. For example, clusters based on shopping behavior need not match age categories, and disagreement does not automatically mean failure.

Stability is another useful lens. Refit on bootstrap samples, perturb the data slightly, or change initialization. If memberships change radically under minor perturbations, the proposed structure is fragile. Stability still cannot establish usefulness: a consistently reproducible partition may reflect a nuisance variable, data collection artifact, or ethically inappropriate attribute.

Ultimately, evaluation should connect to purpose. Are the groups interpretable? Do they persist in new data? Do they support better decisions in a controlled downstream test? Are cluster sizes practical? Do they avoid systematically harming a population? Unsupervised evaluation is difficult because mathematical neatness, scientific truth, and operational value are not the same thing.

Common Failure Modes

Empty clusters

An empty cluster occurs when no observation is assigned to a centroid during an assignment step. Its mean is undefined. This can happen with poor initialization, duplicate centroids, highly discrete data, or an excessive value of $k$.

Implementations handle emptiness in different ways. A common strategy relocates the empty centroid to a distant observation, perhaps the point with the largest current error. Another splits a populous cluster or keeps the previous centroid temporarily. The event should not be ignored: frequent empty clusters may indicate duplicate observations, too many requested groups, or unstable geometry.

Outliers

Means and squared distances make k-means sensitive to extreme points. One outlier can pull a centroid away from the dense body of its cluster. Several outliers can consume an entire cluster, leaving the remaining structure underrepresented.

Investigate extreme values before fitting. Correct data errors, apply justified transformations, use robust scaling, or analyze unusual observations separately. K-medoids is a more robust relative because each center is an actual observation and absolute-like distance objectives can reduce outlier influence. Trimmed clustering methods deliberately exclude a small fraction of points. Removing observations merely because they inconvenience the desired story, however, is not legitimate; exclusions need a defensible rationale.

Non-spherical and unequal clusters

K-means works best when groups are compact, roughly spherical in Euclidean geometry, similarly sized, and similarly dense. Its nearest-centroid boundaries are linear. Two interlocking moons, concentric rings, or a winding geographic corridor violate this geometry. K-means may slice such shapes into artificial pieces.

Unequal variance also causes trouble. A broad diffuse group and a tiny dense group may be partitioned in ways that reduce total squared error but do not preserve the apparent groups. Because every point contributes to the objective, large clusters can dominate improvements, and small clusters may be absorbed.

High dimensionality

As dimensionality grows, distances can concentrate: nearest and farthest observations become less distinguishable. Irrelevant features add noise, and visualization becomes unreliable. Feature selection, domain-informed embeddings, or dimensionality reduction may help. Principal component analysis can compress correlated numeric features before clustering, but the retained components and explained variance must be reviewed. A visually appealing two-dimensional projection is not proof that the full-dimensional clusters are sound.

Mini-Batch K-Means for Large Data

Standard k-means repeatedly scans the dataset. When millions of observations make full iterations costly, mini-batch k-means updates centroids using small random subsets. For a mini-batch, observations are assigned to current centroids, and each centroid is moved using an incremental mean based on the points assigned to it.

The method reduces memory pressure and often trains much faster. Updates are noisy, but with representative batches the final objective can be close to that of full k-means. Batch size trades speed against stability: very small batches are cheap but noisy, while larger batches better approximate a full update.

Mini-batch training is useful for document vectors, image descriptors, telemetry, and streaming-like workloads. Validate the approximation by fitting full k-means on a manageable sample or comparing inertia, stability, and downstream behavior. Faster convergence to a low inertia does not repair inappropriate features or non-spherical geometry.

Scikit-learn provides MiniBatchKMeans, with controls for batch size, initialization, reassignment of underused centers, and stopping. Scaling remains essential, and initialization remains consequential.

Alternatives and What They Model

Choosing an alternative should follow from the geometry and goal, not from a contest to name the most sophisticated algorithm.

Hierarchical clustering

Agglomerative hierarchical clustering starts with each observation in its own cluster and repeatedly merges clusters. A linkage rule defines inter-cluster distance. Single linkage uses the closest pair and can recover winding shapes, though it is vulnerable to chains of noise. Complete linkage uses the farthest pair and encourages compactness. Average linkage balances pairwise distances. Ward linkage merges clusters to minimize the increase in squared variance and is closely related to k-means geometry.

The resulting dendrogram exposes nested structure and lets analysts inspect several cut levels rather than fixing $k$ immediately. Hierarchical methods can be computationally and memory intensive for large $n$, and early merges are generally not undone.

Gaussian mixture models

A Gaussian mixture model, or GMM, assumes data comes from a weighted mixture of Gaussian distributions. Instead of hard assignments only, it estimates posterior membership probabilities. A point can be 70% associated with one component and 30% with another.

Full-covariance GMMs can represent elliptical clusters with different orientations and variances, making them more flexible than k-means. Expectation-maximization fitting still has local optima and initialization sensitivity. Probability estimates are meaningful only to the extent that the mixture assumptions describe the data, and flexible covariance estimates require enough observations.

K-means can be viewed loosely as a limiting special case of a spherical Gaussian mixture with equal variance and hard assignments. This perspective clarifies why k-means struggles when clusters have unequal covariance.

DBSCAN

DBSCAN defines clusters as dense connected regions. It uses a neighborhood radius $\varepsilon$ and a minimum number of nearby observations. Dense core points connect to other core points, border points attach to dense regions, and isolated observations can be labeled as noise.

DBSCAN can recover non-convex shapes and does not require $k$. It is particularly useful when noise detection matters. Its difficulty is parameter selection, especially when density varies across clusters. Distances also degrade in high dimensions, and scaling is still necessary. You can compare behavior interactively with the DBSCAN Calculator.

Other options include spectral clustering for graph-like or non-convex structure, k-medoids for robustness and arbitrary dissimilarities, and HDBSCAN for hierarchical density-based clustering with variable-density improvements. No algorithm eliminates the need to define meaningful features and validate the output.

Applications, Interpretation, and Misuse

K-means is used for customer segmentation, document grouping, image color quantization, vector-quantization codebooks, grouping sensor patterns, preliminary anomaly screening, and compressing large sets of vectors into prototypes. In image color quantization, for example, each pixel’s color vector is assigned to a centroid, and replacing pixels with centroid colors reduces the palette.

Centroids can make a solution interpretable. By comparing feature means across clusters, an analyst might describe one customer segment as frequent, low-value purchasers and another as infrequent, high-value purchasers. Such descriptions should be based on original units, distributions, and uncertainty—not just centroid values. Two clusters can share a centroid-like average while differing strongly in spread or internal composition.

A dangerous misuse is forcing clusters that are not there. K-means always returns $k$ groups when asked, even for a single continuous cloud with no meaningful gaps. The algorithm’s successful completion is not evidence of genuine categories. Segment names and colorful scatterplots can turn arbitrary boundaries into persuasive fiction.

Another misuse is treating cluster assignments as immutable identities. A person near a boundary may switch groups after a tiny data update. Segments derived from behavior can change over time. Attaching loaded labels such as “low value,” “risky,” or “unmotivated” can produce unjustified decisions, especially in employment, credit, education, medicine, or policing.

Sensitive attributes can influence clusters directly or through proxies. Even an unsupervised procedure can reproduce historical inequities. Audit cluster composition, downstream consequences, missingness patterns, and representation. In high-stakes settings, cluster membership should not become a substitute for individualized evidence.

Use clustering as an exploratory model: a structured way to propose patterns. Combine quantitative diagnostics with subject-matter review, temporal validation, uncertainty analysis, and, when appropriate, experiments that measure whether acting on segments improves outcomes.

Practice with Solver360

The K-Means Calculator is a useful place to connect formulas with movement on a plane. Enter a small two-dimensional dataset, choose initial centroids, and follow assignment and update steps. Try intentionally poor seeds placed near one natural group, then compare them with well-separated seeds. Watch how the final inertia and number of iterations change.

Next, add one distant outlier. Observe how a centroid moves toward it and how WCSS responds. Multiply one coordinate by 100 to simulate incompatible units, then compare the partition before and after scaling. Create two elongated diagonal bands and see whether nearest-centroid boundaries preserve them.

Finally, use the DBSCAN Calculator on a curved or noisy dataset. Comparing two algorithms on the same points teaches more than memorizing lists of advantages. Ask what each method calls a cluster, which observations it treats as ambiguous, and which parameters encode that definition.

A productive practice routine is:

1. Predict the next assignments by hand.
2. Calculate at least one centroid update.
3. State what geometry you expect the method to favor.
4. Run the interactive tool.
5. Explain any difference between your prediction and the result.
6. Change one assumption at a time and repeat.

The goal is not merely to obtain labels. It is to develop the habit of connecting labels to objective functions, preprocessing, and assumptions.

Frequently Asked Questions

Is k-means a classifier?

Not in the usual supervised sense. A classifier learns from examples with known class labels and attempts to predict those labels for new inputs. K-means receives no target labels and creates cluster assignments that minimize within-cluster squared distance. After fitting, it can assign a new observation to the nearest centroid, which may look like classification, but the output refers to learned cluster identifiers rather than known semantic classes. Sometimes practitioners cluster data and then train a classifier to reproduce those assignments, but this does not transform the original clusters into ground truth. If labeled categories are available and predicting them is the goal, supervised methods should generally be evaluated directly.

Why does k-means use the mean instead of the median?

The representative point must match the loss function. The arithmetic mean minimizes the sum of squared Euclidean distances, which is exactly the k-means objective. In one dimension, the median minimizes the sum of absolute distances instead. Replacing means with medians while retaining every other step would optimize a different objective. This connection also explains sensitivity to outliers: squared loss heavily penalizes distant points, so the optimal mean moves toward them. If robustness is important, k-medians, k-medoids, trimming, or an explicit outlier model may better match the desired behavior.

Does the lowest inertia identify the best value of k?

No. Inertia decreases monotonically as $k$ increases because additional centroids provide more flexibility. Without a complexity penalty, the absolute minimum over $1\leq k\leq n$ occurs at $k=n$, where each observation can have its own centroid. The elbow method looks for diminishing returns rather than the smallest raw value. Silhouette analysis, stability, interpretability, domain constraints, and validation on new data provide complementary evidence. Even agreement among several internal metrics does not guarantee that the partition is useful or real; those metrics often share assumptions about compact, separated geometry.

Can k-means handle categorical features?

Plain k-means is not naturally suited to nominal categories because arithmetic means and Euclidean distances are not meaningful for them. Encoding “Berlin,” “Cairo,” and “Lima” as 1, 2, and 3 creates a false order and false distance. One-hot encoding avoids explicit ordering, but it changes distance behavior and can cause high-cardinality variables to dominate. Methods designed for categorical data, such as k-modes, use modes and matching dissimilarity. K-prototypes combines numeric and categorical terms for mixed data. Whatever method is used, feature weighting and the meaning of similarity must be justified.

What should I do if different runs return different clusters?

First, use k-means++ and multiple initializations, retaining the solution with the lowest inertia. Fix a random seed when you need exact reproducibility, but do not confuse reproducibility with robustness. Compare solutions across seeds using membership agreement and centroid locations. If several solutions have similar inertia but substantially different memberships, the data may not strongly support a unique partition. Also test resampled data and nearby values of $k$. Report instability rather than hiding it; uncertainty about the grouping is an important result.

How do I assign new observations after training?

Apply exactly the same preprocessing used during training, including column order, missing-value handling, transformations, scaling, and feature selection. Then calculate the distance from the transformed observation to each fitted centroid and choose the nearest. Do not refit the scaler on a single new batch, because that changes the coordinate system. Monitor distances and feature distributions over time. A new observation can always be assigned, even if it is far from every training cluster, so assignment alone does not show that it resembles the training data. A distance threshold or separate drift detector may be needed.

Why can a cluster centroid fail to look like a typical observation?

A centroid is a coordinate-wise average, not necessarily an observed case. For continuous measurements this synthetic prototype may still be meaningful, but correlated or constrained features can make it unrealistic. The average of several valid combinations may violate a physical rule, and the average of sparse documents may look unlike any actual document. Examine medoids—the most central observed cases—alongside centroids when communicating results. Also inspect feature distributions, because averages can hide substructure, skew, and multimodality inside a cluster.

How many observations are needed for k-means?

There is no universal minimum. The answer depends on dimension, noise, separation, desired $k$, and whether the sample represents future data. At a bare minimum, $n$ must be at least $k$, but that condition is practically inadequate. Each cluster needs enough observations to estimate a stable $d$-dimensional mean, and rare groups require sufficient coverage. Assess learning curves or resampling stability: fit on increasing sample sizes and measure whether centroids and memberships stabilize. In high dimensions, many more observations may be needed, especially when numerous features are irrelevant.

Can k-means detect anomalies?

It can provide an anomaly signal because observations far from their assigned centroid have high reconstruction error. This can work when normal data forms compact centroid-like groups. However, k-means is not a dedicated anomaly detector. Outliers influence the centroids during fitting, and a collection of anomalies may form its own cluster and appear internally normal. Distance thresholds also vary by cluster density. Robust covariance methods, isolation forests, local outlier factor, or density-based approaches may be more suitable. If centroid distance is used, validate it against realistic anomalies and monitor false positives.

Is a high silhouette score proof that the clusters are meaningful?

No. A high silhouette score says the partition has strong cohesion and separation under the chosen distance and features. It does not establish causal, semantic, ethical, or operational meaning. A dataset separated by data source, measurement device, or missing-value pattern can achieve an excellent score while revealing only an artifact. A sensitive attribute or proxy might also create sharply separated groups that should not guide decisions. Investigate what drives separation, reproduce the result on independently collected data, and ask domain experts whether the structure corresponds to a useful and legitimate distinction.

Summary

K-means partitions numeric observations by minimizing within-cluster squared Euclidean distance. Its alternating assignment and centroid-update steps are efficient and easy to inspect, but they converge to local optima, making k-means++ initialization and multiple starts important. The objective favors compact, roughly spherical groups and is sensitive to outliers, unequal densities, irrelevant dimensions, and feature units.

Choosing $k$ requires judgment. Elbow plots, silhouette analysis, stability, domain knowledge, cluster sizes, and downstream usefulness should be considered together. Feature scaling is essential because it determines the geometry on which every assignment depends. For large datasets, mini-batch k-means offers a practical approximation; for non-spherical, probabilistic, hierarchical, or noise-heavy structures, alternatives such as DBSCAN, Gaussian mixture models, and hierarchical clustering may be better aligned.

Most importantly, k-means always produces a partition, but a partition is not proof of natural categories. Responsible clustering treats the output as a hypothesis, examines its stability and assumptions, tests its usefulness, and resists turning convenient mathematical groups into unsupported claims about the world.