K-Means: Assign Points, Move Centres, Repeat
Give each group a meeting point
You have customer spending records but no segment names. A simple approach puts a centre near each possible group, assigns customers to their nearest centre, then moves each centre to the average of its assigned customers.
Repeat those two steps until assignments or centres change very little. That is K-Means. K is the number of groups you ask for. A centroid is the mean feature position of a group, not necessarily a real row.
Walk through six points
Use one numeric feature to keep the arithmetic visible: 1, 2, 3, 9, 10, 11. Start K = 2 centres at 1 and 9. The first three points are nearer 1; the last three are nearer 9. Their new means are 2 and 10.
1. Assign to the nearest centre
Starting centres: 1 and 9. Each point joins its nearer centre.
2. Move centres to the means
Group averages: 2 and 10. Repeat assignment with these new centres.
1points = [1, 2, 3, 9, 10, 11]
2centres = [1, 9]
3groups = [[], []]
4for point in points:
5 nearest = min(range(len(centres)), key=lambda j: abs(point - centres[j]))
6 groups[nearest].append(point)
7new_centres = [sum(group) / len(group) for group in groups]
8print("Groups:", groups)
9print("New centres:", new_centres)Groups: [[1, 2, 3], [9, 10, 11]] New centres: [2.0, 10.0]
Assigning again to centres 2 and 10 leaves the groups unchanged. This small example has settled. In more dimensions, the centre takes the mean separately for every feature.
What the algorithm minimises
For the first group, squared distances to centre 2 are 1, 0, and 1. The second group contributes the same amount. The total is 4. This sum is called inertia, or within-cluster sum of squares.
Written compactly, inertia = Σ‖xᵢ − μc(i)‖². xᵢ is row i, c(i) is its assigned cluster, μ is that cluster’s centre, ‖ ‖ means Euclidean distance, and Σ means add over all rows. You do not need to memorise the notation; it is the total squared distance to assigned centres.
1groups = [[1, 2, 3], [9, 10, 11]]
2centres = [2, 10]
3inertia = sum((point - centre) ** 2 for group, centre in zip(groups, centres) for point in group)
4print("Inertia:", inertia)Inertia: 4
Fit an actual K-Means model
scikit-learn performs the repeated updates. The data has one column, so each input is still a feature row such as [1]. We specify starting centres here to make the output easy to compare with the hand calculation.
1import numpy as np
2from sklearn.cluster import KMeans
3
4X = np.array([[1], [2], [3], [9], [10], [11]], dtype=float)
5model = KMeans(n_clusters=2, init=np.array([[1.0], [9.0]]), n_init=1, random_state=7)
6labels = model.fit_predict(X)
7print("Labels:", labels.tolist())
8print("Centres:", model.cluster_centers_.ravel().tolist())
9print(f"Inertia: {model.inertia_:.1f}")
10print("New point 8:", model.predict([[8]]).tolist())Labels: [0, 0, 0, 1, 1, 1] Centres: [2.0, 10.0] Inertia: 4.0 New point 8: [1]
fit_predict fits centres and returns assignments for the fitting rows. predict assigns a new row to an existing centre without moving the centres. Cluster numbers can swap across runs without changing the grouping.
Starting positions can change the answer
K-Means can settle at a local solution rather than the best possible grouping. Poor starting centres can put several centres in one dense region and miss another region.
Key Points
- K-Means++ spreads initial centres using a distance-based sampling rule.
- Several initialisations can reduce dependence on one unlucky start.
- n_init controls how many starts are tried in scikit-learn.
- Record random_state for repeatable experiments, but also check other seeds for stability.
Choose K from evidence, not the lowest inertia
Inertia usually falls as K increases. Giving every point its own centre can make it zero, which is rarely a useful segmentation. The elbow method looks for a point where extra clusters stop reducing inertia substantially; it is a heuristic, not an objective answer key.
A silhouette score compares a point’s distance to its own group with its distance to the nearest other group. Higher scores generally indicate compact, separated groups under the chosen distance. The metric is undefined for a single cluster and can prefer the wrong shape for your decision.
Key Points
- Compare a small sensible range of K values.
- Inspect group sizes, samples, and stability.
- Check whether the groups support distinct actions.
- Do not select K from one structure score alone.
Scaling defines the geometry
Spend in rupees can dominate order count because its numeric differences are much larger. Standardising the features can make them more comparable, but that changes the meaning of distance. Choose scaling according to the similarity you want.
If you use clusters later inside a supervised workflow, fit the scaler and centres only on the allowed training portion. A customer ID is not a useful coordinate merely because it is a number.
Some shapes do not fit the centre idea
K-Means works best when Euclidean distance to a mean describes roughly compact groups. It can split curved groups, struggle with very different densities, and move a centre toward an extreme outlier.
Key Points
- Every row receives a cluster, including unusual rows.
- A mean centre can land in empty space.
- Categories need thoughtful representation; arbitrary integer labels create fake distances.
- For irregular shapes or explicit noise handling, compare density-based methods later in this chapter.
Where people get confused
Key Points
- K in K-Means means groups; K in KNN means neighbours. KNN uses labels for supervised prediction.
- A centroid is an average, not necessarily an observed customer.
- Lower inertia alone does not establish a better segmentation.
- A fixed seed makes a run repeatable, not guaranteed correct.
Quick check
What are the two repeated K-Means steps?
Assign each row to its nearest centre, then replace each centre with the mean of its assigned rows.
Why not choose K only by minimum inertia?
Increasing K reduces inertia, potentially to zero with one group per row. The resulting segmentation may be useless or unstable.
Why can a spend feature dominate an order-count feature?
Distance uses numerical differences. Spend has much larger units unless the feature representation or scale is adjusted.
What does predict do for a fitted K-Means model?
It assigns new rows to the existing centres without learning new centres.
What to keep and what comes next
Key Points
- K-Means needs K, a distance representation, and initial centres.
- Its fit minimises squared distances, not business usefulness.
- Inspect stability, shapes, and examples before naming the groups.
- Next, Hierarchical Clustering shows group relationships at several levels.
Theia
Can you explain this concept?
Practise K-Means: Assign Points, Move Centres, Repeat with realistic interview questions. Choose spoken or typed answers, get feedback on your reasoning, then return to the material that needs work.
Sign in to practise this conceptThe session stays focused on this lesson. Your selected interview level sets the bar.