Multiple Linear Regression: Learning from Several Clues
One clue is rarely the whole story
In the first lesson, we used flat size to predict rent. That was a useful starting point, but two flats with the same area can have different rents. One may have an extra bedroom, a lift, parking, or a shorter trip to the metro.
A model that sees only area is forced to treat those differences as unexplained noise. A model can learn a better signal when we give it several useful measurements for each flat.
Key Points
- One input feature: use area to predict rent.
- Several input features: use area, bedrooms, and distance to transit together.
- One target: the number we want to predict, such as monthly rent.
- More columns can add useful information, but they can also add noise, leakage, or duplicate clues.
Tip:Multiple regression means several input features and one numerical target. It does not mean several targets, and it does not automatically mean a complicated model.
The line gets one term per feature
With one feature, the model was ŷ = wx + b. With three features, the same idea becomes ŷ = w₁x₁ + w₂x₂ + w₃x₃ + b. The model multiplies each input by its own weight, adds the contributions, and then adds the intercept.
For a flat, x₁ might be area in square feet, x₂ might be the number of bedrooms, and x₃ might be kilometres from a metro station. The weights describe how strongly the model uses each feature inside this formula.
Key Points
- ŷ is the prediction.
- b is the intercept, or the starting prediction when all inputs are zero.
- w₁, w₂, and w₃ are learned weights.
- x₁, x₂, and x₃ are the feature values for one row.
- The model is still linear in its learned weights because every weight is multiplied by a feature and then added.
For two features, the graph is a flat surface rather than a line. With more features, we cannot draw the whole surface easily, but the calculation is still a weighted sum. Gradient Descent can update every weight and the intercept using the same cost idea as before.
Read one prediction from the formula
Suppose a simple rent model has b = 8,000, w₁ = 120 for area measured in hundreds of square feet, w₂ = 3,000 for bedrooms, and w₃ = −1,000 for each kilometre from the metro. A 7-hundred-square-foot flat with 2 bedrooms and 3 kilometres of distance gets one prediction.
Key Points
- Area contribution: 120 × 7 = 840.
- Bedroom contribution: 3,000 × 2 = 6,000.
- Distance contribution: −1,000 × 3 = −3,000.
- Prediction: 8,000 + 840 + 6,000 − 3,000 = ₹11,840.
1def predict_rent(area_hundreds, bedrooms, distance_km):
2 intercept = 8000
3 area_weight = 120
4 bedroom_weight = 3000
5 distance_weight = -1000
6
7 return (
8 intercept
9 + area_weight * area_hundreds
10 + bedroom_weight * bedrooms
11 + distance_weight * distance_km
12 )
13
14rent = predict_rent(area_hundreds=7, bedrooms=2, distance_km=3)
15print(f"Predicted rent: ₹{rent:,.0f}")Predicted rent: ₹11,840
The individual contributions make the prediction easier to inspect. In a real model, the learned numbers would come from training data rather than being chosen by hand.
What a coefficient says, and what it does not say
A coefficient has an interpretation tied to its unit. If the area weight is ₹120 per hundred square feet, increasing that feature by one unit changes the prediction by ₹120 while the other features stay fixed in the formula.
That phrase, while the other features stay fixed, matters. A coefficient is a model relationship, not automatically a causal claim. Larger flats may also have better views, newer fittings, or a better location. The model may use area as a clue for several things mixed together.
Key Points
- Read a weight with its feature unit. A weight for bedrooms is not comparable to a weight for square feet without considering units.
- A positive weight raises the prediction as that feature increases, all else equal in the model.
- A negative weight lowers the prediction as that feature increases, all else equal in the model.
- A large weight is not automatically proof that a feature is the most important or causal.
- Changing the unit of a feature changes its coefficient without changing the underlying predictions.
The feature table is the real input
A single row is one example. Its columns are the features the model can use. In Python and in most ML libraries, the feature data is arranged as a table called X, while the target values are kept in a separate sequence called y.
Key Points
- X has shape (number of rows, number of features).
- y has one target value per row.
- The order of rows in X and y must match. Row 2 of X must describe the target in position 2 of y.
- Column names belong in your data documentation, even if the model receives only numbers.
1rows = [
2 {"area_hundreds": 6, "bedrooms": 1, "distance_km": 2},
3 {"area_hundreds": 8, "bedrooms": 2, "distance_km": 5},
4 {"area_hundreds": 10, "bedrooms": 3, "distance_km": 1},
5]
6targets = [18000, 26000, 42000]
7
8feature_names = ["area_hundreds", "bedrooms", "distance_km"]
9X = [[row[name] for name in feature_names] for row in rows]
10
11print("Rows:", len(X))
12print("Features per row:", len(X[0]))
13print("First feature row:", X[0])
14print("Targets:", targets)Rows: 3 Features per row: 3 First feature row: [6, 1, 2] Targets: [18000, 26000, 42000]
This small table is already enough to explain the model interface: fit receives X and y, then predict receives new X rows with the same feature columns in the same order.
Training changes every weight together
The training loop is the same idea as the one-feature version. Make predictions, calculate errors, calculate one gradient for each weight, calculate the intercept gradient, and update all parameters. The extra work is keeping one gradient per feature.
For a dataset with n rows, the weight gradient for feature j uses the prediction errors multiplied by that feature column. The intercept gradient uses the errors alone. The learning rate controls the size of every update.
Key Points
- Start with one weight for each feature and one intercept.
- Calculate predictions using the current weights.
- Use the errors to calculate one direction for every parameter.
- Subtract learning rate times each gradient.
- Repeat for several epochs while watching the cost.
1X = [
2 [1.0, 0.0],
3 [2.0, 0.0],
4 [3.0, 1.0],
5 [4.0, 1.0],
6]
7y = [3.0, 5.0, 8.0, 10.0]
8
9weights = [0.0, 0.0]
10intercept = 0.0
11learning_rate = 0.05
12
13for epoch in range(3000):
14 predictions = [
15 intercept + sum(weight * value for weight, value in zip(weights, row))
16 for row in X
17 ]
18 errors = [prediction - target for prediction, target in zip(predictions, y)]
19
20 gradients = [
21 2 * sum(error * row[j] for error, row in zip(errors, X)) / len(X)
22 for j in range(len(weights))
23 ]
24 intercept_gradient = 2 * sum(errors) / len(X)
25
26 weights = [
27 weight - learning_rate * gradient
28 for weight, gradient in zip(weights, gradients)
29 ]
30 intercept -= learning_rate * intercept_gradient
31
32final_cost = sum(error ** 2 for error in errors) / len(errors)
33print("Weights:", [round(weight, 2) for weight in weights])
34print("Intercept:", round(intercept, 2))
35print("Final MSE:", round(final_cost, 2))Weights: [2.0, 1.0] Intercept: 1.0 Final MSE: 0.0
The weights learned the pattern y = 2x₁ + x₂ + 1 for this tiny dataset. Real data is noisier, so the final MSE usually will not be zero. A perfect training fit can also be a warning when the dataset is small or the features are overly flexible.
Feature units can make training awkward
Suppose one feature is annual income in rupees and another is age in years. A one-unit change in income is tiny compared with a one-unit change in age. Their gradients can have very different sizes, so one shared learning rate may make progress uneven.
Feature scaling puts numeric columns onto more comparable ranges. Standardisation is one common approach: subtract the feature mean and divide by its standard deviation. Scaling changes the coefficient units, but it does not change the underlying information in the column.
Key Points
- Scaling can make Gradient Descent move more smoothly.
- The intercept and coefficients must be interpreted in the units used during training.
- Fit scaling values on the training data, then reuse those same values for validation and future predictions.
- Scaling does not repair a feature that leaks the answer or has no useful relationship with the target.
Feature scaling appeared earlier as a data preparation idea. Here its connection to multiple regression is practical: more features create more directions for the optimiser to navigate, and very different scales can make that walk less comfortable.
Two features can carry the same clue
Area in square feet and the number of rooms are often related. If two columns carry nearly the same information, the model may have difficulty deciding how much weight to assign to each one. This situation is called multicollinearity.
The predictions can still be reasonable while the individual coefficients become unstable. A small change in the training data may move weight from one correlated feature to the other without changing the total prediction much.
Key Points
- Correlated features are not automatically forbidden.
- They make individual coefficient stories harder to trust.
- For prediction, the combined signal may be more important than one coefficient.
- For explanation, inspect correlations, domain meaning, and stability across samples.
- Do not remove a feature only because its coefficient is small. A feature can matter in combination with others.
Tip:If a stakeholder asks which feature caused the rent increase, a multiple-regression coefficient alone is not enough evidence. Explain the model relationship and the limits of the data separately.
Additive models have a boundary
The formula ŷ = w₁x₁ + w₂x₂ + b adds each feature contribution independently. It can say that an extra bedroom adds a fixed amount while the other inputs stay fixed. It cannot, by itself, express that an extra bedroom matters more in a large flat than in a small flat.
That kind of combined effect is called an interaction. You can represent it by creating an additional feature such as area × bedrooms, but that is a feature-engineering decision. It should be guided by the problem, checked carefully, and evaluated on data the model did not train on.
Key Points
- An additive model gives each feature its own contribution.
- An interaction allows one feature effect to depend on another feature.
- More terms can represent richer patterns, but they can also make overfitting easier.
- A feature transformation should have a reason and should be validated, not added just because it improves one training score.
Fit, predict, and inspect the whole pipeline
A multiple-regression workflow has the same shape as the first model, with a wider feature table. Define the target and the feature columns, split data before fitting when you need an honest evaluation, train on the training rows, predict for held-out rows, and calculate a metric that matches the decision.
Key Points
- Choose features using information that would exist at prediction time.
- Keep feature names and order consistent between training and prediction.
- Fit preprocessing only on training data.
- Compare predictions with actual targets on data the model did not use to learn weights.
- Inspect unusually large residuals and surprising coefficients instead of trusting one score blindly.
The model is now more informed because it sees several clues, but it is not automatically wiser. Data quality, the target definition, leakage, missing signals, and the evaluation design still control whether its predictions deserve trust.
Where people get confused
Key Points
- "Multiple" means multiple input features, not multiple target columns.
- "Linear" refers to how the model combines its weights. A model can use transformed columns and still be linear in its coefficients.
- A coefficient is not automatically a causal effect. Holding other columns fixed in a formula is not the same as performing a controlled experiment.
- A large coefficient is not automatically an important feature because units and correlations affect coefficient size.
- A lower training MSE does not prove better performance on new data.
- Adding every available column can introduce noise, leakage, duplicate clues, and maintenance problems.
Quick check
How does multiple linear regression extend the one-feature line?
It gives each input feature its own weight, multiplies each feature by that weight, adds all contributions, and then adds the intercept to produce one numerical prediction.
What does a coefficient mean when the other features are held fixed in the model?
It describes how much the prediction changes for a one-unit increase in that feature, using the feature units and the model relationship. It is not automatically a causal effect.
Why can correlated features make coefficient interpretation difficult?
When features carry similar information, the model can distribute weight between them in different ways while producing similar predictions. Individual weights may therefore be unstable.
What must stay consistent when predicting with a trained multiple-regression model?
The new rows must use the same feature meanings, units, preprocessing, and column order that the model saw during training.
Recap
Key Points
- Multiple linear regression predicts one numerical target from several input features.
- The model adds one weighted contribution per feature and an intercept.
- Gradient Descent can update all weights together using the same MSE objective.
- Feature units, correlated columns, interactions, leakage, and evaluation data affect how much trust to place in the result.
- A coefficient explains a model relationship in its units. It does not by itself prove importance or causation.
Several useful clues can make a model more capable than a size-only line. The next lesson asks what happens when the relationship itself bends instead of staying flat: Polynomial Regression.
Theia
Can you explain this concept?
Practise Multiple Linear Regression: Learning from Several Clues with realistic interview questions. Choose spoken or typed answers, get feedback on your reasoning, then return to the material that needs work.
Sign in to practise this conceptThe session stays focused on this lesson. Your selected interview level sets the bar.