Features, Targets, and What a Model Really Is
Three words we keep using
Over the last five lessons we have said "features", "label" and "model" again and again. You probably have a rough feel for them by now. In this lesson we pin them down properly, because every chapter after this one uses them on almost every line.
We will also clear up something that confuses a lot of beginners: the same idea often has four or five different names, depending on who is writing. By the end, none of those names will throw you.
It all starts with a table
Most data in classic machine learning looks like a spreadsheet. Here is a small one about flats for rent in Pune:
1size_sqft bedrooms floor rent
2 550 1 2 12000
3 700 1 5 15500
4 800 2 1 17000
5 950 2 8 21500
6 1100 2 3 23000Key Points
- Each row is one example: one flat. You will also see a row called a sample, a record, an observation or a data point.
- Each column is one piece of information about every example: its size, its bedrooms, its floor, its rent.
Now we split the columns into two jobs. Which job each column gets depends on one thing: what do we want to predict?
Features: what the model looks at
Features are the columns the model uses to make its guess. Here, if we want to predict rent, the features are size_sqft, bedrooms and floor. They are the clues.
You will see features called by many names. They all mean the same thing:
Key Points
- features
- inputs
- X (a capital X, in code and maths)
- attributes, predictors, variables
- independent variables (mostly in statistics books)
Features come in different kinds. Size is a number. "Has parking" is yes or no. The area, like Kothrud or Baner, is a category. Models only understand numbers, so categories have to be turned into numbers first. Chapter 3 shows exactly how.
The target: what the model predicts
The target is the one column we want the model to predict. Here, it is rent. It is the answer from lesson 1.3. Its other names:
Key Points
- target
- label (used a lot when the answer is a category, like spam or not spam)
- y (a lowercase y, in code and maths)
- output, outcome, response
- dependent variable (statistics books again)
Tip:Target and label mean almost exactly the same thing. Do not waste energy on the difference. If someone says either word, they mean "the column we are trying to predict".
The question decides which is which
Here is something that surprises people. A column is not "naturally" a feature or a target. It depends entirely on your question.
Take the same flats table. If a tenant asks "what rent should I expect for this flat?", then rent is the target and the other columns are features.
But if a builder asks "for ₹20,000 a month, how big a flat can people afford?", then size is what we predict, and rent becomes one of the features. Same table, same columns, different jobs, because the question changed. This is exactly why step 1 of the lifecycle in lesson 1.5 is defining the question.
What makes a good feature
You cannot just throw every column you have at a model. A good feature passes three simple checks.
First, it should be related to the target. Size clearly affects rent. The owner's phone number does not. Useless columns add noise and can make a model worse, not better.
Second, and this is the one people miss: you must know it at the moment you make the prediction. Say you want to predict rent before a flat is listed. "Number of people who enquired about it" might be strongly related to rent, but you only know that after it is listed. A model trained with it looks brilliant in testing and fails in real life. This mistake is called data leakage, and Chapter 5 has a whole lesson on it.
Third, it should not be something that only identifies the row, like a flat ID number or the row number. Those carry no real information about rent, but a model can still find accidental patterns in them.
So what is a model, really?
A model is not a brain, and it is not magic. A model is a rule that turns features into a prediction, where some of the numbers in the rule were learned from data.
You already built one. In lesson 1.2, the machine learned rent = 20.3 × size. That line is the whole model. The rule is "multiply size by something". The 20.3 is the part that was learned. Those learned numbers are the parameters.
With more features, the rule just gets longer. A model for the flats table might look like this:
1rent = 16 × size + 2,069 × bedrooms + 316 × floor + 391Read it like plain English. Every extra square foot adds about ₹16. Every extra bedroom adds about ₹2,069. Every floor higher adds about ₹316. Nobody typed those numbers in. Training found them. (You will see these exact numbers come out of real code in a minute.)
Different models, different shapes of rule
Not every model is a formula like that. Different kinds of models are different shapes of rule:
Key Points
- Linear regression: a formula like the one above. (Chapter 4)
- Decision tree: a flowchart of yes/no questions. "Is it bigger than 800 sq ft? If yes, does it have 2+ bedrooms? If yes, predict ₹22,000." (Chapter 6)
- K-Means from lesson 1.3: a list of group centres, like "the daily shoppers are around 25 visits and ₹150 a bill". (Chapter 7)
- A neural network: a huge formula with millions of learned numbers arranged in layers. (The later Deep Learning subject)
Whatever the shape, the idea is identical. There is a fixed structure, and a set of numbers inside it that were learned from examples.
Algorithm vs model
People often use these two words as if they mean the same thing. They do not, and knowing the difference makes a lot of explanations easier to follow.
Think of a recipe and a dish. The recipe for dal is the same in every kitchen. But the dal your mother makes and the dal a hostel mess makes come out different, because the ingredients are different.
Key Points
- The algorithm is the recipe: the method for learning. "Linear regression" is an algorithm. It says how to find the best numbers for the formula.
- The data is the ingredients.
- The model is the dish: what you get after running the algorithm on your data. "rent = 16 × size + 2,069 × bedrooms + ..." is a model.
So the same algorithm, trained on flats in Pune and flats in Mumbai, gives you two different models. Mumbai's would have a much bigger number in front of size.
In code
Here is everything from this lesson in one place. It uses pandas, a library for working with tables, and scikit-learn. Both come ready in Google Colab, and Chapter 2 introduces them properly.
1import pandas as pd
2from sklearn.linear_model import LinearRegression
3
4flats = pd.DataFrame({
5 "size_sqft": [550, 700, 800, 950, 1100, 1200, 650, 1000],
6 "bedrooms": [1, 1, 2, 2, 2, 3, 1, 3],
7 "floor": [2, 5, 1, 8, 3, 6, 0, 4],
8 "rent": [12000, 15500, 17000, 21500, 23000, 27500, 12500, 24000],
9})
10
11X = flats[["size_sqft", "bedrooms", "floor"]] # features
12y = flats["rent"] # target
13
14model = LinearRegression() # the algorithm (the recipe)
15model.fit(X, y) # training: the recipe meets the data
16
17print("Learned numbers:")
18for name, number in zip(X.columns, model.coef_):
19 print(f" {name:>9}: {number:,.0f}")
20print(f" starting point: {model.intercept_:,.0f}")
21
22new_flat = pd.DataFrame({"size_sqft": [900], "bedrooms": [2], "floor": [3]})
23print("\nPredicted rent:", round(model.predict(new_flat)[0]))Learned numbers:
size_sqft: 16
bedrooms: 2,069
floor: 316
starting point: 391
Predicted rent: 19679Look at the X and y lines. We literally pick which columns are features and which one is the target. Capital X because it holds several columns. Lowercase y because it is just one. That naming habit comes from maths, and you will see it in almost every piece of ML code.
Now look at the output. After all that, the trained model is just four numbers: 16, 2,069, 316 and 391. That is what "learning" produced. To predict the new flat, it simply works out 16 × 900 + 2,069 × 2 + 316 × 3 + 391, using the exact unrounded numbers, which gives ₹19,679.
Tip:The printed 16 is rounded. The real learned number is 15.78, which is why doing the sum with 16 gives a slightly higher answer than the code does. Models keep the full numbers.
Where people get confused
Key Points
- "Label and target are different things." For all practical purposes, they are the same: the column you want to predict.
- "More features always make a better model." Unrelated features add noise, and features you will not have at prediction time cause leakage. Fewer, better features often win.
- "The algorithm and the model are the same thing." The algorithm is the recipe. The model is what you get after training it on your data.
- "The model memorises the training data." Most models only keep their learned numbers, not the data. One well-known exception, K-Nearest Neighbours, really does keep all the data, and you will meet it in Chapter 6.
Quick check
You want to predict whether a student passes, using attendance, study hours and previous marks. What are the features and what is the target? Is it regression or classification?
Features: attendance, study hours and previous marks. Target: pass or fail. Since the target is a category, it is classification.
You are predicting delivery time when an order is placed. Can "the time the rider actually reached the customer" be a feature?
No. You only know it after the delivery is done, so you cannot have it at prediction time. Using it would be data leakage.
Two companies both use linear regression to predict house prices, one in Delhi and one in Chennai. Do they have the same model?
Same algorithm, different models. They trained the same recipe on different data, so the learned numbers inside each model are different.
Recap
Key Points
- In a data table, each row is one example and each column is one piece of information.
- Features (X) are the columns the model looks at. The target or label (y) is the column it predicts.
- The question you ask decides which column is the target.
- Good features are related to the target, known at prediction time, and not just an ID.
- A model is a rule with learned numbers inside it. The algorithm is the recipe that finds those numbers.
That is the end of Chapter 1. You now know what machine learning is, how a machine learns, the two main kinds of learning, where ML shows up, how a real project runs, and the words everything is built on.
Next up, Chapter 2: your toolkit. We start with a question you might already be wondering about. Why does almost everyone in machine learning use Python?
Theia
Can you explain this concept?
Practise Features, Targets, and What a Model Really Is with realistic interview questions. Choose spoken or typed answers, get feedback on your reasoning, then return to the material that needs work.
Sign in to practise this conceptThe session stays focused on this lesson. Your selected interview level sets the bar.