Deploying a Model: Release a Service You Can Roll Back
Working on your laptop is the beginning of deployment
The rent API passes its tests in your notebook. Now a teammate needs to run the same service tomorrow, on a different machine, with a known artifact and a repeatable start command.
Deployment means placing the service in its intended runtime and making it available to clients. A release is a specific combination of application code, dependencies, configuration, and model artifact. Those pieces must agree.
1. Package
Code + artifact + environment + schema
An image or archive is a release package, not yet a running service.
2. Start and verify
Load artifact → smoke prediction → ready
Alive but loading does not mean ready for traffic.
3. Canary
Small traffic share reaches the candidate
Compare errors, latency, and later outcome quality by version.
4. Promote or return
Pass limits → expand. Fail limits → stable release.
Rollback needs the previous code, artifact, dependencies, and compatible schema.
Keep the stable release available throughout the candidate rollout.
Choose how predictions arrive
Not every model needs a live HTTP server. A nightly job can estimate rents for tomorrow’s listings and write a table. A live API predicts when a user changes the area field. These have different timing and availability requirements.
Key Points
- Batch: process a collection on a schedule, with job completion and output-quality checks.
- Online: answer each request, with latency and uptime requirements.
- Choose the simplest mode that meets the decision timing.
- A model that can score offline does not automatically meet an interactive response budget.
Package a real artifact and its description
This standalone example prepares the model portion of a release archive. It contains the fitted pipeline, a manifest with a checksum, and direct dependency versions. The API source and server dependencies must also be included when packaging the whole online service.
1from pathlib import Path
2from tempfile import TemporaryDirectory
3from zipfile import ZipFile
4import hashlib
5import json
6import platform
7import joblib
8import numpy as np
9import sklearn
10from sklearn.pipeline import Pipeline
11from sklearn.preprocessing import StandardScaler
12from sklearn.linear_model import LinearRegression
13
14with TemporaryDirectory() as directory:
15 root = Path(directory)
16 model_path = root / "rent-v1.joblib"
17 pipeline = Pipeline([("scale", StandardScaler()), ("model", LinearRegression())])
18 pipeline.fit([[500], [700], [900], [1100]], [18000, 24000, 30000, 36000])
19 joblib.dump(pipeline, model_path)
20 manifest = {
21 "model_version": "rent-v1",
22 "schema_version": "rent-input-v1",
23 "python_version": platform.python_version(),
24 "sha256": hashlib.sha256(model_path.read_bytes()).hexdigest(),
25 }
26 (root / "manifest.json").write_text(json.dumps(manifest), encoding="utf-8")
27 requirements = (
28 f"scikit-learn=={sklearn.__version__}\n"
29 f"numpy=={np.__version__}\n"
30 f"joblib=={joblib.__version__}\n"
31 )
32 (root / "requirements.txt").write_text(requirements, encoding="utf-8")
33 archive_path = root / "model-release.zip"
34 with ZipFile(archive_path, "w") as archive:
35 for name in ["rent-v1.joblib", "manifest.json", "requirements.txt"]:
36 archive.write(root / name, arcname=name)
37 with ZipFile(archive_path) as archive:
38 print("Release files:", sorted(archive.namelist()))
39 stored = json.loads(archive.read("manifest.json"))
40 digest = hashlib.sha256(archive.read("rent-v1.joblib")).hexdigest()
41 print("Artifact checksum matches:", digest == stored["sha256"])
42 print("Model version:", stored["model_version"])Release files: ['manifest.json', 'rent-v1.joblib', 'requirements.txt'] Artifact checksum matches: True Model version: rent-v1
This archive is not a running deployment. It demonstrates how to collect the model, identity, and environment evidence. The three direct dependency pins are not a complete environment lock; record transitive dependencies or use a tested lockfile for the final release.
Give the runtime a repeatable start
A container image packages the service filesystem and its start command. A Dockerfile for a Python API normally selects a Python base image, installs the tested dependency lock, copies the service and artifact, then starts the application server.
Key Points
- Use a tested Python version and reproducible base-image reference.
- Include FastAPI, Uvicorn, and the model environment in the dependency lock.
- Copy application source and the selected versioned artifact into the image or fetch the artifact through a controlled startup path.
- For app.py exposing app, a start command can be uvicorn app:app --host 0.0.0.0 --port 8000.
- Document the port, artifact path, configuration, and expected resource needs.
An image is still not a deployed service. A machine or managed hosting platform must run it, connect traffic, and supply configuration. Choose one appropriate hosting environment rather than adding orchestration complexity before it is needed.
Liveness and readiness answer different questions
Liveness asks whether the process is functioning enough to remain running. Readiness asks whether this instance can accept prediction traffic now. An instance loading its model may be alive but not ready.
A readiness check can include artifact loading and a known-input smoke prediction. A liveness check should not constantly refit or run a costly model evaluation. A health endpoint alone does not prove prediction accuracy.
1instances = [
2 {"name": "old-release", "alive": True, "model_loaded": True, "smoke_passed": True},
3 {"name": "candidate-loading", "alive": True, "model_loaded": False, "smoke_passed": False},
4 {"name": "candidate-ready", "alive": True, "model_loaded": True, "smoke_passed": True},
5]
6
7for instance in instances:
8 ready = instance["alive"] and instance["model_loaded"] and instance["smoke_passed"]
9 print(f"{instance['name']}: alive={instance['alive']}, ready={ready}")old-release: alive=True, ready=True candidate-loading: alive=True, ready=False candidate-ready: alive=True, ready=True
This table is a simulation of health decisions, not calls to a cloud platform. It makes the distinction visible before you configure the actual hosting checks.
Measure a response budget under representative load
If the listing page can wait 300 milliseconds, your budget includes validation, preprocessing, prediction, network time, and any queueing. A fast notebook predict call measures only part of that path.
Key Points
- Measure tail latency, not only the mean.
- Exercise realistic request sizes and concurrency.
- Measure memory per worker, including its loaded artifact.
- Limit batch size and choose timeouts that match the client workflow.
- Decide how the client falls back when the service is unavailable.
Start with limited traffic
A canary release routes a small share of traffic to the candidate while the stable release serves the rest. Compare errors, latency, versioned outputs, and later outcome quality before increasing the share.
Shadow testing sends a copy of requests to a candidate without using its output for the user-facing decision. It can compare behaviour, but it still needs resource capacity and cannot by itself show the effect of acting on the new prediction.
Make rollback a prepared action
1stable_version = "rent-v1"
2candidate_version = "rent-v2"
3candidate_metrics = {"error_rate": 0.08, "p95_ms": 450}
4limits = {"error_rate": 0.02, "p95_ms": 300}
5
6acceptable = all(candidate_metrics[name] <= limits[name] for name in limits)
7active_version = candidate_version if acceptable else stable_version
8
9print("Candidate passes release limits:", acceptable)
10print("Simulated active version:", active_version)Candidate passes release limits: False Simulated active version: rent-v1
These example limits are chosen for the teaching scenario. The cell changes a local variable only. A real rollback routes clients back to the preserved release and checks that its schema and downstream compatibility still work.
Key Points
- Keep the previous code, artifact, and environment, not only the old model file.
- Record which instance served which version during a partial rollout.
- Define who can stop a release and which signals trigger that decision.
- Test the rollback path before you need it.
Where people get confused
Key Points
- An archive or container image is a package, not a running service.
- Readiness is not the same as liveness, and neither guarantees model quality.
- A candidate may be healthy operationally but inaccurate for an important group.
- Replacing a model file is not a complete rollback if code or input schema changed.
Quick check
What belongs in a model-service release?
The application code, tested dependencies, configuration contract, versioned fitted artifact, and feature schema must be compatible and identifiable.
How does readiness differ from liveness?
Liveness checks whether the process should remain running. Readiness checks whether it can accept prediction traffic, including having the model available.
What does a canary release do?
It sends a limited traffic share to a candidate so operational and outcome evidence can be checked before wider use.
Why keep the whole previous release?
The previous model may need its own code, dependencies, and schema. Keeping only its file may not provide a functioning rollback.
Release with a way back
Key Points
- Choose batch or online serving from the decision timing.
- Package a versioned service and reproduce its environment.
- Check readiness and representative load before traffic.
- Use limited rollout and a tested rollback path.
- Next, monitoring shows what changes after launch.
Theia
Can you explain this concept?
Practise Deploying a Model: Release a Service You Can Roll Back with realistic interview questions. Choose spoken or typed answers, get feedback on your reasoning, then return to the material that needs work.
Sign in to practise this conceptThe session stays focused on this lesson. Your selected interview level sets the bar.