Practical ML with Python and scikit-learn

Begin with the central question

What does a complete, trustworthy ML workflow look like in actual Python?

This question explains why Practical ML with Python and scikit-learn deserves its own lesson. Each definition, calculation, and code example below will answer a specific part of it.

load data → split → pipeline → fit → evaluate → predict → save

Before you continue: three tools for this module

Estimator: a scikit-learn object that learns from data.
Transformer: here, a preprocessing object—not the neural-network architecture.
Pipeline: preprocessing and modeling steps joined into one reproducible object.

You do not need to memorize these yet. Return to this small map whenever a term reappears.

What You Will Understand and Build

Tabular Modeling: Execute a complete supervised learning project from raw customer churn data to final tuned predictions.
Workflow Checklist: Walk through loading, data cleaning, stratified train/test splitting, ColumnTransformer preprocessing, and RandomForest training.
Result Interpretation: Master the grid-search optimization of hyperparameters for F1 score, and interpret predictions using feature importances.

Why the Steps Must Be Assembled in the Correct Order

Every prior module taught individual pieces in isolation. Real ML work requires assembling all of these pieces into one coherent, working pipeline, in the correct order, with the correct discipline (splitting before preprocessing, tuning via cross-validation not the test set, evaluating with metrics that actually matter).

This module exists to show that complete assembly, once, clearly, so you have a genuine reference template.

Cooking the Complete Recipe

Think of this module as watching someone actually cook the full recipe from start to finish, after you’ve already learned each individual technique (chopping, sautéing, seasoning) separately. Seeing it all come together in the right order is what turns “I know the techniques” into “I can actually build the dish.”

4. Core Concept — The Complete Workflow

graph TD
    subgraph "Data Acquisition & Inspection"
        Step1["1. Load Data"] --> Step2["2. Inspect Data (dtypes, shape, nulls)"]
    end
    subgraph "Data Preparation"
        Step2 --> Step3["3. Clean Data (impute nulls, fix entry errors)"]
        Step3 --> Step4["4. Split Data (Stratified train/test split)"]
        Step4 --> Step5["5. Preprocess (Fit scaler/encoder on train ONLY)"]
    end
    subgraph "Model Training & Optimization"
        Step5 --> Step6["6. Train Model (Random Forest Classifier)"]
        Step6 --> Step7["7. Evaluate (Accuracy, Precision, Recall, F1, AUC)"]
        Step7 --> Step8["8. Tune Hyperparameters (GridSearchCV for F1)"]
    end
    subgraph "Inference & Interpretation"
        Step8 --> Step9["9. Predict Churn Probability on New Data"]
        Step9 --> Step10["10. Interpret Results (Tree Feature Importance)"]
    end

We’ll build a realistic churn-prediction model: predicting whether a telecom customer will cancel their subscription, based on account and usage data — a genuinely common, realistic business ML problem.

5. How It Works — Step by Step (The Full Project)

Step 1-2: Load and Inspect Data

# Build a small, inspectable example of Practical ML with Python and scikit-learn.
# Follow the data, learned values, predictions, and evaluation in order.
import pandas as pd
import numpy as np

# Simulated realistic telecom customer data
np.random.seed(42)
n_customers = 1000

data = pd.DataFrame({
    "tenure_months": np.random.randint(1, 72, n_customers),
    "monthly_charges": np.round(np.random.uniform(20, 120, n_customers), 2),
    "contract_type": np.random.choice(["month-to-month", "one-year", "two-year"], n_customers, p=[0.5, 0.3, 0.2]),
    "support_tickets": np.random.poisson(2, n_customers),
    "total_charges": None,  # will compute below
})
data["total_charges"] = np.round(data["tenure_months"] * data["monthly_charges"] * np.random.uniform(0.9, 1.1, n_customers), 2)

# Churn is more likely with: short tenure, month-to-month contracts, many support tickets
churn_probability = (
    0.5
    - 0.01 * data["tenure_months"]
    + 0.05 * data["support_tickets"]
    + np.where(data["contract_type"] == "month-to-month", 0.15, -0.1)
)
data["churned"] = (np.random.rand(n_customers) < churn_probability.clip(0.02, 0.9)).astype(int)

# Introduce some realistic messiness (Module 3)
data.loc[np.random.choice(data.index, 20), "monthly_charges"] = np.nan   # missing values
data.loc[np.random.choice(data.index, 5), "tenure_months"] = -1          # data entry errors

print("Shape:", data.shape)
print("\nFirst few rows:\n", data.head())
print("\nData types:\n", data.dtypes)
print("\nMissing values:\n", data.isna().sum())
print("\nChurn rate:", data["churned"].mean())

Expected Output (approximate):

Shape: (1000, 6)

First few rows:
   tenure_months  monthly_charges contract_type  support_tickets  total_charges  churned
0             52            94.32   two-year                2        4863.31        0
1             15            76.11   month-to-month           3        1178.45        1
...

Missing values:
tenure_months      0
monthly_charges   20
contract_type      0
support_tickets    0
total_charges      0
churned            0

Churn rate: 0.334

🧠 We inspect shape, types, and missing values first — exactly the Module 3 discipline of understanding data quality before doing anything else with it.

Step 3: Clean Data

# Fix the data-entry error: negative tenure is impossible (Module 3)
data.loc[data["tenure_months"] < 0, "tenure_months"] = np.nan

# Impute missing numerical values with the median (robust to outliers, Module 3)
data["tenure_months"] = data["tenure_months"].fillna(data["tenure_months"].median())
data["monthly_charges"] = data["monthly_charges"].fillna(data["monthly_charges"].median())

print("Missing values after cleaning:\n", data.isna().sum().sum(), "total missing values remain")

Expected Output:

Missing values after cleaning:
 0 total missing values remain

Step 4: Split Data (BEFORE preprocessing — Module 4)

# Build a small, inspectable example of Practical ML with Python and scikit-learn.
# Follow the data, learned values, predictions, and evaluation in order.
from sklearn.model_selection import train_test_split

X = data.drop(columns=["churned"])
y = data["churned"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y   # stratified — Module 4
)

print("Training set churn rate:", y_train.mean())
print("Test set churn rate:", y_test.mean())

Expected Output:

Training set churn rate: 0.334
Test set churn rate: 0.335

🧠 Stratified splitting preserves the churn rate in both sets — exactly Module 4’s guidance for imbalanced-ish classification problems.

Step 5: Preprocess (fit on train ONLY — Module 5)

# Build a small, inspectable example of Practical ML with Python and scikit-learn.
# Follow the data, learned values, predictions, and evaluation in order.
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer

numerical_features = ["tenure_months", "monthly_charges", "support_tickets", "total_charges"]
categorical_features = ["contract_type"]

preprocessor = ColumnTransformer(transformers=[
    ("num", StandardScaler(), numerical_features),
    ("cat", OneHotEncoder(drop="first"), categorical_features),
])

# Fit ONLY on training data (Module 4/5 leakage discipline)
X_train_processed = preprocessor.fit_transform(X_train)
X_test_processed = preprocessor.transform(X_test)   # transform, NOT fit_transform

print("Processed training shape:", X_train_processed.shape)

Expected Output:

Processed training shape: (800, 6)

Step 6: Train Model

# Build a small, inspectable example of Practical ML with Python and scikit-learn.
# Follow the data, learned values, predictions, and evaluation in order.
from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42)
model.fit(X_train_processed, y_train)
print("Model trained.")

Step 7: Evaluate (Module 17)

# Build a small, inspectable example of Practical ML with Python and scikit-learn.
# Follow the data, learned values, predictions, and evaluation in order.
from sklearn.metrics import (
    accuracy_score, precision_score, recall_score, f1_score,
    confusion_matrix, roc_auc_score
)

predictions = model.predict(X_test_processed)
probabilities = model.predict_proba(X_test_processed)[:, 1]

print("Accuracy:", accuracy_score(y_test, predictions))
print("Precision:", precision_score(y_test, predictions))
print("Recall:", recall_score(y_test, predictions))
print("F1:", f1_score(y_test, predictions))
print("AUC:", roc_auc_score(y_test, probabilities))
print("\nConfusion Matrix:\n", confusion_matrix(y_test, predictions))

Expected Output (approximate):

Accuracy: 0.815
Precision: 0.719
Recall: 0.612
F1: 0.661
AUC: 0.869

Confusion Matrix:
 [[121  16]
 [ 21  42]]

🧠 We check all of these, not just accuracy (Module 17) — recall here (0.612) tells us the model misses about 39% of customers who actually churn, a genuinely important business number beyond accuracy alone.

Step 8: Tune (Module 15)

# Build a small, inspectable example of Practical ML with Python and scikit-learn.
# Follow the data, learned values, predictions, and evaluation in order.
from sklearn.model_selection import GridSearchCV

param_grid = {
    "n_estimators": [50, 100, 200],
    "max_depth": [3, 5, 7, None],
}

grid_search = GridSearchCV(
    RandomForestClassifier(random_state=42),
    param_grid,
    cv=5,
    scoring="f1",   # optimizing for F1, not accuracy — deliberate choice (Module 17)
)
grid_search.fit(X_train_processed, y_train)

print("Best params:", grid_search.best_params_)
print("Best CV F1:", grid_search.best_score_)

best_model = grid_search.best_estimator_
final_predictions = best_model.predict(X_test_processed)
print("\nFinal tuned test F1:", f1_score(y_test, final_predictions))

Expected Output (approximate):

Best params: {'max_depth': 7, 'n_estimators': 200}
Best CV F1: 0.671

Final tuned test F1: 0.695

Step 9: Make Predictions on New Data

# Build a small, inspectable example of Practical ML with Python and scikit-learn.
# Follow the data, learned values, predictions, and evaluation in order.
new_customer = pd.DataFrame({
    "tenure_months": [3],
    "monthly_charges": [95.0],
    "contract_type": ["month-to-month"],
    "support_tickets": [5],
    "total_charges": [285.0],
})

new_customer_processed = preprocessor.transform(new_customer)
churn_prob = best_model.predict_proba(new_customer_processed)[:, 1]
print(f"Predicted churn probability for this new customer: {churn_prob[0]:.2%}")

Expected Output (approximate):

Predicted churn probability for this new customer: 78.40%

Step 10: Interpret Results

# Build a small, inspectable example of Practical ML with Python and scikit-learn.
# Follow the data, learned values, predictions, and evaluation in order.
feature_names = (numerical_features +
                  list(preprocessor.named_transformers_["cat"].get_feature_names_out(categorical_features)))
importances = best_model.feature_importances_

importance_df = pd.DataFrame({
    "feature": feature_names,
    "importance": importances
}).sort_values("importance", ascending=False)

print(importance_df)

Expected Output (approximate):

                       feature  importance
0                tenure_months    0.342
3                total_charges    0.298
1             monthly_charges    0.187
4  contract_type_month-to-month   0.098
2             support_tickets    0.075

🧠 Feature importance (Module 9’s tree-based interpretability) tells the business why the model predicts churn — tenure_months and total_charges matter most, directly actionable insight for a retention strategy, not just a black-box probability.

6. Mathematical Intuition

Read the mathematics as a story

load data → split → pipeline → fit → evaluate → predict → save

First identify the input, the operation, and the output. Then read the symbols as a shorter way to describe that same journey; do not begin by memorizing the formula.

No new formulas — this module is entirely about correctly assembling Modules 4, 5, 7-9, 13-17’s mathematics into one working, disciplined pipeline, in the right order.

7. Small Worked Example

Walk through the example

Identify what each input number represents.
Follow one operation at a time and keep the units or class meanings attached.
Translate the result back into an ordinary sentence about the original problem.

The goal is not merely to obtain the answer; it is to expose the model’s decision process.

Already fully worked, step by step, in Section 5 above — this module is the simple/complete example.

8. Python Example

What the code will demonstrate

The following Practical ML with Python and scikit-learn code turns the worked example into an experiment you can repeat. First predict the result; then prepare the small dataset, apply the technique, inspect the important intermediate values, and compare the actual output with your prediction.

Python and library symbols used below

NumPy (np) stores and calculates with numeric arrays.
pandas (pd) represents table-shaped data when it is used.
scikit-learn provides tested implementations with a consistent .fit(...) and .predict(...) workflow.

The complete code is Section 5, assembled end to end. Here’s the entire pipeline as one continuous script for reference:

# Build a small, inspectable example of Practical ML with Python and scikit-learn.
# Follow the data, learned values, predictions, and evaluation in order.
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import f1_score, classification_report

# 1-3: Load, inspect, clean (see Section 5 for full data generation/cleaning)
# ... (data preparation as shown above) ...

# 4: Split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# 5: Preprocess
preprocessor = ColumnTransformer(transformers=[
    ("num", StandardScaler(), numerical_features),
    ("cat", OneHotEncoder(drop="first"), categorical_features),
])
X_train_processed = preprocessor.fit_transform(X_train)
X_test_processed = preprocessor.transform(X_test)

# 6-8: Train + tune
param_grid = {"n_estimators": [50, 100, 200], "max_depth": [3, 5, 7, None]}
grid_search = GridSearchCV(RandomForestClassifier(random_state=42), param_grid, cv=5, scoring="f1")
grid_search.fit(X_train_processed, y_train)
best_model = grid_search.best_estimator_

# 7: Evaluate
print(classification_report(y_test, best_model.predict(X_test_processed)))

# 9-10: Predict + interpret
# ... (see Section 5) ...

9. Real-World Example

This exact workflow — with real customer data instead of simulated data — is genuinely how many companies build their first production churn model. The specific business value: identifying customers with high predicted churn probability before they cancel, allowing a retention team to proactively reach out with an offer — directly connecting this technical pipeline to real business impact.

10. How This Is Used in AI

From mechanism to product

scikit-learn remains excellent for structured-data baselines and production pipelines. LLM applications can call these models as tools or combine their predictions with generative output.

How this connects to LLMs

request → data or context preparation → model computation → evaluated output

An LLM may use this idea during training, or an AI application may use a separate ML component around the LLM. Those are different locations in the system, and the explanation below identifies which one applies.

🤖 How Is This Used in AI?

Direct relevance to Agentic AI: Moderate. This exact workflow pattern — load, clean, split, preprocess, train, evaluate, tune, predict, interpret — is precisely what you’d use to build any of the classical ML components inside a larger AI/agent system from Module 20’s architecture: a routing classifier, a content moderation model, a reranking model, or an evaluation scoring model.

The discipline demonstrated here (splitting before preprocessing, tuning via cross-validation, evaluating with metrics that actually matter, interpreting results) is exactly the discipline that should apply to any of those components, not just to standalone churn prediction.

11. How This Is Used in Agentic AI

Trace one agent step

goal + state → model proposes → runtime validates → tool or response → evaluation

The model produces a prediction or proposal. The agent runtime is ordinary software that manages tools, permissions, state, retries, and execution; it may use this ML concept directly, indirectly through an LLM, or not at all.

🤖 If you needed to build, say, a classifier that decides whether an agent’s tool call succeeded or needs to be retried, or a model scoring how relevant a retrieved document is likely to be, this exact ten-step workflow — applied to that specific dataset and problem — is precisely how you’d build it: no different in structure, just a different dataset and target label, feeding into the larger AI/agent architecture from Module 20.

12. Common Beginner Mistakes

⚠️ Mistake

Incorrect idea: Skipping the data inspection step and jumping straight to modeling

Why it is incorrect: As this module’s Step 1-2 shows, data quality issues (missing values, impossible values like negative tenure) are common and need to be caught before they silently corrupt training.

⚠️ Mistake

Incorrect idea: Fitting the preprocessor on the full dataset before splitting

Why it is incorrect: This module deliberately shows fit_transform on training data only, then transform (not fit_transform) on test data — reversing this order reintroduces Module 4’s leakage problem.

⚠️ Mistake

Incorrect idea: Optimizing hyperparameter search for accuracy by default, without considering whether it’s the right metric

Why it is incorrect: This module deliberately tunes for F1, not accuracy, reflecting Module 17’s lesson that the “default” metric isn’t always the right one for an imbalanced business problem like churn.

13. Important Distinctions

Training Data Processing	Test Data Processing
`preprocessor.fit_transform(X_train)`	`preprocessor.transform(X_test)` — NEVER fit again
Establishes the scaling/encoding rules	Applies the SAME rules established from training

Cross-Validation Score (during tuning)	Final Test Score
Used to CHOOSE the best hyperparameters	Used ONCE, to report final, honest performance

14. When Should You Use This?

Use this exact ten-step workflow as a template for any new supervised learning project — classification or regression, tabular or structured data — as a reliable, disciplined default structure, adapting individual steps (model choice, specific preprocessing, specific metrics) to your particular problem.

15. When Should You NOT Use This?

For genuinely simple, low-stakes exploratory analysis, the full discipline of this workflow (formal train/test split, grid search tuning, feature importance interpretation) may be more process than the situation warrants — proportion the rigor to how consequential and long-lived the resulting model/analysis actually needs to be.

16. Production Considerations

Save the fitted preprocessor AND model together — Module 5/21’s guidance: at inference time, new data must go through the exact same fitted preprocessing before reaching the model.
Version and register the final model (Module 21) before deploying, along with its evaluation metrics and the exact hyperparameters used.
Re-run this entire pipeline periodically as new data arrives — churn patterns (like most real-world patterns) can drift over time (Module 21), and periodic retraining/re-evaluation is standard practice.

17. AI Engineer Takeaway

🎯 AI Engineer Takeaway: This ten-step workflow — load, inspect, clean, split, preprocess, train, evaluate, tune, predict, interpret — is the practical backbone of essentially every classical ML project you’ll ever build, whether it’s a standalone business model or a component inside a larger AI/agent system.

The specific algorithms and metrics will vary by problem, but this disciplined structure, and specifically the ordering (split before preprocessing, tune via cross-validation, evaluate honestly on a held-out test set) should remain consistent every time.

18. Interview Questions

Basic Questions

Q: Walk through the typical steps of an end-to-end ML project.

A: Load and inspect the data to understand its shape and quality; clean it (handle missing values, fix data errors); split it into train and test sets; preprocess the data (scaling, encoding), fitting only on the training set; train a model on the processed training data; evaluate it using appropriate metrics on the held-out test set; tune hyperparameters using cross-validation; and finally use the trained model to make predictions on new data, interpreting the results (e.g., via feature importance) to extract actionable insight.

Q: Why is it important to fit your preprocessing steps (like scaling) on the training data only, and just “transform” (not re-fit) the test data?

A: Fitting the preprocessor separately on the test data (or on the combined full dataset before splitting) leaks information about the test set’s distribution into the preprocessing step, compromising the test set’s ability to give an honest, unbiased evaluation of how the model will perform on genuinely new data. The correct approach establishes scaling/ encoding rules using only the training data, then applies those exact same fixed rules to the test data (and any future production data).

Intermediate Questions

Q: In this module’s churn prediction example, why was the model tuned to optimize for F1 score rather than accuracy?

A: Churn prediction is a moderately imbalanced classification problem (about 33% churn rate here) — optimizing purely for accuracy risks favoring a model that’s biased toward predicting the majority class (“not churned”), potentially at the cost of actually catching genuine churn cases (recall). F1 balances precision and recall, which is generally a more meaningful metric to optimize for when both catching true churners (recall) and not over-flagging loyal customers (precision) matter to the business.

Q: Why is feature importance from a trained random forest genuinely useful beyond just having accurate predictions?

A: Feature importance reveals why the model is making its predictions — which factors (tenure, contract type, support tickets, etc.) are actually driving churn risk. This transforms a black-box probability score into actionable business insight: a retention team can specifically target interventions (like better support for high-ticket-volume customers, or incentivizing longer contracts) based on genuinely influential factors, rather than treating the model purely as an opaque prediction generator.

Scenario-Based Questions

Q: After deploying this churn model, the business asks: “the model predicts a customer has a 78% chance of churning — should we automatically send them a large discount offer?” How would you respond, connecting back to concepts from across this course?

A: Thought process: This question tests whether the technical output (a probability) is being appropriately connected to a real business decision, which requires more than just the model’s raw number.

Investigation: A 78% predicted churn probability is a genuinely useful signal, but automatically triggering an expensive action (a large discount) for every customer above this threshold deserves further consideration: what’s the actual cost of a false positive here (offering a discount to someone who wasn’t actually going to churn) versus a false negative (missing a genuine churn risk)? This is exactly Module 17’s precision/recall trade-off, now applied as a real business cost decision — the “right” probability threshold for triggering an intervention isn’t necessarily 50%, and should be chosen based on the actual relative costs involved, not just model output.

Correct answer: Recommend the business explicitly define the cost of each error type (unnecessary discount vs. missed retention opportunity), and use that to choose an appropriate probability threshold for triggering interventions — potentially different from the model’s default 0.5 classification threshold. Also recommend evaluating this decision with a controlled experiment (e.g., a randomized subset actually receiving the offer, compared against a control group) rather than assuming the model’s prediction alone justifies acting on every flagged customer.

Production consideration: This scenario is a good illustration of the gap between “the model works well” (strong F1/AUC in offline evaluation) and “the model is being used correctly” in an actual business process — the technical output needs to be paired with genuine business judgment about costs, thresholds, and validation through real-world experimentation, not applied mechanically without that additional layer of reasoning.

Next: Module 24 — ML Interview Masterclass — a comprehensive interview preparation module covering the full course, with detailed answers and realistic scenario-based questions.

Author: TechByteByByte Editorial Team
Reviewed by: TechByteByByte Admin
Published: August 31, 2026
Last reviewed: August 31, 2026