Classification: Predicting Categories

In the last module, we used Linear Regression to predict a number (house price). But what if you want to predict a category — like "pass or fail", "spam or not spam", "cat or dog"?

This is called Classification.

---

1. Regression vs Classification

Type	What it Predicts	Example
Regression	A number (continuous)	House price: ₹50 Lakhs
Classification	A category (discrete)	Email: Spam or Not Spam

---

2. Logistic Regression (Don't Let the Name Fool You!)

Despite having "Regression" in its name, Logistic Regression is a classification algorithm. It predicts the probability that something belongs to a category.

Example: Predicting Student Pass/Fail

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# Features: [study_hours, attendance_percentage]
X = np.array([
    [8, 90], [2, 40], [6, 85], [1, 30],
    [7, 88], [3, 50], [5, 70], [9, 95],
    [4, 60], [2, 35], [6, 75], [8, 92]
])

# Target: 1 = Pass, 0 = Fail
y = np.array([1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1])

# Split the data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

# Create and train the model
model = LogisticRegression()
model.fit(X_train, y_train)

# Predict on test data
predictions = model.predict(X_test)
print("Predictions:", predictions)
print("Actual:", y_test)

# Check accuracy
accuracy = accuracy_score(y_test, predictions)
print(f"Accuracy: {accuracy * 100:.1f}%")

---

3. Decision Tree (The Most Visual ML Model)

A Decision Tree makes decisions by asking a series of yes/no questions — like a flowchart.

                    Study Hours > 4?
                    /           \
                  Yes            No
                  /               \
          Attendance > 70?        FAIL
          /           \
        Yes            No
        /               \
      PASS             FAIL

Code:

from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score

# Using the same student data from above
model = DecisionTreeClassifier(random_state=42)
model.fit(X_train, y_train)

predictions = model.predict(X_test)
accuracy = accuracy_score(y_test, predictions)
print(f"Decision Tree Accuracy: {accuracy * 100:.1f}%")

---

4. Understanding Model Accuracy

Accuracy

The simplest metric: what percentage of predictions were correct?

from sklearn.metrics import accuracy_score

accuracy = accuracy_score(y_test, predictions)
print(f"Accuracy: {accuracy * 100:.1f}%")

Confusion Matrix

A table that shows exactly where your model got it right and where it made mistakes:

from sklearn.metrics import confusion_matrix

cm = confusion_matrix(y_test, predictions)
print("Confusion Matrix:")
print(cm)

Output (example):

[[2, 0],    ← 2 correctly predicted as Fail, 0 wrongly predicted as Pass
 [1, 1]]   ← 1 wrongly predicted as Fail, 1 correctly predicted as Pass

Classification Report

A complete summary with Precision, Recall, and F1-Score:

from sklearn.metrics import classification_report

report = classification_report(y_test, predictions, target_names=["Fail", "Pass"])
print(report)

Simple rules:

• Precision = "Of all the students I predicted as Pass, how many actually passed?"
• Recall = "Of all the students who actually passed, how many did I correctly predict?"
• F1-Score = A balance between Precision and Recall

---

5. Predicting New Students

import numpy as np

# New student: 7 hours study, 80% attendance
new_student = np.array([[7, 80]])
result = model.predict(new_student)
print("Prediction:", "Pass" if result[0] == 1 else "Fail")

# Predict for multiple students
new_students = np.array([[7, 80], [2, 30], [5, 65]])
results = model.predict(new_students)
for features, result in zip(new_students, results):
    status = "Pass" if result == 1 else "Fail"
    print(f"Hours: {features[0]}, Attendance: {features[1]}% → {status}")

---

Summary

Concept	What it Means
Classification	Predicting a category (pass/fail, spam/not spam)
Logistic Regression	Predicts probability of belonging to a class
Decision Tree	Makes decisions using a flowchart of yes/no questions
Accuracy	% of correct predictions
Confusion Matrix	Shows right vs wrong predictions in detail
Precision / Recall	Deeper measures of model quality

• Regression predicts numbers. Classification predicts categories.
• Always check accuracy and confusion matrix — not just accuracy alone.
• Decision Trees are great for understanding what the model learned (they are visual).