Fundamentals of Machine Learning

What is Machine Learning?

Machine learning is a subset of artificial intelligence that enables systems to learn and improve from experience without explicit programming. Instead of manually programming rules (if revenue > $1M, classify as "high value"), you provide examples of inputs and outputs, and the algorithm learns the underlying patterns.

This is fundamentally different from traditional programming. Traditional: you write rules. Machine learning: you provide data and the algorithm discovers rules.

The Learning Paradigm: Training Data

Every machine learning model learns from training data — examples of inputs paired with known outputs. The model's job is to find patterns in this data that generalize to new, unseen examples.

# Training data: customers (inputs) and whether they churned (output)
customer_data = [
    {'tenure_months': 12, 'monthly_spend': 50, 'support_tickets': 5, 'churned': True},
    {'tenure_months': 36, 'monthly_spend': 120, 'support_tickets': 1, 'churned': False},
    {'tenure_months': 6, 'monthly_spend': 25, 'support_tickets': 10, 'churned': True},
    # ... more examples
]

# The model learns: long tenure + high spend + few support tickets → likely to stay

The model's goal: predict churn on new customers you've never seen before.

Three Types of Machine Learning

Supervised Learning: Learning from Labels

In supervised learning, you provide labeled examples: inputs paired with correct outputs. The model learns the input-output relationship.

Regression: Predicting continuous numbers.

• Example: Predict house price based on square footage, bedrooms, location
• Output: $450,000 (a number)

Classification: Predicting categories.

• Example: Predict if an email is spam or not spam
• Output: "Spam" (a category)

Supervised learning requires labeled data, which is expensive to collect. You need humans to label examples: "this house sold for $500k," "this email is spam."

Unsupervised Learning: Finding Patterns

In unsupervised learning, you have data but no labels. The algorithm finds hidden patterns or groupings.

Clustering: Grouping similar examples.

• Example: Group customers into segments based on spending behavior
• Outcome: Three customer types discovered (high-value, medium, low)

Dimensionality Reduction: Simplifying high-dimensional data.

• Example: Compress 784 pixel features from handwritten digit images into 50 principal components
• Outcome: Same information with less noise

Unsupervised learning is useful for exploration: "What types of customers do we have?" without predefined categories.

Reinforcement Learning: Learning from Feedback

In reinforcement learning, an agent takes actions, receives feedback (rewards or penalties), and learns to maximize cumulative reward.

• Example: A game-playing AI learns to win by trying moves, getting feedback on scores, and improving strategy
• Outcome: Superhuman performance on chess, Go, video games

Reinforcement learning is complex and used in specialized domains. This course focuses on supervised and unsupervised learning.

The Training Process

1. Collect Data

Gather examples relevant to your problem. More data usually means better learning.

2. Prepare Data

Real data is messy: missing values, errors, inconsistent formats. Cleaning and preparation takes 60-80% of project time.

df = df.dropna()  # Remove rows with missing values
df['age'] = pd.to_numeric(df['age'], errors='coerce')  # Convert types

3. Train the Model

Feed the training data to the learning algorithm. The model adjusts its internal parameters to minimize prediction error.

from sklearn.linear_model import LogisticRegression

model = LogisticRegression()
model.fit(X_train, y_train)  # Learn from training data

4. Evaluate on Test Data

Test the model on data it has never seen. This reveals how well it generalizes to new examples.

accuracy = model.score(X_test, y_test)
print(f"Test Accuracy: {accuracy:.3f}")  # 0.87 (87% correct)

5. Iterate

If performance is poor, try different algorithms, more features, or more data. Machine learning is iterative.

The Core Challenge: Generalization

The central challenge in machine learning is generalization: learning patterns that work on new data, not just memorizing training data.

Overfitting: Model memorizes training data but fails on new examples.

• Example: A model learns "if name is 'John' and age is 35, customer will churn" — too specific.
• Result: High training accuracy, low test accuracy.

Underfitting: Model is too simple to capture patterns.

• Example: A model that always predicts "no churn" achieves moderate accuracy on balanced data but misses all patterns.
• Result: Low training accuracy, low test accuracy.

The goal is finding the sweet spot: a model complex enough to capture patterns, simple enough to generalize.

Key Vocabulary

• Features: Input variables (columns in your data). "What do you observe?"
• Target/Label: Output variable you're predicting. "What do you want to predict?"
• Training Data: Examples used to teach the model.
• Test Data: Examples used to evaluate generalization.
• Model: The learned algorithm that makes predictions.
• Prediction: The model's output for a new example.
• Accuracy/Error: How often the model's predictions match reality.

What This Course Covers

This course builds from fundamentals to production:

1. Foundations (This lesson): Core concepts and terminology
2. Supervised Learning (Lessons 2-3): Regression and classification
3. Unsupervised Learning (Lessons 4-5): Clustering and dimensionality reduction
4. Advanced Algorithms (Lessons 6-7): Trees, ensembles, neural networks
5. Model Validation (Lessons 8-9): Evaluation metrics and hyperparameter tuning
6. Production (Lesson 10): Deploying and monitoring models

Each lesson builds on the previous, introducing new algorithms and deeper concepts. By the end, you'll understand how to solve real problems end-to-end, from data to deployment.

---

Summary: Machine learning learns patterns from data rather than following explicit rules. Supervised learning predicts labeled outputs (regression for numbers, classification for categories). Unsupervised learning finds hidden patterns (clustering, dimensionality reduction). The central challenge is generalization — learning patterns that work on new data. This course equips you with the concepts, tools, and practices to build ML systems.

Next: Regression analysis — predicting continuous values from features.