Neural Network Anatomy: Building Your First AI Brain 🧠

"A neural network is like a brain made of math, where data flows through layers of weights, learning patterns one example at a time."

🎯 Exercise Overview

In this hands-on exercise, you'll build a simple neural network from scratch that learns to predict the next word in a sentence. You'll understand exactly how data transforms as it flows through the "brain" of weights and layers.

What You'll Build

 1📊 Data/Tokens → 🧠 Model (Layers + Weights) → 📉 Loss Function → 🔄 Training Loop → 🎯 Prediction

🔬 Part 1: Understanding the Anatomy

Let's start by visualizing the components of our AI brain:

The Big Picture

 1Input Sentence: "The cat sat on the"
 2                      ↓
 3            [Token Conversion]
 4                      ↓
 5              [1, 2, 3, 4, 1] ← Numbers the computer understands
 6                      ↓
 7               [Neural Network]
 8                 Layer 1: 5 → 10 neurons
 9                 Layer 2: 10 → vocab_size neurons
10                      ↓
11              [Probability Distribution]
12                      ↓
13              Prediction: "mat" (87% confidence)

Your output will appear here after running the code.
Compare with the expected results to validate your solution.

🧠 Your First Neural Network Class

Your output will appear here after running the code.
Compare with the expected results to validate your solution.

🔬 Part 2: Data Flow Analysis Drill

Now let's trace exactly how data flows through our neural network:

Your output will appear here after running the code.
Compare with the expected results to validate your solution.

🎯 Part 3: Loss Function - How Wrong Are We?

The loss function measures how "wrong" our prediction is. Let's implement and understand it:

Your output will appear here after running the code.
Compare with the expected results to validate your solution.

🔄 Part 4: Training Loop - Making the Brain Learn

Now let's implement the training loop where our AI actually learns:

Your output will appear here after running the code.
Compare with the expected results to validate your solution.

🧪 Part 5: Testing Our Trained Brain

Let's see how well our AI learned to predict words:

Your output will appear here after running the code.
Compare with the expected results to validate your solution.

🎨 Whiteboard Exercise: Visualize the Learning Process

Instructions for Whiteboard:

1. Draw the Neural Network Architecture:

   • Input layer (vocabulary size)
   • Hidden layer (10 neurons)
   • Output layer (vocabulary size)
   • Show weights as connections

2. Trace Data Flow:

   • Start with "the cat sat"
   • Show one-hot encoding
   • Show transformation through each layer
   • End with probability distribution

3. Illustrate Learning:

   • Show how weights change during training
   • Draw the loss curve over epochs
   • Explain why loss decreases

🏆 Final Challenge: Build Your Own Vocabulary

Your output will appear here after running the code.
Compare with the expected results to validate your solution.

🎯 Key Concepts Mastered

✅ Neural Network Anatomy: Data → Model → Loss → Training → Prediction
✅ Math to Code Translation: Understanding how formulas become Python
✅ Data Flow: Tracing information through layers and weights
✅ Loss Functions: Measuring and minimizing prediction errors
✅ Training Loops: The iterative process of learning
✅ Pattern Recognition: How AI learns to predict sequences

🚀 What's Next?

You've built your first neural network from scratch! Next exercises will cover:

• Epochs and Batch Training (Exercise 2)
• Advanced Loss Functions (Exercise 3)
• Gradient Descent Optimization (Exercise 4)
• Real-world Text Processing (Exercise 5)

"Every expert was once a beginner. Every pro was once an amateur. Every icon was once an unknown." - Robin Sharma