# Coursework 2: Logistic Regression

## Objective

Implement a multi-class logistic regression classifier from scratch using NumPy, and compare the effect of different loss functions on the FashionMNIST dataset.

## What You Need to Implement

| File | What to implement |
|---|---|
| `losses.py` | `softmax()`, `cross_entropy_loss()`, `hinge_loss()`, `exponential_loss()`, `squared_loss()` |
| `model.py` | `softmax()`, `forward()`, `predict()` |
| `optimizer.py` | `GradientDescent.__init__()`, `GradientDescent.step()` |

All TODOs are clearly marked with hints in the code.

## How to Run

```bash
# First, set up the dataset (from project root)
cd ..
python setup_data.py
cd cw2_logistic_regression

# Quick mode (for debugging)
python run.py --quick

# Train with specific loss function
python run.py --loss_type cross_entropy
python run.py --loss_type hinge
python run.py --loss_type exponential
python run.py --loss_type squared

# Compare all 4 loss functions (generates comparison plot for report)
python run.py --compare_all

# Run gradient checks
python -m tests.test_cw2
```

## How to Verify Your Implementation

1. Run `python -m tests.test_cw2` — all gradient checks should pass
2. Run `python run.py --loss_type cross_entropy` — should reach ~84% test accuracy
3. Run `python run.py --compare_all` — generates loss comparison plot

## Deliverables

1. **Code**: Completed `losses.py`, `model.py`, `optimizer.py`
2. **Report** (use the template in `../../report_template/`, see example in `../../report_examples/cw2_example_report.pdf`):
   - Loss comparison with training curves
   - Analysis of convergence speed, final accuracy, and stability
   - Discussion of which loss works best and why

## Grading Rubric (100 points)

| Component | Points | Criteria |
|---|---|---|
| Softmax implementation | 10 | Numerically stable, correct shape, sums to 1 |
| Cross-entropy loss + gradient | 15 | Correct loss, gradient passes numerical check |
| Hinge loss + gradient | 15 | Correct Crammer-Singer formulation |
| Exponential loss + gradient | 10 | Correct, numerically stable |
| Squared loss + gradient | 10 | Correct formulation |
| Gradient descent optimizer | 10 | Correct update with mini-batches |
| Model forward + predict | 5 | Correct logits and argmax |
| Training convergence | 10 | Cross-entropy >= 82% test accuracy |
| Report: loss comparison | 10 | All 4 losses compared with curves and analysis |
| Code quality | 5 | Clean, commented, follows structure |

**Bonus (up to +10):** L2 regularization (+5), learning rate schedule (+5)
