Chapter 2.0: Classification

Classification is similar to regression analysis, however in this setting we are now dealing with categorical data (e.g. an animals dataset my have categories for cat, dog, etc). In a classification problem we will be given \(K\) classes and some data \(X\) which we use to train a model for class predictions. The process of building models and training them is the same as for what we did in regression analysis.

1. Describe a model with unknown parameters (\(\theta\)) that will accurately model the data

2. Define an appropriate loss function for the classification problem

3. Given the training data minimise the loss function with respect to the parameters of your model

One last important thing to mention is that since our data is categorical we one hot encode it so that our models can use the data. As an example if we have the classes cat, dog and pig we can one hot encode cat as \((1,0,0)^T\), dog as \((0,1,0)^T\) and pig as \((0,0,1)^T\). So one hot encoding is mapping class i to \(e_{i}\), the vector of all zeros except a one in the ith position.