Introduction to Linear Regression: Normal Equation

Normal Equation (Method II)

The use of Normal Equation to solve for the optimal \(\ \theta\) and find the hypothesis function is an alternate method to what we talked about in the last post and probably the easiest, but of course they have their own benefits and drawbacks.

Here we don't need to define a cost function, therefore we don't need to select \(\ \alpha \); the learning rate or specify number of iterations. Same as before, X is our feature matrix and for the intercept term \(\ \theta_0\), a column of ones should be added to X before calculation.

Introduction to Linear Regression: Cost Function

Cost Function (Method I)

For calculating the cost in linear regression, typically we use Sum of Squared Error method(SSE)

\(\ J(\theta) = \frac{1}{2m}\sum_{i=1}^m{(h(x^{(i)})-{y^{(i)}})^2}\)

The goal is to minimize \(\ J(\theta)\) and figure out \(\ \theta\) values corresponding to the minimum cost. There are several optimization algorithms used to achieve this.

Introduction to Linear Regression: Polynomial Regression

Polynomial Regression

This form of regression is used to fit more complex functions and this is a general concept not restricted to linear regression, but also used commonly in classification algorithms such as Logistic Regression and Neural Networks as well. I hope to talk about it in detail in a future post. For now take a look at the following scatter plot.

Introduction to Linear Regression: A Machine Learning Approach

Supervised Learning is a form of learning in which we use known data with actual outputs from past experiences to model a relationship and this model is used to predict future outcomes. The known data used to build up the model is called 'training data'.

To build a supervised learning model we need,

1. Training Data
2. Hypothesis
3. Cost Function
4. Optimization method for Minimization