We define a function $h_{\theta}(x)$ to model the $d$ features in each $x^{(i)}$ as:

For $n$ training examples, we also define a cost function that we want to minimize:

Taking the derivative with respect to any $\theta_j$ we get:

With this derivative, we can now use gradient descent to take small steps towards the optimal $\theta$ with the following update rule:

Let $X$ be a matrix that contains each $\left(x^{(i)} \right)^T$ in its rows and has a size of ($n$ by $d+1$) where $d$ is the number of features in $x^{(i)}$ and $+ 1$ is for the intercept term.

Also, $\vec{y}$ is an $n$ dimensional vector that contains the labels for each training example. And $\theta$ is a $d$ dimensional vector that contains the weights for each feature.

Now, since $h_{\theta}(x) = \left(x^{(i)} \right)^T\theta$, we can rewrite this in matrix-vector form as $h_{\theta}(x) = X \theta$

With these, we can now rewrite $J(\theta)$ using the fact that $z^T z = \sum_i z_i^2$

Finally, to minimize $J(\theta)$, we find its derivative with respect $\theta$, set it equal to $0$ and simplify to see that $\theta$ is minimized when,

We assume that $y^{(i)}$ with $\epsilon^{(i)}$ as the noise in the $i$th example such that $\epsilon^{(i)} \sim \mathcal{N}(0, \sigma^2)$

This can be rewritten as $\epsilon^{(i)} = \theta^T x^{(i)} - y^{(i)}$. Now, since $\epsilon^{(i)} \sim \mathcal{N}(0, \sigma^2)$, hence $\left(y^{(i)} - \theta^T x^{(i)} \right) \sim \mathcal{N}(0, \sigma^2)$.

Finally, if we are given $x^{(i)}$, then $y^{(i)} \mid x^{(i)}$ will also follow $\mathcal{N} (0, \sigma^2)$ shifted by $\theta^T x^{(i)}$.

To find the maximum likelihood estimate of $\theta$, we need to maximize:

Since $log$ is a monotonically increasing function, we can maximize the following instead:

Solving this we see that maximizing $\ell(\theta)$ is equivalent to minimizing:

Notice, that the function that we need to minimized does not depend on $\sigma$ which means that we don't need to know that variance of our noise to be able to maximize our likelihood.