Bearded-android-docs

Created Monday 28 April 2014

The error function measures the difference between the expected and actual output and is a function of the weights. There is actually more than one way of defining the error function.

Notation

• Please read the notation page

An Error Function

A common way of defining the error function is to let the error `E[i]` of the ith training example be

`\ \ E[i] = 1/2 ||bb o[i] - bb t[i]||^2`

where `||*||` is the euclidian distance.

Note 1: Since we're dealing with vectors, the error is the sum of the differences of the expected (`t[i]`) and actual (`o[i]`) output for each neuron in the output layer. Each `E[i]` actually looks like this:

`\ \ E[i] = (o_1[i] - t_1[i])^2 + cdots + (o_m[i] - t_m[i])^2`

where `o_j[i]` is the actual output of the jth neuron (in the output layer) and `t_j[i]` is the expected output of the jth neuron (in the output layer).

Note 2: This is just one of many different error functions.

The error function is actually a function of the weights. Each input `bb x[i]` and expected output `bb t[i]` are fixed. The only variables are the weights.

We care about minimizing the total error `E` of the network:

`\ \ E = sum_i E[i]`
`\ \ \ \ = 1/2 sum_i ||bb o[i] - bb t[i]||^2`
`\ \ \ \ = 1/2 sum_i (o_1[i] - t_1[i])^2 + cdots + (o_m[i] - t_m[i])^2`