Blogs/Predicting with a Neural Network

# Predicting with a Neural Network

peterwashington Nov 02 2021 11 min read 0 viewsNeural Networks

Blogs/Predicting with a Neural Network

Neural Networks

Neural networks are made up of *nodes* and *edges*. Nodes are analogous to neurons in the brain, and edges are analogous to synapses (the structures in the brain connecting neurons and sending electrical or chemical signals from neuron to neuron). Nodes are connected by edges (also called *weights*), and they are often visualized like this:

The first set of nodes, on the leftmost side of the neural network, are collectively called the *input layer*. The last set of nodes, on the rightmost side of the neural network, are collectively called the *output layer*. All other layers in between are called *hidden layers*.

After a neural network is trained, the model will have learned values for all of its weights. A trained neural network could end up with the following weights, for example:

To calculate the value of a node in a hidden layer or output layer, you multiply the weight of the arrow coming into that node with the value of the node that arrow is coming from, for each arrow pointing to the node, and add up all of these products. For example, let’s say we want to calculate the value of the hidden node below given the prior layer’s node values and weight values:

We calculate the value of the new node as follows:

*8 × -4+3 × -1+6 × 2= -23*

The edges correspond to the m variables and the nodes correspond to the x variables in mx + b. In this way, the edges are the learned parameters of the neural network.

Nodes also have a value for b, which we call the *bias* parameter. For each node, we could write the value of the node y as y = m_{1}x_{1} + m_{2}x_{2} + … + m_{N}x_{N} + b.

For simplicity in the following examples in this chapter, we will assume that the bias value (the value of b) is set to 0.

The definition of a neural network we have covered so far limits the learned equation to be a linear equation. Let’s say we have this neural network, where the vector [x_{1}, x_{2}, x_{3}] is the input to the neural network and y is the output of the network:

We get the following equation for y (remember that we are setting the bias to 0 for simplicity):

To get the equation for y in terms of the input nodes, we must write out the equation for x_{4} and x_{5}:

The equation for y in terms of the input nodes is obtained by plugging in these equations for x_{4} and x_{5 }back into the equation for y (again, remember that we are setting the bias to 0 for simplicity):

This is just a linear equation of the inputs. We might as well just use linear or logistic regression. So why are these more complicated neural networks so popular today? The reason comes down to *activation functions*. We will cover this topic in the next blog.