Artificial neural networks are mathematical models inspired by the biological structure of the brain. They are used to approximate complex unknown functions which depend on many input parameters. They have the ability to learn from examples and gather experience over time. Neural networks are gaining popularity and are already used in various areas like: function approximation, timeseries prediction, pattern recognition, data classification, computer vision, data processing and compression, robotics, etc. The list goes on…We will try to create and train a neural network for financial timeseries forecasting.
There are different types of neural networks and in this post I will talk about the most commonly used: feedforward neural networks using backpropagation learning algorithm. Let’s first take a look at the structure of a feedforward neural network:
The network consists of several number of smaller units called neurons. The neurons are organized into 3 or more separate layers: an input layer, one or more hidden layers and an output layer. We feed the input neurons with some data (examples during the training), they don’t have other purpose. Then data is forward propagated through all the layers sequentially and finally some output is produced from the neurons in the output layer. Each neuron (except for input ones) is connected to all neurons from the previous layer via weighted links (synapses). The weights are initially random and are continuously adjusted during the training process. They represent the network’s knowlege and experience. There’s no limit to the number of neurons in a layer, nor to the number of hidden layers. Usually when a task is approached several neural networks with different structures are examined to determine the best one.
Each neuron (except for input neurons) perform simple math operations that can be divided in two steps:
- the first step is summation: all the values coming from previous layer neurons multiplied by the weights are summed up;
- in the second step we pass the result from step 1 as an argument to a transfer function. The transfer function can by any math function as soon as it is differentiable. That’s a requirement for the backpropagation algorithm. Sine, cosine, hyperbolic tangent, gaussian function are all examples of commonly used transfer functions in neural networks.
Below is the structure of a single neuron. The function Φ is the transfer function which is the sine function in this example:
1. Σ = x1.w1 + x2.w2
2. y = Φ(Σ) = sin(Σ)
The output of y becomes an input for the neurons in the next layer. Once we get the output values of the last layer, we compare them to the actual (real) values that are expected. Hence the difference between the real and outputed values is the network error. Then the backpropagation algorithm is used to share the output error among all neurons of the network, starting from the output layer and going backward towards the input layer. During this stage distributed error is used to update all the weights such that if the same example is presented to the inputs again, the error at the outputs will be less. For each example, the data is first propagated forward through all layers, the error at the output is calculated and then error is backpropagated through all layers backwards. During the learning process we present a lot of examples to the network but the weights are updated with just a small fraction (learning rate) of the calculated value each time. This process repeats with all examples many times (epochs).This is done until the network error has become small enough or until a certain number of epochs has been performed.
For example, we can create a neural network and train it to recognize the digits from 0 to 9. If we include handwritten examples that will make the task even more difficult. However, if we present a huge amount of examples (digits printed and handwritten by various people) during the training stage, the network will then be able to recognize digits that were not present in the examples. If a new guy comes and writes a digit the neural nework should be able to recognize the unknown example. This method of learning is called supervised learning as we provide examples to learn from.
It should be noted that neural networks are not perfect. They try to classify and recognize input data based on their previous experience but that doesn’t mean they are always correct.
Another important thing is the input data, it should be carefully selected and preprocessed first. Raw data is rarely directly given to the network. Usually we try to extract some features from the raw data and then scale and normalize it. A good practice is to normalize all the data in the range [-1; 1] or [0; 1] as this gives better results.
Back to your main task. We will use a neural network to predict the price movement of EURUSD for the next day. In order to train our network we have to provide enough examples so it can try to discover any patterns in them and use them to predict future price movement. Historical data is freely available. We should decide how to preprocess the historical prices first. We can use some technical indicators since they are based on raw data. Good candidates are Bollinger Bands, RSI, Stochastic Oscillator, custom indocators, etc. And we can also include some additional inputs like month of the year, day of the week in case there’s some seasonality in the price movements.
In the next post I will reveal exactly what data was selected to train the neural network, how it was normalized and what architecture was used. Performance and results will be published and then the same trained network will be used in practice publishing the forecast and actual outcome every day.
To be continued….