Skip to main content

Neural network

· One min read

Intuitive understanding​

A neural network is pretty much just a function that maps a bunch of inputs to a bunch of outputs. First that function does bad at mapping. By showing a lot of input/output pairs the parameters in the function get adjusted to improve the mapping.

So there are three big parts of a neural network. The architecture of the network, the optimization of the parameters and the amount and quality of the data.

Architecture​

  • How many layers?
  • What type of layers?
  • What activation functions?
  • Input and output dimensions?

Optimization​

  • What does the loss function look like?
  • Gradient descent?
  • What optimizer?
  • When and how fast to change the parameters?
  • When to stop training?
  • Is there overfitting?

Data​

  • How much data is there?
  • Is Data argumentation necessary and/or useful?
  • Can there be too much data?
  • Is there bias in data?

Practical Stuff

Perceptron​

The Perceptron is the simplest neural network possible.

Implement small deep learning library from scratch (with numpy)​

At some point!! To help with a deeper understanding of backpropagation and the inner workings in general.