Spread this useful information with your friends if you liked.
- Hidden Units:
- Hidden units are neurons in a neural network that are not part of the input or output layers.
- Hidden units play a critical role in deep learning by allowing the network to learn complex representations of the input data.
- Different types of activation functions can be used for the hidden units, such as sigmoid, tanh, ReLU, and others.
- Cost Functions:
- A cost function, also known as a loss function, measures how well the neural network is performing on a specific task.
- The choice of cost function depends on the task being solved, such as regression, classification, or reconstruction.
- Common cost functions include mean squared error, cross-entropy, and structural similarity index.
- Error Backpropagation:
- Error backpropagation is a technique used to train neural networks by computing the gradient of the cost function with respect to the weights of the network.
- The backpropagation algorithm involves propagating the errors backward through the network, starting from the output layer and moving toward the input layer.
- During backpropagation, the gradients of the weights are computed and used to update the weights in the opposite direction of the gradient.
- Optimization Algorithms:
- Optimization algorithms are used to update the weights of the neural network during training.
- Gradient descent is a common optimization algorithm that uses the gradients computed during backpropagation to update the weights in the direction of the negative gradient.
- Other optimization algorithms, such as stochastic gradient descent, Adam, and Adagrad, use variations of gradient descent to update the weights more efficiently and reliably.
Spread this useful information with your friends if you liked.
