Biasing Neural Networks Towards Exploration or Exploitation Using Neuromodulation

TitleBiasing Neural Networks Towards Exploration or Exploitation Using Neuromodulation
Publication TypeConference Paper
Year of Publication2007
AuthorsParussel, K, Cañamero, L
Editorde Sá, JMarques, Alexandre, LA, Duch, W, Mandic, D
Name of ProceedingsProc. 17th International Conference on Artificial Neural Networks (ICANN 2007), Part II
Series TitleLNCS
Series Volume4669
Date Published09/2007
PublisherSpringer Berlin Heidelberg
Conference LocationPorto, Portugal
ISBN Number978-3-540-74695-9

Taking neuromodulation as a mechanism underlying emotions, this paper investigates how such a mechanism can bias an artificial neural network towards exploration of new courses of action, as seems to be the case in positive emotions, or exploitation of known possibilities, as in negative emotions such as predatory fear. We use neural networks of spiking leaky integrate-and-fire neurons acting as minimal disturbance systems, and test them with continuous actions. The networks have to balance the activations of all their output neurons concurrently. We have found that having the middle layer modulate the output layer helps balance the activations of the output neurons. A second discovery is that when the network is modulated in this way, it performs better at tasks requiring the exploitation of actions that are found to be rewarding. This is complementary to previous findings where having the input layer modulate the middle layer biases the network towards exploration of alternative actions. We conclude that a network can be biased towards either exploration of exploitation depending on which layers are being modulated.