Part II: Creating and Adding Learning Rules
What is considered a learning rule?
Learning rules are necessary for the automated adaption of network parameters during simulation. At present, BindsNET supports two different categories of learning rules:
- Two factor: Associative learning takes place based on pre- and post-synaptic neural activity. Examples include:
The typical example is Hebbian learning, which may be summarized as “Cells that fire together wire together.” That is, co-active neurons causes their connection strength to increase.
Spike-timing-dependent plasticity (STDP) stipulates that the ordering of pre- and post-synaptic spikes matters. A synapse is strengthened if the pre-synaptic neuron fires before the post-synaptic neuron, and, conversely, is weakened if it fires after the post-synaptic neuron. The magnitude of these updates is a decreasing function of the time between pre- and post-synaptic spikes.
- Three factor: In addition to associating pre- and post-synaptic neural activity, a third factor is introduced which modulates plasticity on a more global level; e.g., for all synapses in the network. Examples include:
(Reward, error, attention)-modulated (Hebbian learning, STDP): The same learning rules described above are modulated by the presence of global signals such as reward, error, or attention, which can be variously defined in machine learning or reinforcement learning contexts. These signals act to gate plasticity, turning it on or off and switching its sign and magnitude, based on the task at hand.
The above are examples of local learning rules, where the information needed to make updates are thought to be available at the synapse. For example, pre- and post-synaptic neurons are adjacent to synapses, rendering their spiking activity accessible, whereas chemical signals like dopamine (hypothesized to be a reward prediction error (RPE) signal) are widely distributed across certain neuron populations; i.e., they are globally available. This is in contrast to learning algorithms like back-propagation, where per-synapse error signals are derived by computing backwards from a loss function at the network’s output layer. Such error derivation is thought to be biologically implausible, especially compared to the two- and three-factor rules mentioned above.
Creating a learning rule in BindsNET
At present, learning rules are attached to specific
Connection objects. For
example, to create a connection with a STDP learning rule on the synapses:
from bindsnet.network.nodes import Input, LIFNodes from bindsnet.network.topology import Connection from bindsnet.learning import PostPre # Create two populations of neurons, one to act as the "source" # population, and the other, the "target population". # Neurons involved in certain learning rules must record synaptic # traces, a vector of short-term memories of the last emitted spikes. source_layer = Input(n=100, traces=True) target_layer = LIFNodes(n=1000, traces=True) # Connect the two layers. connection = Connection( source=source_layer, target=target_layer, update_rule=PostPre, nu=(1e-4, 1e-2) )
The connection may be added to a
Network instance as usual. The
update_rule, of type
bindsnet.learning.LearningRule, as well
nu, a 2-tuple specifying pre- and post-synaptic learning rates; i.e., multiplicative
factors which modulate how quickly synapse weights change.
Learning rules also accept arguments
reduction, which specifies how parameter updates are
aggregated across the batch dimension, and
weight_decay, which specifies the time constant
of the rate of decay of synapse weights to zero. By default, parameter updates are averaged across
the batch dimension, and there is no weight decay.
Other supported learning rules include
MSTDP (reward-modulated STDP), and
MSTDPET (reward-modulated STDP with
Custom learning rules can be implemented by subclassing
and providing implementations for the types of
AbstractConnection objects intended to be used.
For example, the
LocalConnection objects rely on the implementation
of a private method,
_connection_update, whereas the