Akida user guide

The Akida Execution Engine

Performing basic tasks with the Akida Execution Engine is quite straightforward. However, in fine tuning or for more complex tasks, an understanding of Spiking Neural Networks, the Akida Neuron Model, and the parameters that define it, is necessary.

Main steps to use the Akida Execution Engine:

  1. The Spiking Neural Network model – what layer types?

  2. Input data format

  3. Determine training mode

  4. Interpreting outputs

1. The Spiking Neural Network model

All Akida Spiking Networks require an input layer and an additional layer. Other layers may be required to optimize each Spiking Neural Network model, depending on the given task. Currently, the Akida Execution Engine supports serial feedforward architectures.

Three principal layer types are available:

These three layer types may be combined as required: a typical model for processing images like the MNIST dataset, might include the following series of layers:

InputData ➤ Convolutional ➤ Convolutional ➤ FullyConnected ➤ FullyConnected

2. Input data format

There are two initial considerations regarding the nature of the data sent to the Akida Execution Engine:

  • Data format: events or images?

  • Buffering: continuous data or frames?

Data format

The Akida Execution Engine processes spikes natively, so for many types of input data, the User will need to select a method of conversion for the input signal into an event-based representation, known as: data-to-spike conversion. The Akida Execution Engine also has built-in data-to-spike converters that will be available in the Akida NSoC. Currently, data-to-spike converters for pixel-based and event-based data are available; while other converters will be added in subsequent releases.

Note

When sending (event-based) pre-generated spikes, the first Akida layer should be of type: InputData.

Note

The Akida integrated spike generator for image data (8-bit grayscale or RGB) is set up by defining the first layer as type: InputConvolutional. This layer is configurable, either using trained-weights imported from a CNN or by setting its kernels through the Akida API. A special InputConvolutional layer with pre-defined kernels is included in the Akida Execution Engine: InputBCSpike. It contains orientation filters to detect edges.

Input events buffering

The Akida Execution Engine processes many packets of events at a time through its InputData layer – from tens to hundreds. This requires buffering the inputs to a layer which, depending on the nature of the data, is achieved in one of two ways:

  • By default, data is processed in the form of discrete samples: the events pertaining to a single subject should be grouped and passed to the Akida Execution Engine as a whole. In that case, no matter how many events are sent at once they are immediately processed as a single packet.

  • For continuous or ongoing types of data, like the stream of data events generated by a DVS camera, the InputData layer offers a packetizing feature. When packetizing is active, any number of events can be sent to the Akida Execution Engine one at a time, or in very large bursts. In either case, when packetizing is enabled, the InputData layer takes events in the order received until it reaches a predefined packet_size, and then sends that packet for processing.

Data frames parallelization

The Akida Execution Engine can process several image data frames in parallel through its InputConvolutional layer (or its InputBCSpike variant). All image frames should be grouped together inside a 4-dimensional array.

3. Determine training mode

The Akida NSoC supports for each layer loading pre-trained weights and parameters. Convolutional and FullyConnected layers support online Akida unsupervised learning of new weights from pre-defined parameters.

The choice between the two modes of operation for each layer depends first on the input type for that layer: Akida unsupervised learning can only be performed on input features in the form of binary events.

It also depends on the model architecture: Akida unsupervised learning is more efficient on shallow models.

For deep models where online learning is required, it is therefore recommended to import the weights of early layers from a pre-trained CNN, and to apply Akida unsupervised learning only on the last layer.

Akida Unsupervised learning

In this mode, an Akida Layer will typically be compiled with specific learning parameters, and then undergo a period of feed-forward training by letting it learn from inputs generated by previous layers from a relevant dataset.

Please refer to Using Akida Unsupervised Learning for details.

CNN to SNN conversion

The CNN2SNN toolkit provides a means to convert a Convolutional Neural Network (CNN) that was trained using Deep Learning methods to an Akida Model that can later be extended by adding additional Akida Layer types.

4. Interpreting outputs

Note that the Akida Execution Engine has an unsupervised learning mode. Consequently, while it can learn to respond differently to different classes of input, by definition it will never attach any meaning or “result” to its outputs. Therefore, many tasks will require a classification stage. Either simple labeling of neurons according to the classes that drive them best or, highly sophisticated classification schemes. The Akida Execution Engine also includes the possibility of (pseudo) supervised classification. To summarize, if a training sample has an attached label, it is possible to define a layer so that, during the training phase, neurons will only be allowed to learn a representation when their target class is presented. During inference, the output of the layer will then supply the class label directly.

Neural Network model

Specifying the Neural Network model

Akida Neural Network models are defined using the sequential API. This comprises creating a Model object and adding layers to it using the .add() method. The available layers are InputData, InputConvolutional, InputBCSpike, FullyConnected, Convolutional and SeparableConvolutional. Layers are built with a name and a list of named parameters that are described in the sections below.

Example of sequential definition of a model:

from akida import Model, InputData, FullyConnected, LearningType
model = Model()
model.add(InputData(name="input",
                    input_width=32,
                    input_height=32,
                    input_features=1))
model.add(FullyConnected(name="fully",
                         num_neurons=32,
                         threshold_fire=40))

Saving and loading

A Model object can be saved to disk for future use with the .save() method that needs a path for the model. The model will be saved as a file, typically .fb or .fbz (compressed), that describes its architecture and weights. A saved model can be reloaded using the Model object constructor with the full path of saved file as a string argument. This will automatically load the weights associated to the model.

model.save("demo_CharacterDVS.fbz")
loaded_model = Model("demo_CharacterDVS.fbz")

Input layer types

The first layer of a neural network must be one of three possible input layer types:

  • InputData – universal input layer type. The User must apply their own event-generating transformation on the data – except where the data is already in an appropriate event-based format, e.g., the output from a neuromorphic camera.

  • InputConvolutional - image-specific input layer, taking either RGB or grayscale pixel input.

  • InputBCSpike – an InputConvolutional layer with pre-defined weights to detect edges in images.

Data-Processing layer types

After the input layer all subsequent layers will be data-processing layers.

As mentioned before, these layers do not process events in isolation, but rather as groups of events – typically tens to hundreds of events together.

Each layer contains several neurons that are connected to the layer inputs according to different topologies defined by the layer type. A weight is assigned to each connection, and that weight is combined with the input to modify the neuron potential.

When the neuron potentials have been evaluated, the layer feeds them to an activation function that may or may not emit a spike.

A data-processing layer can be one of three types:

  • FullyConnected – each neuron is connected to members of the full set of possible inputs – hence ‘fully connected’, even though a much smaller number of connections are likely to be non-zero.

  • Convolutional – each neuron’s connection weights express a localized filter – typically a region that is a small fraction of the input’s height and width. This filter is tested across all x and y positions.

  • SeparableConvolutional - a variant of the Convolutional layer that is less computationally intensive due to simplified filters.

Both the FullyConnected and Convolutional layer types can be trained using the Akida training algorithm.

Activation parameters

The Akida activation function uses a quantization scheme to evaluate the neuron response when its potential goes beyond its firing threshold. The intensity of the response is measured by dividing the difference between the potential and the threshold in several quantization intervals that correspond to a set of quantized spike values. The default quantization scheme is binary : whenever the neuron potential is above the threshold, a spike with a value of one is emitted.

More generally, if we denote:

  • T the threshold,

  • s the length of a quantization interval,

  • p the neuron potential,

  • Q the quantized activation values.

T + n * s < p <= T + (n + 1)*s => response = Q[n]

All data-processing layers share the following activation parameters:

  • threshold_fire: integer value which defines the threshold for neurons to fire or generate an event. When using binary weights and activations, the activation level of neurons cannot exceed the num_weights value.

  • threshold_fire_bits: < one of [1, 2, 4]> Defines the number of bits used to quantize the neuron response (defaults to one bit for binary). Quantized activations are integers in the range [1, 2^(weights_bits) -1].

  • threshold_fire_step: a float value, defining the length of the potential quantization intervals for threshold_fire_bits = 4. For 2 bits, this is 1/4 of the length of the potentials intervals and it is not relevant for 1 bit.

Pooling parameters

The InputConvolutional, Convolutional and SeparableConvolutional layer types share the following pooling parameters:

  • [optional if pooling_type = Average] pooling_width , pooling_height: integer values, sets the width and height of the patch used to perform the pooling. If not specified it performs a global pooling.

  • [optional] pooling_type: PoolingType Sets the effective pooling type (defaults to NoPooling):

    • NoPooling – no pooling.

    • Max – computing the maximum of each region.

    • Average – computing the average values of each region.

  • [optional] pooling_stride_x, pooling_stride_y: integer values, set the horizontal and vertical strides applied when sliding the pooling patches. If not specified, a stride of pooling_width or pooling_height is applied.

Using Akida Unsupervised Learning

The Akida Unsupervised Learning is a unique feature of the Akida NSoC.

In this mode, an Akida Layer will typically be compiled with specific learning parameters and then undergo a period of feed-forward unsupervised or semi-supervised training by letting it process inputs generated by previous layers from a relevant dataset.

Once a layer has been compiled, new learning episodes can be resumed at any time, even after the model has been saved and reloaded.

Learning constraints

The following restrictions apply to Akida Unsupervised Learning:

  • Only FullyConnected and Convolutional layers can be trained,

  • Only layers with binary weights can be trained,

  • Only layers receiving binary inputs can be trained.

Compiling a layer

For a layer to learn using Akida Unsupervised Learning, it must first be compiled with specific learning parameters.

The only mandatory parameter is the number of active (non-zero) connections that each of the layer neurons has with the previous layer, expressed as the number of active weights for each neuron.

Optimizing this value is key to achieving high accuracy in the Akida NSoC. Broadly speaking, the number of weights should be related to the number of events expected to compose the items’ or item’s sub-features of interest.

For example, in the MNIST dataset, sample images comprise a 28x28 pixel squares, with substantial area of blank space, and a number of dark pixels composing the characters. If only the x-y locations of the dark pixels were sent as events to the neural network model, each sample would comprise of a few hundred events.

Note

This case is only given as an example – there are better ways of encoding image data as events.

To train the MNIST dataset, using a very simple neural network model configuration with a single FullyConnected type layer, it would be acceptable to set the number of weights per neuron in that range (in this case perhaps: 300). If neurons have more weights than required, they will acquire some ‘generalization’ that is: tolerance to slightly different forms of the pattern they are intended to detect – but, will also lose some ‘specificity’, or become more responsive to members of different classes.

The num_weights parameter has other dependencies, such as an event-generating threshold or threshold_fire and the number of events processed at a time, defined either by the size of an input image, the number of input events, or the packet_size of an InputData layer.

Learning parameters

The following learning parameters can be specified when compiling a layer:

  • num_weights: integer value which defines the number of connections for each neuron and is constant across neurons. When determining a value for num_weights note that the total number of available connections for a Convolutional layer is not set by the dimensions of the input to the layer, but by the dimensions of the kernel. Total connections = kernel_height x kernel_width x num_features , where num_features is typically the num_neurons of the preceding layer. num_weights should be much smaller than this value – not more than half, and often much less.

  • [optional] num_classes: integer value, representing the number of classes in the dataset. Defining this value sets the learning to a ‘labeled’ mode, when the layer is initialized. The neurons are divided into groups of equal size, one for each input data class. When an input packet is sent with a label included, only the neurons corresponding to that input class are allowed to learn.

  • [optional] initial_plasticity: floating point value, range 0–1 inclusive (defaults to 1). It defines the initial plasticity of each neuron’s connections or how easily the weights will change when learning occurs; similar in some ways to a learning rate. Typically, this can be set to 1, especially if the model is initialized with random weights. Plasticity can only decrease over time, never increase; if set to 0 learning will never occur in the model.

  • [optional] min_plasticity: floating point value, range 0–1 inclusive (defaults to 0.1). It defines the minimum level to which plasticity will decay.

  • [optional] plasticity_decay: floating point value, range 0–1 inclusive (defaults to 0.25). It defines the decay of plasticity with each learning step, relative to the initial_plasticity.

  • [optional] learning_competition: floating point value, range 0–1 inclusive (defaults to 0). It controls competition between neurons. This is a rather subtle parameter since there is always substantial competition in learning between neurons. This parameter controls the competition from neurons that have already learned – when set to zero, a neuron that has already learned a given feature will not prevent other neurons from learning similar features. As learning_competition increases such neurons will exert more competition. This parameter can, however, have serious unintended consequences for learning stability; we recommend that it should be kept low, and probably never exceed 0.5.