RETINAL AND CORTICAL ARRAY LAYOUT
The routines in this file assume the following organizations:
World Retina Cortex
RN-1 ------------ 0 ----------- 0 -----------
| | | | | |
| | | | | |
y | | r | | j | |
| | | | | |
| | | | | |
0.0 ----------- RN-1 ----------- N-1 -----------
0.0 x RN-1 0 c RN-1 0 i N-1
Unfortunately, even though the `i' index is called a row and corresponds to a C/C++ array row, it is currently plotted along the vertical dimension instead of the horizontal, as shown above. This will hopefully change soon so that the memory layout is a better match to the plotted data.
The following arrays all share the cortical map structure described above:
map 2D grid of units; weights active only in certain
rows.
resp_to_inp Activity level of each unit due only to the
afferent input, i.e. the response before settling
due to the lateral interactions.
map_activity Activity level of each unit in the current
iteration during and after settling.
prev_map_activity Settled activity level of each unit in the
previous iteration.
activity_marker Each neuron marked true if any neighbor in
a given radius is active; the rest are exempt from
further calculation.
DISTRIBUTED CORTICAL MAPS
For those arrays which are distributed across multiple PEs, the `rows' are strided across the cortex such that for each local row i in [0,perows), the row in the map is pe+i*NPEs. (E.g. if there are 64 PEs handling 3 rows each, PE 32 handles rows 32, 96, and 160). I.e.:
Cortex Local Rows Map Rows
0.-----------. 0.---. 0...-....-...
| | | | : | | | | :
| | | | : | | | | :
j | | j | | j : | | | | :
| | | | : | | | | :
| | | | : | | | | :
N-1`-----------' N-1`---' N-1..`-'..`-'..
0 i N-1 0 i (perows-1) 0 MAPROW(i) N-1
(where MAPROW(i) = pe+i*NPEs for local row i)
Unfortunately, the terms `i' and `row' are often used interchangeably for either of these two types of indexes, so be careful to distinguish between them when reading the code.
In this file, the only distributed cortical map is the `wts' array, which has the weights to each neuron stored on this PE (perows worth of data). The weights are also linked in to the global map at appropriate positions, which provides for an easy translation between `local' [0,perows) and `map' [0,N) rows.
The afferent weights are stored as a 3D array of values, one dimension being the eye number (usually 0 or 1) and the other two being the two spatial dimensions of the retina. The lateral weights are each stored as a 1-D array whose format is described in section LATERAL CONNECTIONS below.
COLLAPSED ROW-ORIENTED ARRAYS
Other arrays contain a single element for each row; these are often used when computing a quantity first for each row on the various PEs, then combining the result from all the PEs.
One element per local row:
activity_list
For each local row, contains a list of the column numbers of
units which were active in that row. The first element in each
row is the `count' of active units in that row, and the
following `count' elements are the numbers of the active
columns in that row. At the end of the iteration, this data is
propagated to the other processors and stored in the
map-oriented prev_activity_list.
One element per map row:
prev_activity_list
For each row in the map, contains a list of the column numbers
of units which were active in the preceding iteration. See the
description of activity_list below. Only weights to/from units
active in the preceding iteration are modified in this one.
gather_activity
Like prev_activity_list, except that each element is an activity
level, not a column number. (The first element still represents
the `count}.)
LATERAL CONNECTIONS
Instead of being stored in a two-dimensional array like the afferent weights, lateral connections of each type are stored as one-dimensional arrays for which the 2D coordinates are computed by hand. I imagine that this complicated arrangement was chosen by Joseph Sirosh because the size of the lateral connection array varies periodically during training. If a fixed size but partially-populated 2D array was used instead, the active connections would be scattered throughout the array, resulting in poor locality of reference, which would lead to a high cache miss rate, which would lead to poor performance.
However, the current implementation contains many useless connections at the cortex boundaries, where as many as half of the connections are unused. Plus it contains all the dead weights after connections have been killed, so it doesn't get any faster as training progresses. Finally, the current implementation is an unholy mess to program, with index calculations duplicated all over the code in different forms. There must be a better way, probably some sort of optimized sparse array format.
In any case, as it currently stands, the formula for computing the 1-dimensional array index for the connection to unit (ui,uj) from unit (k,l) is:
index(k,l) = (k-(ui-radius))*ar_width + (l-(uj-radius))
where radius is either the current `inh_rad' or the current `exc_rad', depending upon the connection type, and ar_width is similarly either `inh_array_width' or `exc_array_width'.
This equation can be simplified slightly to:
index(k,l) = (k-ui+radius)*ar_width + l - uj + radius
which is how it usually appears in the code unless otherwise optimized.
If computing a number of such indexes for various values of l (various `columns' in a `row', using the terminology used in the code), the index can be partially computed beforehand to save calculation time:
partial_index(k) = (k-ui+radius)*ar_width - uj + radius)
The full index is then
index(k,l) = partial_index(k) + l
Macros have been defined in kernel.h to hide all of these details, but for computational reasons it's important to understand what's going on when they are used.
Definition in file kernel.c.
#include <math.h>
#include <vector>
#include "ind_types.h"
#include "ipc.h"
#include "cmdparam.h"
#include "binarysave.h"
#include "shuffledrand.h"
#include "globals.h"
#include "kernel.h"
#include "kernelwrapper.h"
#include "stringutils.h"
#include "neuralregion.h"
Include dependency graph for kernel.c:
Go to the source code of this file.
Compounds | |
| struct | Input |
| Record containing all we need to store about a given input region. More... | |
| class | LissomMap |
| Deprecated class offering old-style interface to routines in kernel.c. More... | |
| class | setfnobj2_exc_rad_setfn2 |
Defines | |
| #define | ACTLIST_START(actlist) 1 |
| Wrappers to hide structure of activity lists (actlists). | |
| #define | ACTLIST_COUNT(actlist) ((actlist)[0]) |
| #define | VERIFY_WEIGHT_SAVES_ERROR_LIMIT 10 |
| Maximum number of errors to report when verifying a weight file. | |
| #define | WITHIN_RADIUS(xdistance, ydistance, radius_sq) (((xdistance)*(xdistance) + (ydistance)*(ydistance)) <= (radius_sq)) |
| Returns true if the x,y coordinate is inside the given radius_sq. | |
| #define | WITHIN_RADIUS_SQ(xdistance_sq, ydistance_sq, radius_sq) ((xdistance_sq) + (ydistance_sq) <= (radius_sq)) |
| Returns true if the x,y coordinate (in squared form) is inside the given radius_sq. | |
| #define | assert_equals(x, y) if (x!=y) { failed=true; ipc_notify(IPC_ALL,IPC_ERROR,"Assertion failed: " #x "!=" #y " (%d!=%d)",x,y); } |
| User-visible assertion of the equality of two integer values. | |
| #define | assert_ge(x, y) if (!(x>=y)) { failed=true; ipc_notify(IPC_ALL,IPC_ERROR,"Assertion failed: !(" #x "=>" #y ") !(%d=>%d)",x,y); } |
Functions | |
| NeuralRegion * | LissomMap_create (const string &name_i, NeuralRegion::Dimensions dims, bool init_weights_, ParamMap *params) |
| |
|
|
Wrappers to hide structure of activity lists (actlists). The activity_list and prev_activity_list are ordered lists stored as arrays whose first element indicates the number of elements in the list, and whose remaining "count" elements (starting at index 1) are the elements in the list (up to one element per neuron in the row, thus N maximum). The gather_activity array is the same type except that the elements are floating-point. Definition at line 255 of file kernel.c. Referenced by LissomMap::compute_responses(), LissomMap::crop_actlist(), and LissomMap::present_inputs(). |
1.3-rc2