4.2 Training parameters

The model consisted of an array of 192 × 192 neurons, and a retina of 24 × 24 ganglion cells. The circular anatomical receptive field of each neuron was centered in the portion of the retina corresponding to the location of the neuron in the cortex. The RF consisted of random-strength connections to all ganglion cells less than 6 units away from the RF center. For example, the neuron at the center of the cortex was connected to the ganglion cells inside a circle of radius 6 at the center of the retina. The top left neuron was connected to the top left retinal ganglion, and to the other ganglion cells in the top left corner. Sample initial receptive fields are shown in figure 4.2.

The cortex was self-organized for 30,000 iterations on oriented Gaussians which each had a major axis of half-width a=7.5 units and a minor axis of half-width b=1.5 . The initial lateral excitation radius was 19 and was gradually decreased to 1 . The lateral inhibitory radius of each neuron was 47 , and inhibitory connections whose strength was below 0.00025 were pruned away at 30,000 iterations. The lateral inhibitory connections were preset to a Gaussian profile with $\sigma=100$, and the lateral excitatory connections to a Gaussian with $\sigma=15$. The lateral excitation $\gamma_e$ and inhibition strength $\gamma_i$ were both 0.9 . The learning rate $\alpha_{\rm A}$ was decreased from 0.007 to 0.0015 , $\alpha_{\rm E}$ from 0.002 to 0.001 and $\alpha_{\rm I}$ was a constant 0.00025 . The lower and upper thresholds of the sigmoid was increased from 0.1 to 0.24 and from 0.65 to 0.88 , respectively. The number of iterations for which the lateral connections were allowed to settle at each training iteration was initially 9 , and was gradually increased to 13 over the course of training.

These parameters were chosen by Sirosh (1995) in order to develop a biologically realistic orientation map, prior to any of the experiments done on tilt illusions or aftereffects for this thesis. Small variations of these parameters produce roughly equivalent results. The training took 8 hours on 64 processors of a Cray T3D at the Pittsburgh Supercomputing Center. The model requires more than three gigabytes of physical memory to represent the more than 400 million connections in this small section of the cortex.

Although there is significant order in the model even before self-organization, with training the RFs will sharpen into smooth profiles selective for orientation, the topographical organization will be refined, and the lateral connections will become patchy and focused. The initial order represents part of the genetically-determined development of the visual cortex, and the self-organization to be described in the following sections represents the activity-dependent developmental processes, as outlined in section 3.7.