Besides the specific topics for future study discussed so far (including ocular transfer, saturation, and dark decay), this research has opened up several areas that can now be studied computationally. These areas include running simulations that more closely parallel psychophysical experiments, testing low-level phenomena closely related to the tilt aftereffect, and examining illusions and aftereffects in other modalities. This section will sketch possible avenues for investigation of these topics.
In addition to the tilt aftereffects studied in this thesis, the RF-LISSOM model should exhibit direct tilt illusions between simultaneous spatially-separated stimuli (Calvert and Harris, 1988; Carpenter and Blakemore, 1973; Gilbert and Wiesel, 1990; O'Toole, 1979; Smith and Over, 1977; Wenderoth and Johnstone, 1988; Westheimer, 1990). The stimuli would interact with each other as the lateral interactions settle, inhibiting the feature detectors tuned to orientations between those of the two patterns. This would drive the perceived orientation of each pattern away from that of the other.
However, this effect cannot yet be tested with the orientation map used for the other experiments in this thesis. The training inputs to the map were single oriented Gaussians, so correlations occurred only between local areas along the orientation of the Gaussian. Thus lateral connections develop along a single orientation: the orientation preference of that neuron (section 4.3). So tilt illusions would only occur for overlapping stimuli centered around the same location on the retina, because only those stimuli would have significant activation of feature-detectors linked by lateral connections.
Because the retinal representations of such lines overlap, the cortical responses will overlap as well, so it would not be possible to quantify the effect using the techniques in section 4.5. That is, the vector averaging procedure only allows a single perceived orientation to be calculated per local area of the retina. In contrast, the cortex appears to be able to perceive multiple overlapping orientations separately (as observed in humans, Blakemore et al. 1970; Carpenter and Blakemore 1973).
The segmentation mechanisms of the spiking neuron version of RF-LISSOM (Choe and Miikkulainen, 1996; Miikkulainen et al., 1997) may provide a way to separate the representation of each line (Blakemore et al., 1970; Carpenter and Blakemore, 1973). In this model, the set of activated units can divide itself into overlapping populations of neurons, each population firing out of phase with the others (von der Malsburg, 1973). For two overlapping lines, the active units will form into two groups consisting of neurons firing in synchrony (as mentioned in section 6.2.3. The perceived orientation calculation from section 4.5 could then be performed on each group separately to yield a perceived orientation of each line. These perceived orientations could be compared to the orientation perceived when only one line is present, to determine how large a tilt illusion is present.
A simpler test procedure would be to self-organize an orientation map using inputs such as sinusoidal gratings that have longer-range correlations between similar orientations. Such patterns would represent objects with parallel lines or edges, which are very common in the visual environment. Long-range connections would then develop between widely separated orientation detectors in parallel directions, in addition to the relatively local connections now present. A version of the RF-LISSOM model trained on such patterns should demonstrate direct tilt illusions through lateral inhibition between two separated stimuli, as described above. Although such experiments require at least twice as large a cortex and retina as that used for this thesis, they should become practical in the near future.
Although indirect effects have been found for the TI much like those for the TAE, it is not yet known whether they will be found in the RF-LISSOM model. The simultaneous indirect effect may depend upon facilitation of weakly activated units by units at distant orientations. This facilitation would be mediated by lateral connections whose effective sign would depend upon local contrast, the extension to RF-LISSOM proposed in section 3.6. The indirect TI may also depend upon factors not yet considered, so it should be an interesting topic for research.
Assumption 2 is clearly true for the model, since learning can be turned off entirely so that presentation of test patterns does not have any lasting effect. However, it is clearly not true in general for humans, because adaptation cannot be measured psychophysically without presenting a test pattern. The cortex will adapt to all patterns presented, not just those the experimenter has labeled ``adaptation'' stimuli. Only in a computational model such as RF-LISSOM can the TAE be evaluated without any side effects, although various presentation techniques have been devised to limit the residual effects of the test pattern in psychophysical experiments.
The contribution of those side effects could be evaluated in RF-LISSOM by duplicating the psychophysical experiments in greater detail. The RF-LISSOM learning rate would remain at the same level throughout the experiment, thus causing adaptation to every stimulus presented. A particular experiment would be chosen for replication, e.g. Mitchell and Muir (1976). The reports of this experiment would need to indicate how long the test stimulus was presented, relative to how long the training stimulus was presented, so that these parameters could be duplicated in the model. The experiment would then be recreated as follows.
First, a fixed testing stimulus would be chosen and presented for the period of time used in the psychophysical experiment, in order to compute a baseline for the perceived orientation of the test line. Next, the model would adapt for a fixed amount of time to another line at a particular orientation. The testing stimulus would then be presented again, with learning still on, and the magnitude of the TAE would be measured. Finally, the cortex would be returned to equilibrium in order to account for the decay of the TAE, either by resetting all weights to their initial values, or by presenting randomly oriented stimuli long enough to erase the effects of adaptation. This procedure would be repeated for each point on the angular function of the TAE from figure 5.2.
The curve from this procedure could then be compared to one where learning is turned off for the testing pattern, in order to show the effect of the presentation of the test pattern. In humans, large aftereffects (greater than 5°) have been found for very short test presentations (Wolfe, 1984). With the tests above it might be possible to decide whether different mechanisms are operating over these short time scales (Harris and Calvert, 1989; Wolfe, 1984), or whether the effect of the test pattern is sufficient to explain the larger aftereffect found in those circumstances. Computing a single point on the TAE curve with the above procedure takes more computing time than it took to create the entire set of curves in figure 5.9, but limited tests of this nature should be computationally feasible.
However, some of the effects noted by Harris and Calvert (1989) appear to suggest influences of the contrast-dependent lateral connections proposed in section 3.6. For some combinations of inputs, the TAE appears much weaker at low adapting contrasts than at high contrasts. This could be a result of having long-range lateral connections that are excitatory at very low contrasts. For low-contrast inputs, the inhibitory lateral interactions would be negligible, which should result in a much smaller TAE as observed. Experiments with a version of RF-LISSOM extended with such connections may help determine which mechanisms can account for these effects.
Most studies of the tilt aftereffect are performed using stimuli at a fixed contrast, and require the subject to judge the orientation of the stimuli by some means. An alternative way to study the effects of adaptation is to measure the detection threshold for each orientation before and after adaptation to a particular stimulus. The results from such procedures are generally similar to the results for the TAE measurements: adapting to one orientation masks stimuli at nearby orientations, effectively raising the detection threshold for those orientations and lowering them for somewhat more distant orientations (Blakemore and Nachmias, 1971; Over et al., 1972; Ross et al., 1993; Smith and Over, 1977; Vidyasagar, 1990; Virsu and Taskinen, 1974; Waugh et al., 1993).
However, there are numerous differences between masking and the TAE. For instance, there doesn't appear to be any facilitation observed for very distant orientations, yet the TAE shows an indirect effect for those orientations (Over et al., 1972). In addition, masking persists in the presence of inhibitory receptor blockade (Vidyasagar, 1990) that would be expected to prevent the TAE (see section 6.3). The differences may provide clues about how the contrast-dependent lateral interactions are implemented. In the masking paradigm, the stimulus used during adaptation is high-contrast, while that used to test the detection threshold is at a very low contrast, by definition. Thus this test condition should show how the adaptation of the lateral inhibitory connections affects the long-range lateral excitatory connections. The evidence suggests that the excitatory connections decrease in strength with adaptation, but further study would be needed to explain how or why that occurs.
Besides experiments with tilt adaptation, certain other psychophysical tests also appear to give information about learning processes within the primary visual cortex. For instance, Fahle et al. (1995) and Weiss et al. (1993) found that performance in hyperacuity tasks, such as deciding whether two lines of same orientation are separated by a small perpendicular offset, improves with practice. The improvement occurs even without verbal or other feedback indicating whether each judgment is correct. The effect is specific to position and orientation, but transfers between eyes to some degree. This is thought to indicate that at least some part of the effect arises in V1, since V1 is the first stage in the visual pathway where binocular inputs are combined.
Shiu and Pashler (1992) reported similar results for orientation discrimination tasks, although they found that the effect also depends on cognitive factors. The ability to distinguish the orientations of two similarly oriented lines only improved with practice if the subjects were directed to pay attention to the orientation. However, the effect was specific to the location on the retina on which examples had been presented, so it did not consist of some more effective deliberate strategy that the subject learns during the experiment. This suggests that attentional mechanisms may activate circuitry in V1 (or other early visual areas) that regulates plasticity.
The RF-LISSOM model should be able to account for such psychophysical learning phenomena. The active feature detectors and lateral connections between them would adapt during repeated presentations. Over time, this would expand the area of the cortical feature map responding to those features. This would result in representation and discrimination of smaller differences. Since the orientation discrimination testing paradigm is simple yet shows clear attentional effects, it might also form a good testbed for extensions of RF-LISSOM that include feedback from higher cortical levels. Such studies might help clarify how and when adaptation occurs in the early visual system.
Aftereffects appear to be a nearly universal feature of cortical sensory processing (Barlow, 1990). Many visual aftereffects similar to the TAE have been documented in humans, including aftereffects of curvature, motion, spatial frequency, size, position, and color (Barlow, 1990; Howard and Templeton, 1966; Wolfe, 1984). In all of these, the cortex appears to adapt to a long-lasting stimulus, changing the perceived value of subsequent stimuli. For instance, after prolonged viewing of a moving stimulus, stationary stimuli appear to be moving in the opposite direction (the movement aftereffect, also known as the waterfall illusion.)
Since topographically-organized detectors for most of these features have been found, RF-LISSOM is expected to be able to account for their aftereffects by the same process of decorrelation mediated by self-organizing lateral connections. The current RF-LISSOM model is clearly suitable for investigating some of these aftereffects, such as those of spatial frequency, size, and position. Cortical maps for these dimensions have already been demonstrated in RF-LISSOM, so these maps can be tested for aftereffects using techniques similar to those used in this thesis. Development of maps for the other visual dimensions has not yet been modeled in RF-LISSOM, so testing their aftereffects will have to wait until the maps have been studied.
Analogous aftereffects have also been found for other modalities such as hearing, touch, muscle positioning, and posture (Howard and Templeton, 1966, p.84-85,p.162-163). For instance, hearing a sound in one location can influence the perceived location of later sounds. After adaptation, sounds presented in nearby locations appear to be farther away than they actually are, and the effect appears to peak at a certain distance, much like the direct TAE. Aftereffects might also be present for taste and smell, though these parameters are relatively difficult to control in an experimental setting, and they have not been tested as extensively as vision, hearing, or touch.
If development in these areas can be modeled with RF-LISSOM, as is expected, aftereffects will be present in the behavior of the organized maps. Finding that the aftereffects found in the model match those observed for each modality would indicate that the same decorrelating processes studied for vision in RF-LISSOM also apply to other types of perception, both during development and in the adult. It would thus strongly suggest that similar computations are being performed in areas of the cortex performing very different tasks. Combined with studies of the tilt aftereffect at different levels in the visual hierarchies, it would represent evidence that processes like those in RF-LISSOM are a ubiquitous feature of the cortex. Thus these simple decorrelating principles may account for a large part of the apparent complexity of the cortex. Achieving a unified explanation of such disparate phenomena would represent a significant advance in our understanding of the functioning of the brain.