In the cortex, inhibitory connections appear to be local, and restricted to within a local column. Long range excitatory connections do not appear to target or effectively excite inhibitory interneurons (see the introduction in [39] for further evidence on this point). Feedforward connections between cortical areas thus seem to be excitatory, at least in initial response [14]. Feedback connections between such areas would also be expected to be so in order that the phenomenon of mental imagery can be effective [31]. It is possible that imagery could be achieved by feedback disinhibition, but there would very likely be links in the feedback chain which have excitatory effects, from what is known of cortical microcircuitry. This would especially be true from the lack of evidence of long-range outputs from inhibitory neurons. This lack of long-range inhibition in cortical circuitry would seem to lead to a problem: how can there be anything resembling global competition carried out by such a system? By ``global'', we mean over a whole cortical area, between different cortical areas in a given modality, or even between those processing different modalities. We can summarise the above paragraph by stating that there is little evidence for global competition to be in the cortex.
One of the basic theses of attention is that it involves some sort of filtering process [6], this filtering being regarded as based on a competition between different neural representations of external (or internal) objects or thoughts. If inhibition is only short-ranged, then the usual cortical neural implementation of competition between neural activities by means of a Mexican hat lateral connectivity [2], or by feedback inhibition from all other cells in the relevant network [22] would be biologically unrealistic. How, then, would such long-range competition be achievable in a neurally realistic manner? It is the purpose of this paper to explore possible models for such processing which attempts to keep to neurobiological realism in the large. It explores the possibility that the globally connected sheet of inhibitory neurons related to cortex, namely (a) the nucleus reticularis thalami (hencefort, NRT, a sheet of cells interposed between thalamus and cortex and activated by both of these), and (b) the basal ganglia (more specifically, the striatum and globus pallidus, each composed of mainly inhibitory cells and possessing lateral connections), function in a manner to allow such global competition to take place. The discussion uses a simplified neuron model, and is performed both by mathematical analysis and simulations. The results are shown to support the theses that
There has been a considerable amount of earlier work discussing competitive neural networks and their relation to attention and more general information processing in the cortex. Thus sensory segmentation [51] was achieved by feedback to a set of sensory cells from a common inhibitory interneuron. Selective visual attention was modelled [29] by means of selection based on a winner-takes-all (WTA) net. This latter was suggested as using either suitable inhibition from all other neurons of the net or by means of a hierarchical network with long-range feedback inhibition between layers. A more recent recurrent feedback inhibition was described by Coultrop, Granger & Lynch [12], but was only proposed as allowing for local competition in a single cortical column. Other models for cortical competition have been proposed by other workers (see, for example, references in the previous reference), but nearly all seem to require the existence of long-range cortical inhibition. An important exception is that of Reggia et al. [39], which only uses excitatory lateral connections. However, this model requires modifiable synaptic weights to a given neuron from its nearest neighbours, with the total sum of such weights being a constant; thus the competition for input activity is fought out on the weight space. Until experimental evidence for rapid but totally conserved lateral weights to a given neuron in cortex is discovered, we will explore possibilities which use fixed weights. We conclude that, modulo especially variable weight models, we know of no neurobiologically realistic models for global competition.
The model we propose for the NRT has been explored by us elsewhere [45,46,48] and by La Berge [32] and by La Berge, Carter, and Brown [33]. The former discussions concerned the global features of competition between inputs. The main thesis there was that for suitable connectivity on the thalamus-NRT-cortical complex, inputs cause global wave-like activity on the NRT sheet, which produces competitive processing on the inputs. The waves emphasise the global nature of the competition, where the former can be excited even if the lateral NRT connectivity is not global. This was seen to be particularly effectively achieved by the dendro-dendritic synapses on NRT, which are as prominent there as in the outer plexiform layer of the retina. This analogy was used in [45] and [46] to consider NRT as a negative Laplacian net. The resulting `bunching' or `anti-diffusion' of activity would be expected, and was shown to be, a very flexible substrate for competition. The analysis by La Berge and colleagues [33] supported this, but only local competitive activity was considered, with a WTA response replacing lateral connectivity on the NRT.
The frontal cortex is ``the peak of a hierarchy of anterior neural structures dedicated to the execution of actions'' [19]. In particular, it is dedicated to memory and to motor set, and helps to organise behaviour temporally. To model these processes, it seems necessary also to include sub-cortical structures, in particular basal ganglia and thalamus, all of which are crucially involved in these tasks, as deficits arising from loss or degeneration show in Huntington's chorea and Parkinson's disease. More is being discovered about these systems, from neuroanatomy, neurophysiology and neurochemistry [50]. It has been claimed that ``Frontal lobe has traditionally been long on theory, short on facts'' [19]. That situation now seems to be reversed, especially if by theory one means neural modelling.
In this paper, a simplified neural model of the frontal system is proposed (frontal cortex, basal ganglia and thalamus), which has been termed the ACTION network. This system has already been used to model [49] activity of neurons in the premotor area (PMA) and motor cortex in behaving monkeys [42]. It has also been used as the framework from which to derive neurobiological constraints on recurrent net models of certain temporal tasks, originally analysed by Cohen & Servan-Schreiber [10], but more recently from a neuroanatomical perspective in Monchi & Taylor [36,37]. It has also been described in the more general context of the frontal system in [4].
Previous modelling of the frontal system tended to treat the frontal cortex separately from other sub-cortical centres which are now being realised as important for the total function. Thus temporal sequences [5] only considered the cortical regions explicitly. The same can be said for the neural modelling of motor sequence control by population vector coding [20]. There is some uncertainty of the neural location of the model of temporal tasks in Cohen & Servan-Schreiber [10], but the simplest interpretation of the architecture used there is that it is purely cortical.
There have been independent models of the basal ganglia, especially their use in spatial navigation by functioning as a resistive grid [11,7]. There is involvement of frontal cortex in this model as the net which sets the initial activity on the basal ganglia. It also provides the boundary conditions for the basal ganglia, regarded as a Laplacian net. However it is not clear that the detailed neuroanatomy of the basal ganglia will support such an interpretation of its action, since the lateral connections are, in the main, purely inhibitory. The net would then act as a negative Laplacian net, that is, one with `bunching'of inputs rather that dispersion. Such a principled action is at the basis of the model of the nucleus reticularis (NRT) as part of the cortical attentional system [46,47,48], and may be valid in its extension to the basal ganglia to help preserve identity of throughput. That will be explored elsewhere.
There are interesting models of active memory of Zipser [52] and Zipser et al. [53] and of motor sequence generation of Lukashin et al. [34]. The former model gives an insight into the possible manner in which attractors might be involved in active memory neurons, the latter in the way temporal sequences may be encoded by population vectors. However both models appear to use only cortical neurons. One of the purposes of this paper is to explore the manner in which basal ganglia and thalamus inclusion may aid in understanding how the crucial features of frontal lobe --- active memory, comparison, sequence storage and generation, and attention --- may be supported. The model to be presented is the ACTION network, suggested as a prototype for the action of connected networks in the frontal system. Thus ACTION can be regarded as an extension of the feedback networks of Zipser [53], at the same time supporting further activity than purely active memory. The ACTION network is based on the gross connectivity of the frontal system. It also takes ideas from a number of neurophysiological features of the frontal system, in particular the general disinhibitory action of basal ganglia output on thalamus [26], and the presence of lateral inhibition on the mainly inhibitory (GABAergic) neurons of the striatum and globus pallidus. In combination with the presence of cortico-thalamic feedback loops, one arrives at the basic architecture of the ACTION network. It is this net which will be explored as a supporter of active memory [21]. It is also suggested as the basis of temporal sequence control and motor set [49], of the solution to delayed memory tasks [36,37] and of attention [4]. Because it is constrained by the neurobiology there is a reason for supposing that ACTION avoids the fate of becoming merely a corollary of the universal approximation theorem.
In the Simplified Global ... section, a simplified version of the thalamus-NRT-cortex model is presented, which is analysed mathematically and by simulation in the Modeling the ...section . The ACTION model is presented in the ACTION Network section, and its application to active memory and attention indicated briefly in the Conclusions section.