The MIR Sentence Processing package contains the C source code for the MIR system, as well as a selection of scripts with the relevant training and testing data. MIR has been designed for rapid prototyping of typical architectures used in NLP research such as SRN and SOM which depend heavily on (one or more) lexicons, and can be easily extended to handle any other algorithms the user may be interested in. Because of the emphasis on rapid prototyping, the package is written in C using the TCL/TK libraries. Provided a number of basic commands and widgets, the user can quickly set up networks to test and train a variety of architectures. The use of the TCL scripting language extends the facility with which experiments can be run once an architecture has been designed, as well as manipulating and displaying interesting aspects of the design. TK provides the means to develop a graphical interface for purposes of visualizing not just the network and its components, but its dynamics as well. The TCL/TK extension BLT has proved very useful for this purpose.The current implementation uses the TCL7.6 (tcl7.6p2.tar.gz) and TK4.2 (tk4.2p2.tar.gz) libraries (for more information on the release of TCL7.6/TK4.2, see the Scriptics home page). The software will also work with the BLT2.3 extension (BLT2.3.tar.gz), but its use is optional; visit the BLT site for general information about the BLT extension. An upgrade to TCL/TK8.0 is in the works.
A number of scripts, including sardsrn.tcl and semflip.tcl (see below for references to demos and papers), have been provided to illustrate how typical (as well as not-so-typical) networks can be constructed which use the commands and widgets in the MIR program. Click here for the README, which gives information on how to get, uncompress, and untar the package. (You can also download the software via the tar icon below.)
Demos of these scripts in action can be found at:
- The SARDSRN Sequential Network (sardsrn.tcl)
- Semantic Disambiguation in Sentence Processing (semflip.tcl)
Comments to martym@cs.utexas.edu
Version:
v1.0 10/27/98 martym
The SPEC package contains the C-code and data for training and testing the SPEC system for processing complex sentences with embedded clauses. The full SPEC system consists of an SRN parser, RAAM stack, and feedforward Segmenter networks (the software also supports experiments with a single SRN parser alone). It includes a "real-time" X11(R5) graphics display for visualization and debugging and routines for collecting performance statistics throughout training and testing. For more details, see the SPEC paper .Even though the SPEC package is really one specific sentence processing model, it can serve as a starting point for other experiments with modular connectionist NLP architectures. It includes fairly general code for the SRN, RAAM, and feedforward backprop networks, for the FGREP method for developing distributed representations for words, and also routines for managing the simulation, collecting performance statistics, and visualizing the networks "real-time" on an X11 graphics display.
Comments to discern@cs.utexas.edu.
Versions:
v1.0 8/12/94 risto
v1.1 9/21/94 risto
- The training and testing scheme is now controlled with running, chain, stacknouns, and includeall parameters and command-line options and allows for more varied strategies.
v1.1.1 1/8/97 risto
- fixed color allocation bug (screens with more than 256 colors)
DISCERN is a large, modular neural network system for reading, paraphrasing and answering questions about stereotypical (script-based) stories. The main reference isMiikkulainen, R. (1993). Subsymbolic Natural Language Processing: An Integrated Model of Scripts, Lexicon, and Memory. Cambridge, MA: MIT Press.
A precis of this book and a short summary paper should give you a quick overview of this research. To get an idea what the DISCERN programs are like (without having to first port them), take a look at the on-line demo. It runs remotely on cascais.cs.utexas.edu, with graphics display on your X11 screen.
The DISCERN software consists of four components: 1. PROC for training and testing the backpropagation-based processing modules (for parsing, generation, and question answering), 2. HFM for training and testing the hierarchical feature maps that form the basis for the episodic memory, 3. DISLEX for training and testing the lexicon (lexical and semantic feature maps and associative connections between them), and 4. DISCERN, which is the integrated complete story processing model put together from the final results of the above three programs.
Comments to discern@cs.utexas.edu.
The PROC package contains the C-code and data for training and testing the story processing modules of the DISCERN system. PROC is trained to read and paraphrase script-based stories and answer questions about them. It includes a "real-time" X11(R5) graphics display for visualization and debugging and routines for collecting performance statistics throughout training and testing. For more details, see the Subsymbolic NLP book , or a (slightly old) PROC paper.Even though PROC was developed as part of DISCERN, it itself can serve as a starting point for various experiments in modular connectionist NLP architectures. It includes code for sequential input and sequential output SRN as well as nonrecurrent backpropagation networks, the FGREP mechanism for developing distributed representations for words, and for cloning words with the content+ID technique. There are also general routines for managing the simulation, collecting performance statistics, and visualizing the networks "real-time" on an X11 graphics display.
Comments to discern@cs.utexas.edu.
Versions:
v1.0 9/21/94 risto
v1.0.1 1/8/97 risto
- fixed color allocation bug (screens with more than 256 colors)
The HFM package contains the C-code and data for training and testing the HFM memory organization and hierarchical classification model. HFM consists of a pyramid of feature maps that self-organize according to the hierarchical taxonomy of the data. It was developed as the episodic memory organization for the DISCERN story processing system but could be used in other domains as well. HFM includes a "real-time" X11(R5) graphics display for visualization and debugging and routines for collecting performance statistics throughout training and testing. For more details, see the Subsymbolic NLP book , or a (slightly old) HFM paper.Even though HFM was developed as part of DISCERN, it itself can serve as a starting point for various experiments in connectionist cognitive architectures. It includes code for self-organizing feature maps and general routines for managing the simulation, collecting performance statistics, and visualizing the feature map organization on a "real-time" X11 graphics display.
Comments to discern@cs.utexas.edu.
Versions:
v1.0 9/21/94 risto
v1.0.1 1/8/97 risto
- fixed color allocation bug (screens with more than 256 colors)
This package contains the C-code and data for training and testing the DISLEX model of the lexicon, which is also part of the DISCERN story processing system. DISLEX consists feature maps for lexical symbols and of word meanings, and associative connections between them that translate between symbols and meanings. DISLEX includes a "real-time" X11 graphics display for visualization and debugging and routines for collecting performance statistics throughout training and testing. For more details, see the Subsymbolic NLP book , or a DISLEX paper.Even though DISLEX was developed as part of DISCERN, it itself can serve as a starting point for various experiments in modular connectionist NLP architectures. It includes code for self-organizing feature maps and Hebbian associative connections that form many-to-many mappings between maps. There are also general routines for managing the simulation, collecting performance statistics, and visualizing the feature map organization and associative connections on a "real-time" X11 graphics display.
Comments to discern@cs.utexas.edu.
Versions:
v1.0 10/3/91 risto
- HP Starbase graphics version (called lex)
v2.0 9/21/94 risto
- X11 graphics
- data and simulation management rewritten
v2.0.1 1/8/97 risto
- fixed color allocation bug (screens with more than 256 colors)
v2.0.2 9/27/98 risto
- fixed "within" statistics bug
v2.1 1/3/99 risto
- added the phonological map simulation data
The DISCERN package contains the C-code and data for running the complete, trained DISCERN system for reading, paraphrasing, and answering questions about script-based stories. It was put together from the final results of the PROC, HFM, and DISLEX training programs, as well as code for the Trace Feature Map memory system. The software allows the user to interact with the model in real time through an X11(R5) graphics interface. It also prints detailed log output during processing on the standard output and collects statistics about the overall system performance. For more details, see the Subsymbolic NLP book .Although the DISCERN performance package does not contain any training code, it can serve as a starting point for various demo programs in modular connectionist NLP. It shows how various architectures (SRN and feedforward backpropagation, feature maps, association matrices) can be put together into a single, large model that interacts with the user on-line. There are general routines for managing the simulation, collecting performance statistics, visualizing the system performance and interacting with it through a command interface and the mouse.
Comments to discern@cs.utexas.edu.
Versions:
v1.0 8/9/93 risto
v2.0 8/9/94 risto
- rewrote the X code, cleaned up
v2.1 9/21/94 risto
- added mouse support for examining lexicon and episodic memory
- added options, reorganized the structure somewhat, updated file format
v2.1.1 1/8/97 risto
- fixed color allocation bug (screens with more than 256 colors)
The FGREPNET package contains the C-code and data for training and testing an FGREP network in developing distributed representations for words in the sentence case-role assignment task. It includes a "real-time" X11(R5) graphics display for visualization and debugging and routines for collecting performance statistics throughout training and testing. For more details, see the Subsymbolic NLP book , or a (slightly old) paper on FGREP.This package contains the original FGREP experiment which was based on the data from McClelland and Kawamoto's article in the PDP book. FGREP has been since then used in several other models, including the DISCERN story processing model and the SPEC sentence processing system (both available in this same site). The ID+content technique for extending the vocabulary is not included in this package, but it is part of the DISCERN/PROC software. The idea is that it should be easy to modify and apply this package to developing I/O representations in various other tasks. The software includes general code for the feedforward backprop network, FGREP, simulation management, performance statistics, and X11 graphics display.
Comments to discern@cs.utexas.edu.
Versions:
v1.0 10/3/91 risto
- HP Starbase graphics version (called fgrep)
v2.0 9/21/94 risto
- X11 graphics
- data and simulation management rewritten
v2.0.1 1/8/97 risto
- fixed color allocation bug (screens with more than 256 colors)
The LISSOM package contains the C++ and Scheme source code and examples for training and testing firing-rate LISSOM models, specifically RF-LISSOM, CRF-LISSOM, and HLISSOM. These self-organizing models support detailed simulations of the development and function of the mammalian visual system.
The simulator includes a graphical user interface (GUI), a command language for scripts, and a command-line interface. Sample command files are provided for running a variety of orientation, ocular dominance, motion direction, and face perception simulations. Extensive documentation is also included on disk and via online help at the command line. For more details about LISSOM-based models, see this paper on RF-LISSOM (and others under Visual Cortex and Self-Organization), and the Visual Cortex and Self-Organization research descriptions.
In addition to the supplied sample simulations, the simulator allows you to define arbitrary networks of maps that you can arrange into a hierarchy representing the visual system. Currently-supported map types include input regions (e.g. a Retina), convolving regions (e.g. ON/OFF cell layers), and RF-LISSOM regions (with modifiable afferent and lateral connections.) Environmental input is controlled by a simple but flexible language that allows arbitrary patterns and natural images to be rendered, scaled, rotated, combined, etc. This language makes it possible to use LISSOM for many of your own projects without having to write any new simulator code.
The simulator can also serve as a good starting point for writing a batch-mode neural-network or related simulator. In particular, the code includes independent and general-purpose routines for image creation from matrices, PPM format image input and output, gnuplot plotting, saving in matlab format, polymorphic datatypes, 2D input drawing, streams of inputs from different distributions, convolution kernel (or initial weight) specification, and cortical map measurement, as well as many general-purpose support algorithms and datatypes.
The installation instructions, GUI documentation, command language documentation, and code documentation for the current version are available online.
New! (1/2003): A short LISSOM tutorial is now available.
Comments to jbednar@cs.utexas.edu.
Versions:
v1.0 07/08/1994 sirosh@cs.utexas.edu
- Initial public K&R C release, without RFs.
v2.0 10/28/1998 jbednar@cs.utexas.edu, sirosh@cs.utexas.edu
- Reimplemented in ANSI C supporting RF-LISSOM, interactive prompt, picture generation, and online help.
V2.1 11/09/1998 jbednar@cs.utexas.edu
- ANSI C release with enhanced command language, orientation handling, etc.
v3.0a1 08/21/2000 jbednar@cs.utexas.edu
- Now C++; added input command language; last version with full Cray T3E support.
v3.0b1 04/08/2001 jbednar@cs.utexas.edu
- Added support for multiple maps, arbitrary map sizes, and map scaling.
v3.0 11/25/2001 jbednar@cs.utexas.edu
- Fully released version of 3.0a1 (alpha) and 3.0b1 (beta).
v4.0 01/19/2003 jbednar@cs.utexas.edu
- Added GUI interface.
- Additional sample orientation, ocular dominance, direction, and face simulations.
- Added support for backprojection, transparent input images, and Matlab plot output.
Version 2.1 may still be useful on platforms without GNU C++ or for those who prefer C, and v3.0a1 may still be useful on the Cray T3E. See README.First in the v4.0 release for more details about the differences between each version.
This package is a simulator for the PGLISSOM model of perceptual grouping and self-organization in the visual cortex. The PGLISSOM package is mainly written in C++, but it also contains C, Tcl/Tk, Perl, shell scripts, and octave (matrix language) code for data analysis and visualization. The README file included in the package describes the package in detail.For a formal description of the PGLISSOM model, see Yoonsuck Choe's Ph.D. dissertation.
Comments to choe@tamu.edu.
Versions:
v1.1 3/11/02 yschoe
- updated public release: available for the first time on UTCS nn web site.
The SOFM package contains C- and TK/TCL-code (integrated through SWIG) for the standard feature map algorithm for forming a map of a set of n-dimensional vectors. Its primary use is to allow visualization of the clusters formed by the self-organizing process by labelling the map units with the labels for the vectors.Versions:
v1.0 10/3/91 risto
- HP Starbase graphics version (called fm_nd)
v2.0 9/21/94 risto
- X11 graphics version (called fm)
- data and simulation management rewritten
v2.0.1 1/8/97 risto
- fixed color allocation bug (screens with more than 256 colors)
v3.0 1/22/03 martym
The JavaSANE package contains the source code for the Hierarchical SANE system, based on SANE-C, but rewritten extensively in Java. The JavaSANE version is designed especially to make it possible to apply SANE to new tasks with minimal effort. It is also intended to be platform-independent and reasonably efficient implementation of SANE. Other revisions (from SANE-C) include more documentation and more parsimonious code, so that JavaSANE can serve as a starting point for further research in neuroevolution algorithms as well. Some experimental features from SANE-C have been removed, including local learning and some functions designed to output networks to visualization tools, in an attempt to make the code more streamlined and simpler to apply.SANE is designed as an efficient method for forming neuro-control networks in reinforcement learning tasks. For more details on SANE and some of its applications, see a paper on SANE (and others under Neuroevolution Methods and Applications), or the Neuroevolution Methods and Applications research descriptions.
This package is designed to be an easy starting point for applying JavaSANE to a new domain. First, change the values in NUM_INPUTS and NUM_OUTPUTS in Config.java to the appropriate number of inputs and outputs for your network. Rewrite the function Evaluate_net in Domain.java, which takes a neural net as a parameter and returns its "fitness value," where fitness is defined as ability to perform a task in the domain being implemented. Finally, run JavaSANE: It will try to evolve networks that optimize the value returned by your function. The software will take care of creating and evolving the network entirely.
Comments to cyndy@cyc.com or risto@cs.utexas.edu.
Versions:
v1.0 12/15/98 cyndy
v1.1 1/15/99 cyndy
- Implementation of Domain.java
- Expansion and rearrangement of Config.java
- Vastly improved ease-of-use for new domain implementation
v1.2 8/22/00 Alex Lubberts
- Fixed a bug in SANE_EA.java that caused some neurons to receive too high fitness value
- A few other minor bug fixes.
The SANE-C package contains the source code for the Hierarchical SANE system, written in C. This package has been rewritten in Java and extensively revised as the JavaSANE package; it is recommended that JavaSANE is used instead of SANE-C, which is mostly research code.SANE is designed as an efficient method for forming neuro-control networks in reinforcement learning tasks. For more details on SANE and some of its applications, see a paper on SANE (and others under Neuroevolution Methods and Applications), or the Neuroevolution Methods and Applications research descriptions.
This package is designed to be an easy starting point for applying SANE to a new domain. First, decide and set the number of inputs and outputs for the network in sane.h. You should also set the number of hidden neurons and the population sizes to your liking. Then, simply write the function eval_net() in skeleton-main.c. eval_net is a function that takes a neural network as input and returns it's fitness. The current eval_net is just a stub that returns 1.0 for all networks. Once eval_net is defined, you can run SANE and it will try to find networks that optimize the value returned by your eval_net function.
Comments to moriarty@isi.edu.
Versions:
v1.0 5/9/96 moriarty
v2.0 5/1/97 moriarty
- Population seeding functions
- Local learning functions
- More cohesive neuron/network data structures
- Floating point chromosomes instead of bitwise representations
v2.1 8/21/2000 by Bram Stolk (bram@sara.nl)
- Converted code from K&R to ANSI C
- cleaned up warnings etc.
This simulator contains the code used to compare (neuron-level) SANE to one- and two-layer adaptive heuristic critics in the inverted pendulum problem. The code for the single-layer Adaptive Heuristic Critic was adapted from Rich Sutton's pole-balancing simulator. The code for the two-layer AHC was adapted from Charles Anderson's pole simulator.Codes for all three methods are included in this package, together with sample runs for each. The simulators are currently set up to balance a pole on a cart starting from random positions with random initial velocities. To always start from a centered pole and cart with no initial velocities, set RANDOM_START to 0. For more details, see the paper on pole-balancing comparisons.
Comments to moriarty@isi.edu.
Versions:
v1.0 2/21/95 moriarty
The ESP package contains the source code for the Enforced Sup-Populations system written in C++. ESP is an extension to the SANE algorithm that segregates the neurons into subpopulations, one for each hidden unit in the networks being evolved. This allows neuron specializations to form more rapidly through constrained mating and enables the system to evolve recurrent networks.This package applies ESP to a non-Markov version of the double pole balancing problem. This is a difficult control task that requires short-term memory. For more details on ESP and/or the double pole system, see this paper on ESP or others found under Neuroevolution Methods and Applications.
Comments to inaki@cs.utexas.edu.
Versions:
v3.0 8/5/02 inaki (Documentation)
The ESP package contains the source code for the Enforced Sup-Populations system written in Java. This package is a nearly a direct port of the ESP C++ package available on this page. It should be easily portable although C++ is approximately 2.5 times faster.Comments to oursland@cs.utexas.edu.
Versions:
v1.0 2/19/02 oursland
The NEAT package contains source code implementing the NeuroEvolution of Augmenting Topologies method. The source code is written in C++. NEAT is a method for evolving speciated neural networks of arbitrary structures and sizes. NEAT leverages the evolution of structure to make neuroevolution more efficient. For more information on NEAT, see our paper on NEAT or the Neuroevolution Methods page.The package includes implementations of experiments for XOR, single pole balancing, and both Markovian and non-Markovian double pole balancing. A 17-page postscript documentation file is included to make getting started easier.
For answers to common questions, refer to our FAQ .
Please contact kstanley@cs.utexas.edu for comments, including ideas or plans for expanding the open source software.
Versions:
v1.0 8/16/01 kstanley
v1.1 7/14/02 kstanley
-removed extraneous files from package
-fixed array bound error
-made text output default on instead of off
The JNEAT package contains Java source code for the NeuroEvolution of Augmenting Topologies method (see the NEAT package above). It includes a nice GUI (see screenshots ), and implementations of experiments for XOR and 3-bit parity.
For answers to common questions, refer to our FAQ .
JNEAT was written by Ugo Vierucci based on the original C++ package by Kenneth Stanley. Please direct bug reports to ugo.vierucci@virgilio.it. Please contact kstanley@cs.utexas.edu for other comments, including ideas or plans for expanding the open source software.
Versions:
v1.2 6/24/02 vierucci
The Windows NEAT package contains C++ source code for the NeuroEvolution of Augmenting Topologies method (see the NEAT package above). It includes an animated demo task and a precompiled executable that runs under Windows. The included task is a minesweeper problem in which sweepers have to cover as much area as possible in a finite amount of time (see screenshot ).
For answers to common questions, refer to our FAQ .
Windows NEAT was written by Mat Buckland, based on the NEAT method by Kenneth Stanley. Please direct bug reports to fup@btinternet.com. Please contact kstanley@cs.utexas.edu for other comments, including ideas or plans for expanding the open source software.
Versions:
v1.0 9/7/02 buckland
Windows ZIP FILE
The TEAM package contains C++ implimentations of both EuA (The Eugenic Algorithm) and TEAM (The Eugenic Algorithm with Modeling), two algorithms for evolving binary chromosomes. For details of the TEAM algorithm, please read our paper on TEAM.Included are several sample experiments, as well as instructions for writing new chromosome evaluation functions.
Please direct questions, comments, etc. to malden@cs.utexas.edu.
Versions:
v1.0 7/23/02 malden
The SignalSim Spiking Neuron package is a Tcl/Tk GUI built on top of an event-driven simulator of an interconnected network of spiking neurons.The current implementation uses the TCL8.3 and TK8.3 libraries (for more information on the release of TCL/TK8.3, see the Scriptics home page). Comments to martym@cs.utexas.edu
Versions:
v1.0 9/14/90 risto
v2.0 12/03/02 martym