The Temporal Convolution Machine (TCM) is a neural architecture for learning temporal sequences that generalizes the Temporal Restricted Boltzmann Machine (TRBM). A convolution function is used to provide a trainable envelope of time sensitivity in the bias terms. Gaussian and multi-Gaussian envelopes with trainable means and variances are evaluated as particular instances of the TCM architecture. First, Gaussian and multi-Gaussian TCMs are shown to learn a class of multi-modal distributions over synthetic binary spatiotemporal data better than comparable TRBM models. Second, these networks are trained to recall digitized versions of baroque sonatas. In this task, a multi-Gaussian TCM performs effective sequence mapping when the input sequence is partially hidden. The TCM is therefore a promising approach to learning more complex temporal data than was previously possible.

PDF

Technical Report AI-09-04, Department of Computer Sciences, the University of Texas at Austin, 2009.

@techreport{lockett:aitr09-04, title={Temporal Convolution Machines for Sequence Learning}, author={Alan J Lockett and Risto Miikkulainen}, number={AI-09-04}, institution={Department of Computer Sciences, the University of Texas at Austin}, url="http://nn.cs.utexas.edu/?lockett:aitr09-04", year={2009} }