A learning algorithm for boltzmann machines

The computational power of massively parallel networks of simple processing elements resides in the communication bandwidth provided by the hardware connections between elements. These connections can allow a significant fraction of the knowledge of the system to be applied to an instance of a problem in a very short time. One kind of computation for which massively parallel networks appear to be well suited is large constraint satisfaction searches, but to use the connections efficiently two conditions must be met: First, a search technique that is suitable for parallel networks must be found. Second, there must be some way of choosing internal representations which allow the preexisting hardware connections to be used efficiently for encoding the constraints in the domain being searched. We describe a general parallel search method, based on statistical mechanics, and we show how it leads to a general learning rule for modifying the connection strengths so as to incorporate knowledge about a task domain in an efficient way. We describe some simple examples in which the learning algorithm creates internal representations that are demonstrably the most efficient way of using the preexisting connectivity structure.     

Learning-forgetting independence, unidimensional memory models, and feature models: comment on Bogartz (1990)

In his recent articles, Bogartz offered a definition of what it means for forgetting rate to be independent of degree of original learning. He showed that, given this definition, independence is confirmed by extant data. Bogartz also criticized Loftus's (1985b) proposed method for testing independence. In this commentary, we counter Bogartz's criticisms and then offer two observations. First, we show that Loftus's horizontal-parallelism test distinguishes between two interesting class of memory models: unidimensional models wherein the memory system's state can be specified by a single number and multidimensional models wherein at least two numbers are required to specify the memory system's state. Independence by Loftus's definition is implied by a unidimensional model. Bogartz's definition, in contrast, is consistent with either model. Second, to better understand the constraints on memory mechanisms dictated by the mathematics of the models under consideration, we develop a simple but general feature model of learning and forgetting. We demonstrate what constraints must be placed on this model to make learning and forgetting rate independent by Loftus's and by Bogartz's definitions.