Adaptive resonance theory


Adaptive resonance theory is a theory developed by Stephen Grossberg and Gail Carpenter on aspects of how the brain processes information. It describes a number of neural network models which use supervised and unsupervised learning methods, and address problems such as pattern recognition and prediction.
The primary intuition behind the ART model is that object identification and recognition generally occur as a result of the interaction of 'top-down' observer expectations with 'bottom-up' sensory information. The model postulates that 'top-down' expectations take the form of a memory template or prototype that is then compared with the actual features of an object as detected by the senses. This comparison gives rise to a measure of category belongingness. As long as this difference between sensation and expectation does not exceed a set threshold called the 'vigilance parameter', the sensed object will be considered a member of the expected class. The system thus offers a solution to the 'plasticity/stability' problem, i.e. the problem of acquiring new knowledge without disrupting existing knowledge that is also called incremental learning.

Learning model

The basic ART system is an unsupervised learning model. It typically consists of a comparison field and a recognition field composed of neurons, a vigilance parameter, and a reset module.
There are two basic methods of training ART-based neural networks: slow and fast. In the slow learning method, the degree of training of the recognition neuron's weights towards the input vector is calculated to continuous values with differential equations and is thus dependent on the length of time the input vector is presented. With fast learning, algebraic equations are used to calculate degree of weight adjustments to be made, and binary values are used. While fast learning is effective and efficient for a variety of tasks, the slow learning method is more biologically plausible and can be used with continuous-time networks.

Types

ART 1 is the simplest variety of ART networks, accepting only binary inputs.
ART 2 extends network capabilities to support continuous inputs.
ART 2-A is a streamlined form of ART-2 with a drastically accelerated runtime, and with qualitative results being only rarely inferior to the full ART-2 implementation.
ART 3 builds on ART-2 by simulating rudimentary neurotransmitter regulation of synaptic activity by incorporating simulated sodium and calcium ion concentrations into the system's equations, which results in a more physiologically realistic means of partially inhibiting categories that trigger mismatch resets.
ARTMAP also known as Predictive ART, combines two slightly modified ART-1 or ART-2 units into a supervised learning structure where the first unit takes the input data and the second unit takes the correct output data, then used to make the minimum possible adjustment of the vigilance parameter in the first unit in order to make the correct classification.
Fuzzy ART implements fuzzy logic into ART's pattern recognition, thus enhancing generalizability. An optional feature of fuzzy ART is complement coding, a means of incorporating the absence of features into pattern classifications, which goes a long way towards preventing inefficient and unnecessary category proliferation. The applied similarity measures are based on the L1 norm. Fuzzy ART is known to be very sensitive to noise.
Fuzzy ARTMAP is merely ARTMAP using fuzzy ART units, resulting in a corresponding increase in efficacy.
Simplified Fuzzy ARTMAP constitutes a strongly simplified variant of fuzzy ARTMAP dedicated to classification tasks.
Gaussian ART and Gaussian ARTMAP use Gaussian activation functions and computations based on probability theory. Therefore, they have some similarity with Gaussian mixture models. In comparison to fuzzy ART and fuzzy ARTMAP, they are less sensitive to noise. But the stability of learnt representations is reduced which may lead to category proliferation in open-ended learning tasks.
Fusion ART and related networks extend ART and ARTMAP to multiple pattern channels. They support several learning paradigms.
TopoART combines fuzzy ART with topology learning networks such as the growing neural gas. Furthermore, it adds a noise reduction mechanism. There are several derived neural networks which extend TopoART to further learning paradigms.
Hypersphere ART and Hypersphere ARTMAP are closely related to fuzzy ART and fuzzy ARTMAP, respectively. But as they use a different type of category representation, they do not require their input to be normalised to the interval . They apply similarity measures based on the L2 norm.
LAPART The Laterally Primed Adaptive Resonance Theory neural networks couple two Fuzzy ART algorithms to create a mechanism for making predictions based on learned associations. The coupling of the two Fuzzy ARTs has a unique stability that allows the system to converge rapidly towards a clear solution. Additionally, it can perform logical inference and supervised learning similar to fuzzy ARTMAP.

Criticism

It has been noted that results of Fuzzy ART and ART 1 depend critically upon the order in which the training data are processed. The effect can be reduced to some extent by using a slower learning rate, but is present regardless of the size of the input data set. Hence Fuzzy ART and ART 1 estimates do not possess the statistical property of consistency. This problem can be considered as a side effect of the respective mechanisms ensuring stable learning in both networks.
More advanced ART networks such as TopoART and Hypersphere TopoART that summarise categories to clusters may solve this problem as the shapes of the clusters do not depend on the order of creation of the associated categories.