Intelligence and intelligent systems has to be able to evolve, self-develop, self-learn continuously in order to reflect the dynamically evolving environment. The concept of Evolving Intelligent Systems was conceived around the turn of the century with the phrase EIS itself coined for the first time in and expanded in. EISs develop their structure, functionality and internal knowledge representation through autonomous learning from data streams generated by the possibly unknown environment and from the system self-monitoring. EISs consider a gradual development of the underlying system structure and differ from evolutionary and genetic algorithms which consider such phenomena as chromosomes crossover, mutation, selection and reproduction, parents and off-springs. The evolutionary fuzzy and neuro systems are sometimes also called "evolving" which leads to some confusion. This was more typical for the first works on this topic in the late 1990s.
Implementations
EISs can be implemented, for example, using neural networks or fuzzy rule-based models. The first neural networks which consider an evolving structure were published in. These were later expanded by N. Kasabov and P. Angelov for the neuro-fuzzy models. P. Angelov introduced the evolving fuzzy rule-based systems as the first mathematical self-learning model that can dynamically evolve its internal structure and is human interpretable and coined the phrase EFS. Contemporarily, the offline incremental approach for learning an EIS, namely, EFuNN, was proprosed by N. Kasabov. P. Angelov, D. Filev, N. Kasabov and O. Cordon organised the first IEEE Symposium on EFSs in 2006. EFSs include a formal learning mechanism to extract it from streaming data. One of the earliest and the most widely cited comprehensive survey on EFSs was done in 2008. Later comprehensive surveys on EFS methods with real applications were done in 2011 and 2016 by E. Lughofer. Other works that contributed further to this area in the following years expanded it to evolving participatory learning, evolving grammar, evolving decision trees, evolving human behaviour modelling, self-calibrating sensors, evolving fuzzy rule-based classifiers, evolving fuzzy controllers, autonomous fault detectors. More recently, the stability of the evolving fuzzy rule-based systems that consist of the structure learning and the fuzzily weighted recursive least square parameter update method has been proven by Rong. Generalized EFS, which allow rules to be arbitrarily rotated in the feature space and thus to improve their data representability, have been proposed in with significant extensions in towards 'smartness' of the rule bases, allowing more interpretability and reducing curse of dimensionality. The generalized rule structure was also successfully used in the context of evolving neuro-fuzzy systems in. Several facets and challenges for achieving more transparent and understandable rule bases in EFS have been discussed by E. Lughofer in. EISs form the theoretical and methodological basis for the Autonomous Learning Machines and autonomous multi-model systems as well as of the Autonomous Learning Systems. Evolving Fuzzy Rule-based classifiers, in particular, is a very powerful new concept that offers much more than simply incremental or online classifiers – it can cope with new classes being added or existing classes being merged. This is much more than just adapting to new data samples being added or classification surfaces being evolved. Fuzzy rule-based classifiers are the methodological basis of a new approach to deep learning that was until now considered as a form of multi-layered neural networks. Deep Learning offers high precision levels surpassing the level of human ability and grabbed the imagination of the researchers, industry and the wider public. However, it has a number of intrinsic constraints and limitations. These include:
The "black box", opaque internal structure which has millions of parameters and involves ad hoc decisions on the number of layers and algorithm parameters.
Most, if not all, of the above limitations can be avoided with the use of the Deep Rule-based Classifiers, which were recently introduced based on ALMMo, while achieving similar or even better performance. The resulting prototype-based IF...THEN...models are fully interpretable and dynamically evolving. They are non-parametric and, therefore, their training is non-iterative and fast. Moreover, they can be trained incrementally/online/in real-time. Another aspect of Evolving Fuzzy Rule-based classifiers has been proposed in, which, in case of multi-class classification problems, achieves the reduction of class imbalance by cascadability into class sub-spaces and an increased flexibility and performance for adding new classes on the fly from streaming samples.
