Introduction

In school we learned many things: e.g. vocabulary, grammar, geography, solving mathematical equations, and coordinating movements in sports. These are very different things which involve declarative knowledge as well as procedural knowledge or skills in principally all fields. We are used to subsume these various processes of obtaining this knowledge and skills under the single word “learning”. And, we learned that learning is important. Why is it important to a living organism?

Learning is a crucial capability if the effective environment cannot be foreseen in all relevant details, either due to complexity, or due to the nonstationarity of the environment. The mechanisms of learning allow nature to create and re-produce organisms or systems which can evolve — with respect to the later given environment —optimized behavior.

This is a fascinating mechanism, which also has very attractive technical perspectives. Today many technical appliances and systems are standardized and cost-efficient mass products. As long as they are non-adaptable, they require the environment and its users to comply to the given standard. Using learning mechanisms, advanced technical systems can adapt to the different given needs, and locally reach a satisfying level of helpful performance.

Of course, the mechanisms of learning are very old. It took until the end of the last century, when first important aspects were elucidated. A major discovery was made in the context of physiological studies of animal digestion: Ivan Pavlov fed dogs and found that the inborn (“unconditional”) salivation reflex upon the taste of meat can become accompanied by a conditioned reflex triggered by other stimuli. For example, when a bell was rung always before the dog has been fed, the response salivation became associated to the new stimulus, the acoustic signal. This fundamental form of associative learning has become known under the name classical conditioning. In the beginning of this century it was debated whether the conditioning reflex in Pavlov's dogs was a stimulus–response (S-R) or a stimulus–stimulus (S-S) association between the perceptual stimuli, here taste and sound. Later it became apparent that at the level of the nervous system this distinction fades away, since both cases refer to associations between neural representations.

The fine structure of the nervous system could be investigated after staining techniques for brain tissue had become established (Golgi and Ramón y Cajal). They revealed that neurons are highly interconnected to other neurons by their tree-like extremities, the dendrites and axons (comparable to input and output structures). D.O. Hebb (1949) postulated that the synaptic junction from neuron A to neuron B was strengthened each time A was activated simultaneously, or shortly before B. Hebb's rule explained the conditional learning on a qualitative level and influenced many other, mathematically formulated learning models since. The most prominent ones are probably the perceptron, the Hopfield model and the Kohonen map. They are, among other neural network approaches, characterized in chapter 3. It discusses learning from the standpoint of an approximation problem. How to find an efficient mapping which solves the desired learning task? Chapter 3 explains Kohonen's “Self-Organizing Map” procedure and techniques to improve the learning of continuous, highdimensional output mappings.

The appearance and the growing availability of computers became a further major influence on the understanding of learning aspects. Several main reasons can be identified:

First, the computer allowed to isolate the mechanisms of learning from the wet, biological substrate. This enabled the testing and developing of learning algorithms in simulation.

Second, the computer helped to carry out and evaluate neuro-physiological, psychophysical, and cognitive experiments, which revealed many more details about information processing in the biological world.

Third, the computer facilitated bringing the principles of learning to technical applications. This contributed to attract even more interest and opened important resources. Resources which set up a broad interdisci plinary field of researchers from physiology, neuro-biology, cognitive and computer science. Physics contributed methods to deal with systems constituted by an extremely large number of interacting elements, like in a ferromagnet. Since the human brain contains of about  neurons with  interconnections and shows a — to a certain extent — homogeneous structure, stochastic physics (in particular the Hopfield model) also enlarged the views of neuroscience.

Beyond the phenomenon of “learning”, the rapidly increasing achievements that became possible by the computer also forced us to re-think about the before unproblematic phenomena “machine” and “intelligence”. Our ideas about the notions “body” and “mind” became enriched by the relation to the dualism of “hardware” and “software”. With the appearance of the computer, a new modeling paradigm came into the foreground and led to the research field of artificial intelligence. It takes the digital computer as a prototype and tries to model mental functions as processes, which manipulate symbols following logical rules – here fully decoupled from any biological substrate. Goal is the development of algorithms which emulate cognitive functions, especially human intelligence. Prominent examples are chess, or solving algebraic equations, both of which require of humans considerable mental effort.

In particular the call for practical applications revealed the limitations of traditional computer hardware and software concepts. Remarkably, traditional computer systems solve tasks, which are distinctively hard for humans, but fail to solve tasks, which appear “effortless” in our daily life, e.g. listening, watching, talking, walking in the forest, or steering a car. This appears related to the fundamental differences in the information processing architectures of brains and computers, and caused the renaissance of the field of connectionist research. Based on the von-Neumannarchitecture, today computers usually employ one, or a small number of central processors, working with high speed, and following a sequential program. Nevertheless, the tremendous growth in availability of costefficiency computing power enables to conveniently investigate also parallel computation strategies in simulation on sequential computers.

Often learning mechanisms are explored in computer simulations, but studying learning in a complex environment has severe limitations - when it comes to action. As soon as learning involves responses, acting on, or inter-acting with the environment, simulation becomes too easily unrealistic. The solution, as seen by many researchers is, that “learning must meet the real world”. Of course, simulation can be a helpful technique, but needs realistic counter-checks in real-world experiments. Here, the field of robotics plays an important role.

The word “robot” is young. It was coined 1935 by the playwriter Karl Capek and has its roots in the Czech word for “forced labor”. The first modern industrial robots are even younger: the “Unimates” were developed by Joe Engelberger in the early 60's. What is a robot? A robot is a mechanism, which is able to move in a given environment. The main difference to an ordinary machine is, that a robot is more versatile and multi-functional, and it can be programmed, or commanded to perform functions normally ascribed to humans. Its mechanical structure is driven by actuators which are governed by some controller according to an intended task. Sensors deliver the required feed-back in order to adjust the current trajectory to the commanded motion and task.

Robot tasks can be specified in various ways: e.g. with respect to a certain reference coordinate system, or in terms of desired proximities, or forces, etc. However, the robot is governed by its own actuator variables. This makes the availability of precise mappings from different sensory variables, physical, motor, and actuator values a crucial issue. Often these sensorimotor mappings are highly non-linear and sometimes very hard to derive analytically. Furthermore, they may change in time, i.e. drift by wear-and-tear or due to unintended collisions. The effective learning and adaption of the sensorimotor mappings are of particular importance when a precise model is lacking or it is difficult or costly to recalibrate the robot, e.g. since it may be remotely deployed.

Chapter 2 describes work done for establishing a hardware infrastructure and experimental platform that is suitable for carrying out experiments needed to develop and test robot learning algorithms. Such a laboratory comprises many different components required for advanced, sensorbased robotics. Our main actuated mechanical structures are an industrial manipulator, and a hydraulically driven robot hand. The perception side has been enlarged by various sensory equipment. In addition, a variety of hardware and software structures are required for command and control purposes, in order to make a robot system useful.

The reality of working with real robots has several effects:

It enlarges the field of problems and relevant disciplines, and includes also material, engineering, control, and communication sciences. The time for gathering training data becomes a major issue. This includes also the time for preparing the learning set-up. In principle, the learning solution competes with the conventional solution developed by a human analyzing the system. The faced complexity draws attention also towards the efficient structuring of re-usable building blocks in general, and in particular for learning.

And finally, it makes also technically inclined people appreciate that the complexity of biological organisms requires a rather long time of adolescence for good reasons; Many learning algorithms exhibit stochastic, iterative adaptation and require a large number of training steps until the learned mapping is reliable. This property can also be found in the biological brain. There is evidence, that learned associations are gradually enhanced by repetition, and the performance is improved by practice - even when they are learned insightfully. The stimulus-sampling theory explains the slow learning by the complexity and variations of environment (context) stimuli.

Since the environment is always changing to a certain extent, many trials are required before a response is associated with a relatively complete set of context stimuli. But there exits also other, rapid forms of associative learning, e.g. “oneshot learning”. This can occur by insight, or triggered by a particularly strong impression, by an exceptional event or circumstances. Another form is “imprinting”, which is characterized by a sensitive period, within which learning takes place. The timing can be even genetically programmed. A remarkable example was discovered by Konrad Lorenz, when he studied the behavior of chicks and mallard ducklings. He found, that they imprint the image and sound of their mother most effectively only from 13 to 16 hours after hatching. During this period a duckling possibly accepts another moving object as mother (e.g. man), but not before or afterwards.

Analyzing the circumstances when rapid learning can be successful, at least two important prerequisites can be identified:

First, the importance and correctness of the learned prototypical association is clarified.

 

And second, the correct structural context is known.

This is important in order to draw meaningful inferences from the prototypical data set, when the system needs to generalize in new, previously unknown situations. The main focus of the present work are learning mechanisms of this category: rapid learning – requiring only a small number of training data. Our computational approach to the realization of such learning algorithm is derived form the “Self-Organizing Map” (SOM). An essential new ingredient is the use of a continuous parametric representation that allows a rapid and very flexible construction of manifolds with intrinsic dimensionality up to 4  8 i.e. in a range that is very typical for many situations in robotics.

This algorithm, is termed “Parameterized Self-Organizing Map” (PSOM) and aims at continuous, smooth mappings in higher dimensional spaces. The PSOM manifolds have a number of attractive properties. We show that the PSOM is most useful in situations where the structure of the obtained training data can be correctly inferred. Similar to the SOM, the structure is encoded in the topological order of prototypical examples.

As explained in chapter 4, the discrete nature of the SOM is overcome by using a set of basis functions. Together with a set of prototypical training data, they build a continuous mapping manifold, which can be used in several ways. The PSOM manifold offers auto-association capability, which can serve for completion of partial inputs and simultaneously mapping to multiple coordinate spaces.

The PSOM approach exhibits unusual mapping properties, which are exposed in chapter 5. The special construction of the continuous manifold deserves consideration and approaches to improve the mapping accuracy and computational efficiency. Several extensions to the standard formulations are presented in Chapter 6. They are illustrated at a number of examples.

In cases where the topological structure of the training data is known beforehand, e.g. generated by actively sampling the examples, the PSOM “learning” time reduces to an immediate construction. This feature is of particular interest in the domain of robotics: as already pointed out, here

the cost of gathering the training data is very relevant as well as the availability of adaptable, high-dimensional sensorimotor transformations. Chapter 7 and 8 present several PSOM examples in the vision and the robotics domain. The flexible association mechanism facilitates applications:

feature completion; dynamical sensor fusion, improving noise rejection; generating perceptual hypotheses for other sensor systems; various robot kinematic transformation can be directly augmented to combine e.g. visual coordinate spaces. This even works with redundant degrees of freedom, which can additionally comply to extra constraints.

Chapter 9 turns to the next higher level of one-shot learning. Here the learning of prototypical mappings is used to rapidly adapt a learning system to new context situations. This leads to a hierarchical architecture, which is conceptually linked, but not restricted to the PSOM approach. One learning module learns the context-dependent skill and encodes the obtained expertise in a (more-or-less large) set of parameters or weights.

A second meta-mapping module learns the association between the recognized context stimuli and the corresponding mapping expertise. The learning of a set of prototypical mappings may be called an investment learning stage, since effort is invested, to train the system for the second, the one-shot learning phase. Observing the context, the system can now adapt most rapidly by “mixing” the expertise previously obtained. This mixture-of-expertise architecture complements the mixture-of-experts architecture (as coined by Jordan) and appears advantageous in cases where the variation of the underlying model are continuous within the chosen mapping domain.

Chapter 10 summarizes the main points. Of course the full complexity of learning and the complexity of real robots is still unsolved today. The present work attempts to make a contribution to a few of the many things that still can be and must be improved.