BISC Seminar, 19th June 1997, 310 Soda Hall, 4-5pm

Frank Hoffmann (
Mon, 9 Jun 1997 12:50:35 +0200

TO: BISC group
FROM: Frank Hoffmann

Incremental Class Learning approach and its application
to Handwritten Digit Recognition

BISC Seminar

Jacek Mandziuk

International Computer Science Institute, Berkeley
and EECS Department UC Berkeley

June 19th, 1997
310 Soda Hall


Incremental Class Learning (ICL) provides a feasible framework for development
of scalable learning systems. Instead of learning the whole problem at once,
ICL focuses on learning subproblems incrementally, one at a time, using the
results of prior learning during subsequent learning, and then combining the
solutions in an appropriate manner.
Therefore, the ``cathastrophic interference problem'', which occurs
in sequential learning, is significantly alleviated.

The ICL approach presented in this talk can be summarized as follows.
Primarily the system focuses on one category and after it learns this
category, it tries to identify the compact subset of features (nodes) in the
hidden layers, that are crucial for recognition of this category.
The system then freezes these crucial nodes (features) by fixing their
incoming weights. As a results, they cannot be obliterated in subsequent
learning. These frozen features are availiable for learning other categories,
and serve as parts of weight structures build subsequently to recognize other
categories. The set of features gradually stabilizes and eventually,
learning a new category requires less effort, and primarily
consists of combining existing features in an appropriate manner.
Eventually a large number of nodes are {\em shared} among various categories.

We present results of applying the ICL approach to the Handwritten Digit
Recognition problem, based on the Spatio-Temporal representation of patterns.

Joint work with Lokendra Shastri.

Frank Hoffmann UC Berkeley
Computer Science Division Department of EECS
Email: phone: 1-510-642-8282
URL: fax: 1-510-642-5775