Re: Who invented "backprop"?

Stefano Rovetta (ste@dibe.unige.it)
Wed, 24 Jul 1996 14:17:47 +0200


On Mon, 15 Jul 1996 19:53:38 GMT, Mike Fouche
<mikef@hsvaic.hv.boeing.com> wrote:
> .... Dr. Werbos
> was the "inventor" of backprop - I had always thought that Dr. Rumelhart
> and possibly Dr. Widrow were the authors/creators. Can anybody clear
> this up for me?

Stephen Wolstenholme replied:
> I think Werbos was the first to publish a backprop method. The first
> that method that I became aware of was that of Brian Dolby in 1971 but
> he never published anything.

I confirm that the first reference I know of is [1].
But, as far as I know, the roots date even before that (1950's, I think).

If you think of Backprop from a mathematical point of view, it is only
a matter of computing partial derivatives by applying the "chain rule",
albeit in a smart and distributed fashion.

However, the contribution of its developers is itself a form of
"distributed invention", each researcher adding a step to the
final (?) result by Rumelhart et al. See also [2] for a brief
historical review (p.124).

Hope it can be of help.
--Stefano

References
----------
[1] P. Werbos, "Beyond regression: new tools for prediction and analysis
in the behavioral sciences," PdD thesis, Applied Mathematics Dept.,
Harvard University, 1974
[2] R. Hecht-Nielsen, "Neurocomputing," Addison-Wesley, 1989

------------------------------------------------------------
Stefano Rovetta, PhD student Phone: +39-10-353 2268
DIBE -- University of Genova Fax: +39-10-353 2175
Via all'Opera Pia 11a E-mail ste@dibe.unige.it
16145 GENOVA (Italy)
------------------------------------------------------------
The opinions expressed here are neither mine
nor those of my department