We could extend them either by scaling up small connection­ist models or by combining small-scale networks into some larger organization. In the firs… - Marvin Minsky

" "

We could extend them either by scaling up small connection­ist models or by combining small-scale networks into some larger organization. In the first case, we would expect to encounter theoretical obstacles to maintaining [generalized delta rule]’s effectiveness on larger, deeper nets. And despite the reputed efficacy of other alleged remedies for the deficiencies of hill-climbing, such as “annealing,” we stay with our research conjecture that no such procedures will work very well on large-scale nets, except in the case of problems that turn out to be of low order in some appropriate sense. The second alternative is to employ a variety of smaller networks rather than try to scale up a single one... No single-method learning scheme can operate efficiently for every possible task; we cannot expect any one type of machine to account for any large portion of human psychology.