Jay McClelland | Neural Networks: Artificial and Biological

The Cartesian Cafe

Jay McClelland is a pioneer in the field of artificial intelligence and is a cognitive psychologist and professor at Stanford University in the psychology, linguistics, and computer science departments. Together with David Rumelhart, Jay published the two volume work Parallel Distributed Processing, which has led to the flourishing of the connectionist approach to understanding cognition.

In this conversation, Jay gives us a crash course in how neurons and biological brains work. This sets the stage for how psychologists such as Jay, David Rumelhart, and Geoffrey Hinton historically approached the development of models of cognition and ultimately artificial intelligence. We also discuss alternative approaches to neural computation such as symbolic and neuroscientific ones.

Patreon (bonus materials + video chat):
https://www.patreon.com/timothynguyen

Part I. Introduction

• 00:00 : Preview
• 01:10 : Cognitive psychology
• 07:14 : Interdisciplinary work and Jay's academic journey
• 12:39 : Context affects perception
• 13:05 : Chomsky and psycholinguists
• 8:03 : Technical outline

Part II. The Brain

• 00:20:20 : Structure of neurons
• 00:25:26 : Action potentials
• 00:27:00 : Synaptic processes and neuron firing
• 00:29:18 : Inhibitory neurons
• 00:33:10 : Feedforward neural networks
• 00:34:57 : Visual system
• 00:39:46 : Various parts of the visual cortex
• 00:45:31 : Columnar organization in the cortex
• 00:47:04 : Colocation in artificial vs biological networks
• 00:53:03 : Sensory systems and brain maps

Part III. Approaches to AI, PDP, and Learning Rules

• 01:12:35 : Chomsky, symbolic rules, universal grammar
• 01:28:28 : Neuroscience, Francis Crick, vision vs language
• 01:32:36 : Neuroscience = bottom up
• 01:37:20 : Jay’s path to AI
• 01:43:51 : James Anderson
• 01:44:51 : Geoff Hinton
• 01:54:25 : Parallel Distributed Processing (PDP)
• 02:03:40 : McClelland & Rumelhart’s reading model
• 02:31:25 : Theories of learning
• 02:35:52 : Hebbian learning
• 02:43:23 : Rumelhart’s Delta rule
• 02:44:45 : Gradient descent
• 02:47:04 : Backpropagation
• 02:54:52 : Outro: Retrospective and looking ahead

Further reading:

Rumelhart, McClelland. Parallel Distributed Processing.

McClelland, J. L. (2013). Integrating probabilistic models of perception and interactive neural networks: A historical and tutorial review

Twitter: @iamtimnguyen

Webpage: http://www.timothynguyen.org