Modelling Brain Function

The World of Attractor Neural Networks

By: Daniel J. Amit

Write A Review

Paperback | 21 December 1992

At a Glance

Paperback
524 Pages

Dimensions(cm)
22.86 x 15.24 x 2.97

Paperback

RRP $143.95

$129.75

10%OFF

or 4 interest-free payments of $32.44 with

Ships in 5 to 7 business days

One of the most exciting and potentially rewarding areas of scientific research is the study of the principles and mechanisms underlying brain function. It is also of great promise to future generations of computers. A growing group of researchers, adapting knowledge and techniques from a wide range of scientific disciplines, have made substantial progress understanding memory, the learning process, and self organization by studying the properties of models of neural networks - idealized systems containing very large numbers of connected neurons, whose interactions give rise to the special qualities of the brain. This book introduces and explains the techniques brought from physics to the study of neural networks and the insights they have stimulated. It is written at a level accessible to the wide range of researchers working on these problems - statistical physicists, biologists, computer scientists, computer technologists and cognitive psychologists. The author presents a coherent and clear nonmechanical presentation of all the basic ideas and results. More technical aspects are restricted, wherever possible, to special sections and appendices in each chapter. The book is suitable as a text for graduate courses in physics, electrical engineering, computer science and biology.

Industry Reviews

"...of interest to those following the neural net field...takes off from discoveries that link areas of physics with the emerging neural network paradigm." Intelligence Monthly "...regard this book as an opening of a discussion--undoubtedly a very qualified one." Journal of Mathematical Psychology

Shipping

	Standard Shipping	Express Shipping
Metro postcodes:	$9.99	$14.95
Regional postcodes:	$9.99	$14.95
Rural postcodes:	$9.99	$14.95

Orders over $89.00 qualify for free shipping.

How to return your order

At Booktopia, we offer hassle-free returns in accordance with our returns policy. If you wish to return an item, please get in touch with Booktopia Customer Care.

Additional postage charges may be applicable.

Defective items

If there is a problem with any of the items received for your order then the Booktopia Customer Care team is ready to assist you.

For more info please visit our Help Centre.

You Can Find This Book In

Non-Fiction Medicine Clinical & Internal Medicine Neurology & Clinical Neurophysiology Science Biology, Life Sciences Life Sciences in General Neurosciences Computing & I.T.Computer Science Artificial Intelligence Neural Networks & Fuzzy Systems

Computer Networking & Communications Pre-Clinical Medicine & Basic Sciences Physiology

Preface	p. xiii
Introduction	p. 1
Philosophy and Methodology	p. 1
Reduction to physics and physics modeling analogues	p. 1
Methods for mind and matter	p. 3
Some methodological questions	p. 5
Neurophysiological Background	p. 9
Building blocks for neural networks	p. 9
Dynamics of neurons and synapses	p. 12
More complicated building blocks	p. 15
From biology to information processing	p. 17
Modeling Simplified Neurophysiological Information	p. 18
Neuron as perceptron and formal neuron	p. 18
Digression on formal neurons and perceptrons	p. 20
Beyond the basic perceptron	p. 25
Building blocks for attractor neural networks (ANN)	p. 27
The Network and the World	p. 31
Neural states, network states and state space	p. 31
Digression on the relation between measures	p. 33
Representations on network states	p. 35
Thinking about output mechanism	p. 38
Spontaneous Computation vs. Cognitive Processing	p. 44
Input systems, transducers, transformers	p. 44
ANN's as computing elements -- a position	p. 45
ANN's and computation of mental representations	p. 48
Bibliography	p. 53
The Basic Attractor Neural Network	p. 58
Networks of Analog, Discrete, Noisy Neurons	p. 58
Analog neurons, spike rates, two-state neural models	p. 58
Binary representation of single neuron activity	p. 63
Noisy dynamics of discrete two-state neurons	p. 65
Dynamical Evolution of Network States	p. 68
Network dynamics of discrete-neurons	p. 68
Synchronous dynamics	p. 70
Asynchronous dynamics	p. 72
Sample trajectories and lessons about dynamics	p. 74
Types of trajectories and possible interpretation - a summary	p. 79
On Attractors	p. 81
The landscape metaphor	p. 81
Perception, recognition and recall	p. 84
Perception errors due to spurious states - possible role of noise	p. 85
Psychiatric speculations and images	p. 87
The role of noise and simulated annealing	p. 89
Frustration and diversity of attractors	p. 91
Bibliography	p. 95
General Ideas Concerning Dynamics	p. 97
The Stochastic Process, Ergodicity and Beyond	p. 97
Stochastic equation and apparent ergodicity	p. 97
Two ways of evading ergodicity	p. 101
Cooperativity as an Emergent Property in Magnetic Analog	p. 105
Ising model for a magnet - spin, field and interaction	p. 105
Dynamics and equilibrium properties	p. 108
Noiseless, short range ferromagnet	p. 112
Fully connected Ising model: real non-ergodicity	p. 119
From Dynamics to Landscapes - The Free Energy	p. 125
Energy as Lyapunov function for noiseless dynamics	p. 125
Parametrized attractor distributions with noise	p. 126
Free-energy landscapes - a noisy Lyapunov function	p. 127
Free-energy minima, non-ergodicity, order-parameters	p. 129
Free-Energy of Fully Connected Ising Model	p. 131
From minimization equation to the free-energy	p. 131
The analytic way to the free-energy	p. 133
Attractors at metastable states	p. 140
Synaptic Symmetry and Landscapes	p. 141
Noiseless asynchronous dynamics - energy	p. 141
Detailed balance for noisy asynchronous dynamics	p. 142
Noiseless synchronous dynamics - Lyapunov function	p. 143
Detailed balance for noisy synchronous dynamics	p. 145
Appendix: Technical Details for Stochastic Equations	p. 146
The maximal eigen-value and the associated vector	p. 146
Differential equation for mean magnetization	p. 147
The minimization of the dynamical free-energy	p. 150
Legendre transform for the free-energy	p. 152
Bibliography	p. 153
Symmetric Neural Networks at Low Memory Loading	p. 155
Motivations and List of Results	p. 155
Simplifying assumptions and specific questions	p. 155
Specific answers for low loading of random memories	p. 158
Properties of the noiseless network	p. 162
Properties of the network in the presence of fast noise	p. 166
Explicit Construction of Synaptic Efficacies	p. 169
Choice of memorized patterns	p. 169
Storage prescription - "Hebb's rule"	p. 170
A decorrelating (but nonlocal) storage prescription	p. 172
Stability Considerations at Low Storage	p. 174
Signal to noise analysis - memories, spurious states	p. 174
Basins of attraction and retrieval times	p. 178
Neurophysiological interpretation	p. 180
Mean Field Approach to Attractors	p. 181
Self-consistency and equations for attractors	p. 181
Self-averaging and the final equations	p. 187
Free-energy, extrema, stability	p. 189
Mean-field and free-energy - synchronous dynamics	p. 191
Retrieval States, Spurious States - Noiseless	p. 192
Perfect retrieval of memorized patterns	p. 192
Noiseless, symmetric spurious memories	p. 194
Non-symmetric spurious states	p. 198
Are spurious states a free lunch?	p. 199
Role of Noise at Low Loading	p. 200
Ergodicity at high noise levels - asynchronous	p. 200
Just below the critical noise level	p. 201
Positive role of noise and retrieval with no fixed points	p. 206
Appendix: Technical Details for Low Storage	p. 208
Free-energy at finite p - asynchronous	p. 208
Free-energy and solutions - synchronous dynamics	p. 209
Bound on magnitude of overlaps	p. 211
Asymmetric spurious solution	p. 212
Bibliography	p. 213
Storage and Retrieval of Temporal Sequences	p. 215
Motivations: Introspective, Biological, Philosophical	p. 215
The introspective motivation	p. 215
The biological motivation	p. 216
Philosophical motivations	p. 218
Storing and Retrieving Temporal Sequences	p. 221
Functional asymmetry	p. 221
Early ideas for instant temporal sequences	p. 221
Temporal Sequences by Delayed Synapses	p. 226
A simple generalization and its motivation	p. 226
Dynamics with fast and slow synapses	p. 229
Simulation examples of sequence recall	p. 231
Adiabatically varying energy landscapes	p. 235
Bi-phasic oscillations and CPG's	p. 238
Tentative Steps into Abstract Computation	p. 239
The attempt to reintroduce structured operations	p. 239
Ann counting chimes	p. 241
Counting network - an exercise in connectionist programming	p. 241
The network	p. 243
Its dynamics	p. 245
Simulations	p. 248
Reflections on associated cognitive psychology	p. 251
Sequences Without Synaptic Delays	p. 253
Basic oscillator - origin of cognitive time scale	p. 253
Behavior in the absence of noise	p. 255
The role of noise	p. 256
Synaptic structure and underlying dynamics	p. 259
Network storing sequence with several patterns	p. 262
Appendix: Elaborate Temporal Sequences	p. 262
Temporal sequences by time averaged synaptic inputs	p. 262
Temporal sequences without errors	p. 266
Bibliography	p. 267
Storage Capacity of ANN's	p. 271
Motivation and general considerations	p. 271
Different measures of storage capacity	p. 271
Storage capacity of human brains	p. 273
Intrinsic interest in high storage	p. 275
List of results	p. 275
Statistical Estimates of Storage	p. 278
Statistical signal to noise analysis	p. 278
Absolute informational bounds on storage capacity	p. 283
Coupling (synaptic efficacies) for optimal storage	p. 285
Theory Near Memory Saturation	p. 289
Mean-field equations with replica symmetry	p. 289
Retrieval in the absence of fast noise	p. 294
Analysis of the T = 0 equations	p. 299
Memory Saturation with Noise and Fields	p. 304
A tour in the T-[alpha] phase diagram	p. 304
Effect of external fields - thresholds and PSP's	p. 308
Fields coupled to several patterns	p. 311
Some technical details related to phase diagrams	p. 312
Balance Sheet for Standard ANN	p. 315
Limiting framework and analytic consequences	p. 315
Finite-size effects and basins of attraction: simulations	p. 318
Beyond the Memory Blackout Catastrophe	p. 324
Bounded synapses and palimpsest memory	p. 324
The 7 [plus or minus] 2 rule and palimpsest memories	p. 328
Appendix: Replica Symmetric Theory	p. 330
The replica method	p. 330
The free-energy and the mean-field equations	p. 332
Marginal storage and palimpsests	p. 339
Bibliography	p. 342
Robustness - Getting Closer to Biology	p. 345
Synaptic Noise and Synaptic Dilution	p. 345
Two meanings of robustness	p. 345
Noise in synaptic efficacies	p. 347
Random symmetric dilution of synapses	p. 352
Non-Linear Synapses and Limited Analog Depth	p. 355
Place and role of non-linear synapses	p. 355
Properties of networks with clipped synapses	p. 357
Non-linear storage and the noisy equivalent	p. 359
Clipping at low storage level	p. 362
Random vs. Functional Synaptic Asymmetry	p. 363
Random asymmetry and performance quality	p. 363
Asymmetry, noise and spin-glass suppression	p. 366
Neuronal specificity of synapses - Dale's law	p. 368
Extreme asymmetric dilution	p. 370
Functional asymmetry	p. 375
Effective Cortical Cycle Times	p. 375
Slow bursts and relative refractory period	p. 375
Neuronal memory and expanded scenario	p. 377
Simplified scenario for relative refractory period	p. 378
Appendix: Technical Details	p. 380
Digression - the mean-field equations	p. 380
Dilution requirement	p. 384
Bibliography	p. 385
Memory Data Structures	p. 387
Biological and Computational Motivation	p. 387
Low mean activity level and background-foreground asymmetry	p. 387
Hierarchies for biology and for computation	p. 388
Local Treatment of Low Activity Patterns	p. 389
Demise of naive standard model	p. 389
Modified ANN and a plague of spurious states	p. 391
Constrained dynamics - monitoring thresholds	p. 396
Properties of the constrained biased network	p. 398
Quantity of information in an ANN with low activity	p. 403
More effective storage of low activity (sparse) patterns	p. 405
Hierarchical Data Structures in a Single Network	p. 409
Early proposals	p. 409
Explicit construction of hierarchy in a single ANN	p. 410
Properties of hierarchy in a single network	p. 412
Prosopagnosia and learning class properties	p. 412
Multy-ancestry with many generations	p. 414
Hierarchies in Multi-ANN: Generalization First	p. 418
Organization of the data and the networks	p. 418
Hierarchical dynamics	p. 420
Hierarchy for image vector quantization	p. 422
Appendix: Technical Details for Biased Patterns	p. 423
Noise estimates for biased patterns	p. 423
Mean-field equations in noiseless biased network	p. 424
Retrieval entropy in biased network	p. 424
Mean-square noise in low activity network	p. 425
Bibliography	p. 426
Learning	p. 428
The Context of Learning	p. 428
General Comments and a limited scope	p. 428
Modes, time scales and other constraints	p. 430
The need for learning modes	p. 432
Results for learning in learning modes	p. 433
Learning in Modes	p. 434
Perceptron learning	p. 434
ANN learning by perceptron algorithm	p. 438
Local learning of the Kohonen synaptic matrix	p. 441
Natural Learning - Double Dynamics	p. 443
General features	p. 443
Learning in a network of physiological neurons	p. 444
Learning to form associations	p. 447
Memory generation and maintenance	p. 450
Technical Details in Learning Models	p. 455
Local Iterative Construction of Projector Matrix	p. 455
The free energy and the correlation function	p. 458
Bibliography	p. 458
Hardware Implementations of Neural Networks	p. 461
Situating Artificial Neural Networks	p. 461
The role of hardware implementations	p. 461
Motivations for different designs	p. 462
The VLSI Neural Network	p. 465
High density high speed integrated chip	p. 465
Smaller, more flexible electronic ANN's	p. 469
The Electro-Optical ANN	p. 474
Shift Register (CCD) Implementation	p. 477
Bibliography	p. 479
Glossary	p. 481
Index	p. 487
Table of Contents provided by Syndetics. All Rights Reserved.

Modelling Brain Function

The World of Attractor Neural Networks

At a Glance

Paperback

Industry Reviews

Shipping

How to return your order

Defective items

You Can Find This Book In

More in Neural Networks & Fuzzy Systems

Artificial Neural Networks for System Identification and Control

Prompt Engineering for Generative AI

Future-Proof Inputs for Reliable AI Outputs

Hands-On Machine Learning with Scikit-Learn, Keras, and Tensorflow

Concepts, Tools, and Techniques to Build Intelligent Systems

Practical Machine Learning for Computer Vision

End-To-End Machine Learning for Images

Effective Machine Learning Teams

Best Practices for ML Practitioners

Intelligent Estimation

A Soft Computing Paradigm

Hands-On Machine Learning with Scikit-Learn and Pytorch

Concepts, Tools, and Techniques to Build Intelligent Systems

Quantum Artificial Intelligence

Technologies, Algorithms and Programming

Neural Interfaces

Bridging Cognitive Neuroscience and Artificial Intelligence

Deep Learning

Principles and Implementations

An Introduction to Large Language Models

An Introduction to Large Language Models

Machine Translation

Best Practices using Deep Learning and Generative AI

Neuroscience and the Dream of Digital Immortality

The Upload Illusion

Neuroscience and the Dream of Digital Immortality

The Upload Illusion

Surgical Robotics and Computer-Assisted Surgery

Fundamentals and Advances

Superhuman

Beyond the Limits of Our Intelligence

Reinforcement Learning in Action

From Foundations to Frontier AI

Reinforcement Learning in Action

From Foundations to Frontier AI

Modern Computing

Principles and Paradigms

Fuzzy Theory Applications in Engineering

Fuzzy Theory Applications

Computational Approaches to Emotion in Artificial Psychology

Computational Approaches to Emotion in Artificial Psychology

Mathematical Foundations of Deep Learning

Theory and Algorithms

This product is categorised by