Syllabus for Phil 3395 Introduction to Cognitive Science 1998
Jim Garson x 3208 Office Hours: MWF 11-12 e-mail:garson@menudo.uh.edu
Texts: M Mind , Paul Thagard
CS Cognitive Science , Stillings et. al.
Packet #56 at the UC Copy Center or the Bookstore

Week 1 Jan 21-26 What is Cognitive Science?
M Ch. 1 CS Ch. 1
Top-Down: Classical Cognitive Science
Week 2 Jan 28-Feb 2 Logic
M pp. 23-34 of Ch 2.
Week 3 Jan Feb 4-9 Reasoning
M pp. 34-41 (rest of Ch. 2) CS pp. 168-174
Week 4 Feb 11-16 Rules
M Ch 3. CS pp. 139-151 (skim if you like) pp. 155-159, 164-168
Week 5 Feb 18-23 Concepts and Learning
M Ch. 4. CS pp. 159-164, 192-213
Week 6 Feb 25-27 Review and QUIZ Feb 27
Bottom-Up: Neurally Inspired Cognitive Science
Week 7 Mar 2-6 Neuroscience
CS Ch. 7 (skim 270-275; 282-289; 291-298; 321-325)
Week 8-9 Mar 9-23 Vision
M Ch. 6. CS Ch. 12 (skim 467-479; 487-490; 506-512)
Week 9-10 Mar 25- Apr 3 Connectionism
M Ch. 7. CS pp. 63-83; 324-325; 92-93; 114-116; 121-124
Week 11 Apr 6 - Apr 10 Review and QUIZ Apr 10
In the Middle: Cognitive Psychology, Linguistics and Philosophy
Week 12 Apr 13-17 Cognitive Psychology
CS Ch. 2.1-2.3 M Ch. 6. CS 2.7
Weeks 13 Apr 20-24 Linguistics
CS Ch 6.1, 6.2, 6.3
Weeks 14-15 Apr 27-May 1 Philosophical Issues
M Ch. 9 CS Ch. 8.3 Qualia
May 4 Review

Final or Project MAY 8

Quizes 30% each. Final/Project 40%

1. Introductions
   
2. Books, There are 2:
   a. Stillings et al. will be used for many reading selections: Chs. 1-4, 6.1-6.3, 7, 8.3, 12
   b. Thagard is our text. We will cover all of it except (perhaps) Ch. 5
   
3. Your Duties: Read week's assignment before first class. Be prepared to answer oral questions about its content from me.
   
4. Evaluation: 2 Quizzes and a Final/Project. The project may be a paper, a website or a computer program, or you may opt to take a final exam. If you chose to do a project, a proposal of 2 pages explaining your plans must be provided to me on April 1 . If I do not receive it on time you will take the final. All quizzes 60%, Final/Project 40%. These notes contain an annotated list of resources: Books, Websites etc. to help you with projects. For more details see me.
   
   

B. Introduction: What is Cognitive Science?

1. Cognition = Thinking
   

(So Cognitive Science = the Science of Thinking or the Science of the Mind)

1. So it is just Psychology?
   True there is Cognitive Psychology.
   This brand of psychology is a new movement that has to some extent displaced the Behaviorist School.
   
2. But there are also essential contributions from other fields:
   Linguistics, Computer Science, Neuroscience, Anthropology, and Philosophy.
   
3. So any course in Cog Sci is multidisciplinary.
   a. Problem: Since each professor is schooled in a given discipline, each teacher will have gaps and prejudices. Here is some account of mine. Strong areas: Philosophy, (Logic), Linguistics, and Computer Science. Weak ones: Neuroscience, Psychology and Anthropology
   b. We will have a few visitors to help fill some of my gaps.
   
4. In each of the disciplines the contributions are only from a subgroup:
   a. Philosophy: Philosophy of Mind, Logic
   b. Psychology: Cognitive Psychology
   c. Computer Science: A wing of Artificial Intelligence
   d. Linguistics: Computational Linguistics
   e. Neuroscience: Functional neuroscience
   f. Anthropology: Cultural or Cognitive Anthropology
   
5. How to conceive of this interdisciplinary overlap: Each is about the same topic, but each one differs markedly in their methods: (See the discussion in CS section 1.6.)
   a. Philosophy: More abstract issues about the nature of thinking.
   
1. What are the principles of correct thinking?
   
2. How can a brain (or a machine) represent the world?
   
3. The Mind Body Problem. What is a Mind and how is it related to the Body?
   
4. Subjective Experience. Are subjective experiences just activities of the Brain?
   

b. Cognitive Psychology: Controlled experiments with human subjects (and sometimes smaller animals) to try to characterize the nature of human thinking and to figure out and test theories of the mechanisms that might produce those abilities.


1. Perception: How is visual information processed into a picture of the world?
   
2. Memory: What different memory systems are there and how do they work?
   
3. Reasoning: How do humans reason? What explains their success and failure?
   
4. Concepts: What concepts do humans use and where do they come from?
   
5. Emotions: How do emotions affect and/or contribute to thinking?
   
6. Planning: How do humans formulate plans and monitor their execution?
   
7. Movement: How can we explain human abilities at graceful motion?
   

c. Artificial Intelligence: Create computer models that actually simulate human cognition.


1. Many AI researchers do not care to figure out how humans actually do it. Often their goal may be to create a system to do some task better than humans do by doing it a better way: (medical diagnosis).
   
2. Creating models can help reveal how hard the problems are that the human being takes for granted. For example robotics has a long way to go to understand and duplicate the seemingly simple feat of walking.
   

d. Computational Linguistics: Formulate the structures found in language as a set of rules and try to see how such rules might be carried out in machines or the brain.
e. Functional Neuroscience: Study the brain using PET, MRI, probes, and the effects of brain damage etc., to try to see how various parts of the brain contribute to human cognitive abilities.
f. Cognitive Anthropology: Cultural anthropology studies of ways of life, including how humans in different cultures perceive, reason and act. By examining how human cognition differs in different cultures, clues to the nature of human thinking can be developed. For example, there are universals in name systems for colors: Black White Red, Blue/Green etc.

C. Applications of Cognitive Science

1. What could be more exciting than to cross the last major intellectual frontier?
   
2. Understand reading so as to treat reading problems.
   
3. Speech therapy for stroke victims.
   
4. Understand memory so as to predict reliability of legal witnesses
   (Witness blending: At what speed did the car smash into the truck? vs.
   At what speed did the car run into the truck?)
   
5. Imagery for training athletes
   
6. Expert systems and robotics
   
   

D. Roots of Cognitive Science

1. History
   a. Cog Sci (like sciences in general) has its roots in philosophy, especially the philosophy of nature. All other sciences related to cog sci branched out of philosophy:
   

Philosophy: 300 BC
Neurology: mid 1800s
Psychology: 1860s
Anthropology: 1920s
Linguistics: 1920s
Computer Science: 1960
(PhD is doctor of philosophy, In Library of Congress Book codes, B is philosophy, BD is psychology). b. So at the roots we can expect to see philosophers and a key philosophical issue: Is the Mind appropriate for study by science? This question is related to the fundamental struggle between dualists and naturalists.

1. Dualism: Mind Body are distinct.
   Plato: Reasoning is something distinct and higher than the physical. (Man is rational animal.) Thought is communication with the world of eternal and perfect Forms.
   Descartes: (and the Medieval Christian tradition). The Mind (Soul) is radically distinct from (and higher than) the body. It is eternal and therefore non-physical. It interacts with the physical world, but is essentially outside what physical science can explain.
   Today: The general public and such philosophers as David Chalmers and Thomas Nagel believe in remnants of this view.
   
2. Naturalism: There is nothing beyond the physical world so a Mind must be some feature or aspect of the Body
   Some historical sources:
   Aristotle: There is no world of forms, though each thing has a built-in essence that helps us understand what it is and what it strives for. Formulation of the rules of logic.
   La Mettrie: Man is a machine. Human thinking (like all else) is subject to the laws of nature.
   Hobbes: Thought is processing of non-numerical symbols.
   Leibniz: Reasoning is like numerical calculation (Ars Combinatoria).
   Wundt: Birth of Experimental Psychology (though he was a philosophical dualist)
   Watson, Skinner: Behaviorist Psychology. There is nothing to explain except human behavior; the Mind/Soul is a myth.
   Classical Cognitive Science: The Mind is not a myth. We can theorize about internal (or "subjective") states in scientifically acceptable ways. Thought is akin to the information processing carried out by symbolic computers, and this processing can be the subject of science.
   Connectionism: A better theory of how the mind works can be forged by taking our cue from neurology. Forms of parallel computation (parallel distributed processing, neural nets) that are essentially non-symbolic are better suited to explaining how thinking works.
   

E. A Controversy: Top-Down vs. Bottom Up

1. Top-Down The classical paradigm: CRUM (Computational-Representational Understanding of Mind) GOFAI (Good Ol' Fashion Artificial Intelligence)
   a. Characterize what the mind does using logic and computer science. Once these abilities are simulated on a computer you can then understand the general principles that underlie human intelligence.
   b. Thinking = processing of representations.
   c. Some analogies:
   
1. Data Structures + Algorithms = Programs
   
2. Ingredients + Instructions = Meal
   
3. Mental Representations + Computational Procedures = Thinking
   
4. Mind : Brain :: Software : Hardware
   

d. Understanding cognition does not require detailed information about how the brain actually implements information processing.
Instruction: cream the butter can be done in various ways. It doesn't matter (much) to the end product exactly how you do it
e. After all. You can do software science without knowing anything about hardware.
f. The project: Define the task. Theorize about what are likely representations and procedures to get job done. Model the theory as a data structure and algorithm. Program actual computer code, and run it on a computer platform to test the ideas.
g. When the project is done we will understand the Mind as the result of computation in the brain.


2. Bottom - Up: Connectionism
   a. We need to look more carefully at the actual neurology of the brain to understand what it does and how it does it.
   b. The symbolic computer is a very bad model of what the brain does and how it does it.
   c. A search for alternative mechanisms guided by what we find out about neural structure will be more fruitful in explaining thought.
   
   

Reading
Von Ekhart, Barbara What is Cognitive Science? A good discussion of the nature of the discipline at the turn of the 90s. Discusses worries that cognitive science may not count as a cohesive science.

1. Logic is a formal theory about how reasoning works.
   
2. Thinking appears to depend centrally on reasoning ability.
   
3. We now know how to program computers to process the rules of logic, so this might be an excellent model for understanding (human) intelligence.
   

B. History


1. Aristotle is the father of logic: His work on the syllogisms was refined by logicians of the middle ages.
   a. At root is the idea of a syllogism - a pattern of reasoning:
   All men are mortal All A are B
   Socrates is a man S is A
   Therefore: Socrates is mortal So: S is B
   b. Fundamental idea: good reasoning depends on using the right form. The content does not matter to whether the reasoning is correct. Any example of reasoning that matches the right-hand pattern above is good reasoning.
   c. Medieval logic taught which were correct syllogisms with a poem.
   d. But it was recognized that this system had its limitations. For example there seemed to be no way to handle the following reasoning with the Aristotelian approach:
   All horses are mammals
   So every tail of a horse is a tail of a mammal
   

The fundamental problem was that the system couldn't handle relations: is a tail of.

1. The work of de Morgan, Frege and Russell lead to the development of predicate logic, which has much stronger expressive powers, and forms the core of the modern approach to formal logic.
   

C. Predicate Logic


1. The Language of Predicate Logic
   a. Connectives (Symbols of Propositional Logic)
   & . ^ And
   Or
   ~ - Not
   -> If .. then
   <-> If and only if
   b. Symbols from Predicate Logic
   a b c Names R(a) Albert is rowdy
   P Q R Predicate (Letters) L(a,b) Albert loves Betty
   f g h Function (Letters) R(f(a)) Albert's father is rowdy
   x y z Variables UxR(x) Everything is rowdy
   = Identity Ux(R(x)->x=a) Only Albert is rowdy
   U E Quantifiers UxEyL(x,y) Everything loves something
   The conventions of computer science are more long-winded but more legible:
   (for-all x)(Man(x) -> Mortal(x))
   Man(Socrates)
   Mortal(Socrates)
   
2. Some Rules for Predicate Logic
   UxA(x) / A(c) Universal Instantiation (Specification) (UOut)
   A, B / A&B Conjunction (&In)
   A, A->B / B Modus Ponens (>Out)
   A->B, ~B / ~A Modus Tollens
   
3. Proofs in Predicate Logic
   One may use proofs to verify that a pattern of reasoning is correct (valid). A proof is a sequence of steps that result from applying the rules to previous members of the sequence to obtain the conclusion from the given formulas. For example, here is a proof for the Socrates is mortal argument:
   
1. (for-all x)(Man(x) -> Mortal(x)) Given
   
2. Man(Socrates) Given
   
3. Man(Socrates) -> Mortal(Socrates) 1 Universal Instantiation
   
4. Mortal(Socrates) 2 3 Modus Ponens
   
4. The power of Predicate Logic
   a. By adding to these rules one can formulate a consistent (sound) and complete system for predicate logic. This means that every correct reasoning pattern can be proven by the rules (completeness), and no incorrect arguments can be proven (consistency).
   b. The process of proof finding can be mechanized with computer programs called theorem provers. Most theorem provers have the feature that when an reasoning pattern is correct, the theorem prover is guaranteed to find the proof (sooner or later). So all correct patterns of reasoning can be identified mechanically, simply by applying symbolic rules to symbolic representations of arguments. Good reasoning can be computerized!
   

D. Problem Solving, Logic Programming and PROLOG


1. It may be hard to imagine how a theorem prover might be used to solve problems. Isn't problem solving essentially creative? There is a branch of artificial intelligence (AI) called logic programming that uses predicate logic and its rules as the basis for a programming language called PROLOG. This school of AI believes that intelligent problem solving (and so all computer programming) can be based simply on predicate logic.
   
2. To make the idea more plausible, we will work through an example. (For simplicity I will not use PROLOG notation, but a variant of the notation of Predicate Logic.)
   
3. First some basic ideas. It can be shown that any formula of predicate logic can be written equivalently in the form of a set of sentences with the form:
   A&..&B -> C .. D
   

PROLOG simplifies things by only allowing sentences of the form:
A&..&B -> C
You can think of this as a rule saying that if you can prove A, .., B then you will have C. Or to put it another way, to prove C, you should prove A, .., B. Note also that
-> C
says that C is proven. It can also be shown that
B ->
(where the right hand side of the -> is empty) amounts to saying that not B is proven. So in PROLOG notation you express 'not' by putting a sentence on the other side of the arrow. Finally, in PROLOG, you leave off universal quantifiers: Ux, as they are understood.

1. PROLOG is based on the resolution rule . Here are two examples of the rule in operation:
   A(x) -> C(x)
   C(a) ->
   So A(a) ->
2. A(x)&B(x) -> C(x)
3. -> A(a)
4. So B(a) -> C(a)

The idea is that two matching sentences (C(x) and C(a) in the first example) on different sides of the -> can be cancelled, in which case the variable x takes on the value it would get in the match in the result. (So in the first example, we cancel the C(x) in the first line, and set x to a, obtaining A(a) ->. It can be shown that the resolution rule is the only rule needed for a correct system of predicate logic!

1. A simple problem solving example. We have the following paths:
   

This data can be expressed in PROLOG in the following set of sentences. where 'C' abbreviates 'connects to', and where extra parentheses are omitted for legibility
1. -> Caf
2. -> Cab
3. -> Cbe
4. -> Cbc
5. -> Ccd
6. Cxy&Cyz -> Cxz
Note the last trivial claim about connectedness: if x is connected to y and y to z, then x is connected to z.
a. Suppose we want to solve the problem of how to get from a to d. It turns out that Cad can be proven in predicate logic from 1-6, and the proof provides instructions for getting from a to d.
b. Let us see how the resolution rule turns up the proof. c. Begin by adding NOT Cad to steps 1-6. Our strategy is to prove Cad by showing NOT Cad leads to a contradiction.
7. Cad ->
8. Cay&Cyd -> 6 7 Resolution
9. Cbd -> 2 8 Resolution
10. Cby&Cyd -> 6 9 Resolution
11. Ccd -> 4 10 Resolution
12. > 5 11 Resolution
Line 12 is an arrow with nothing on the right or the left. This is indicates a contradiction in PROLOG. So line 7 has lead to the contradiction on line 12 and we know Cad follows from 1-6. Note also that the data used in the proof: 1. -> Cab, 4. -> Cbc, and 5. -> Ccd amounts to a solution for the problem. In proving there is a connection between a and d, PROLOG has found a pathway from a to d, and so solved the problem.

II. Strengths and Weaknesses of Logic for Cognitive Science
A. Good News: Predicate Logic is sound and complete. A completely rigorous and correct system for predicate logic can be computerized so that any correct pattern of reasoning in the language can be discovered by a computer.
B. Good News: Turing's Thesis: There is excellent evidence that any process that can be expressed with a finite set of rules can processed by a digital computer that operates on representations in the language of predicate logic. So it would seem that any coherent reasoning is always something that is captured by logic programming.
C. Bad News: A fully correct mechanism for logic problem solving may spend way too much time solving problems. (There is strong evidence that logic problem solving requires exponential processing times in a worst case.) For this reason PROLOG has had to sacrifice correctness to obtain efficiency.
D. Bad News: Turing's Theorem: Predicate Logic has no decision procedure. Although good reasoning can always be discovered (eventually) by a logic problem solver, there is no guarantee that bad reasoning can be identified as such in a finite amount of time.
E. Bad News: Godel's Theorem: There is no correct finite system of rules for mathematics.
F. Bad News: Many sentences of English resist representation in the language of predicate logic: John believes that all men are mortal on Sunday but not Monday.
G. Good News: Systems to handle belief, time and other so-called intensional concepts have been developed, and are being adopted by AI researchers. However, many of these systems are controversial, and there are few standards as there are with predicate logic and PROLOG.
H. Bad News: Predicate Logic doesn't let you take it back. If I say that all tigers are striped in the data, then asserting that Tigger is an albino tiger causes a contradiction. (I assume the data includes a rule that says albino means not striped.) From a contradiction anything follows in predicate logic. The problem is that predicate logic uses monotonic reasoning, which means that the more information you have the more you can prove from it. What is needed is a way to add data that removes previous lines of reasoning.
I. Good News: Non-monotonic logics have been developed that are modifications of predicate logic. In these systems you can say 'All birds fly', and then assert 'Penguins don't fly' without causing contradictions. Arrangements are made so that the data 'Penguins don't fly' automatically creates an exception to the rule: 'All birds fly'.
J. Bad News: Predicate Logic doesn't (conveniently) let you handle matters of degree. If I write 'Tall(John)' then I have said that John really is tall. There is no way to say he is sort of tall, or somewhat tall. Similarly you can't (easily) say that the probability that John is tall is 90%.
K. Good News: Many-Valued Logics, Logics of Probability, and Fuzzy Logics allow expression of matters of degree. Fuzzy logics have been found to be quite useful in AI especially in controlling machines.

III. The Psychological Plausibility of Logic
A. Probably the most damaging complaint about the use of logic as a foundation for cognitive science is that it is a poor model of human reasoning. Logic may be a fine standard for good reasoning but is a bad picture of what human beings actually do when they reason. There are two kinds of evidence for this claim.

1. Logic is too time intensive. The computation time for solving the real world problems humans manage to solve on a daily basis is very long. A lot of research in AI has been directed to solving the problem of time-complexity in the use of theorem proving, but simulation of practical real-time reasoning similar to what humans can do is still beyond us. This suggests that Mother Nature found logic too expensive in designing human intelligence. Human reasoning is probably fast and scruffy; and a good thing too, since humans needed to respond to the environment quickly.
   
2. People do not live up to the rules of logic. Here are two famous experiments to illustrate the point.
   a. Wason's selection task:
   A B 4 7 Which card(s) must be turned over to determine whether the following rule holds:
   If there is a vowel on one side, then there must be an even number on the other.
   People rarely realize they must turn over the 7, for if there is an A on the other side the rule is broken. Also people mistakenly say to turn over the 4.
   b. Tversky and Kahneman's work on probability judgements. Mary went to college and was involved in leftist political organizations. Which is more likely:
   
1. Mary is a bank teller
   
2. Mary is a feminist bank teller.
   

B. Response of the Logically-Minded Cognitive Scientist


1. Time complexity problems can be overcome in a massively parallel system like the brain. Poor reasoning of humans does not show that people fail to have logical machinery in their brains. There are just limitations on how they use it: bad memory, poor attention, etc..
   
2. So what if humans don't use logic. They ought to. Cognitive science is the study of intelligence not stupidity.
   
   

Further Reading for Weeks 2-3
Predicate Logic H. Pospesel A good introduction to Predicate Logic. (If you don't already know propositional logic, read his Propositional Logic .)
Logic for Problem Solving R. Kowalski The classic text on logic programming.
Minimal Rationality C. Cherniak Argues that logic is too expensive for humans to use.

1. Semantic nets Nodes and Links
   isa: dog isa canine isa mammal isa animal haspart: dog haspart leg haspart toe haspart nail
   
2. Frames, Scripts
   Labeled slots and fillers often filled with defaults. Also has attached procedures. This makes them like objects in computer science These can be inherited and overridden by instances. Scripts are like frames but also allow representation of standard sequences of events. (Example: Birthday script p. 161 CS)
   
3. Rules
   Information is stored in rule format: If situation then action. Rules may express probabilities.
   

B. Operating on the Data


1. Traversing a semantic net. Is a dog a mammal? Does a dog have toes?
   

2. The Interpreter Cycle for a Production System
a. Forward Chaining
Examine the context to locate the rules that apply.
Decide which rules to fire
Fire the rules and calculate their effects on the context
Repeat.
b. Backward chaining:
Examine goal and locate rules that would produce it.
Decide which rules are good bets.
Set their antecedents (the if parts) as new goals.
Repeat.
c. Both

1. Culling out Multiple solutions. Cost measures.
   Example. There are lots of routes from my house to UH. What is interesting to me is either which is fastest, or which uses least gas.
   
2. Heuristic Search
   Example. In chess, there is a immense space of possible moves, responses, moves, responses etc.. Since the search space is so large, we try to focus attention on likely moves by introducing rules: 1) Do not get your queen out early. 2) Do not get a knight on the side of the board. 3) Place pieces so that they control the center squares. 4) Be willing to sacrifice a pawn to preserve good pawn structure.
   Example: What is wrong with my car. There are so many possibilities so explore those possibilities that can be fixed most quickly, cheaply, or are more likely.
   
3. Blackboard models
   In a blackboard model, we have many processes going at once. When one process needs some information, or the solution to some goal, it posts the request on the blackboard. Then any other process that is free may try to resolve the problem and post the solution when it is found. You can arrange blackboards hierarchically. You can also arrange for "demons" to look over blackboards to see that requests do not use too much resources by considering costs, benefits and likelihood of success.
   
4. Learning (We will discuss this in more detail next week.)
   We can model the idea that rules are acquired from experience (Samuel's CHECKERS program) from other rules (SOAR's chunking). Also decisions about what rules to try first can be made on the basis of how successful a rule has been in the past.
   
   

V. An Assessment of AI
A. Bad News: Representational Power
As we explained in the case of PROLOG, rules do not have the full expressive power of predicate logic.
B. Bad News: Correctness
In logic we can certify that a reasoning process will be correct. There are no similarly straighforward ways to certify the correctness of rule-based systems. For example the project of certifying programs (of reasonable size) is still far beyond us.
C. Good News: Flexibility
You can program rules to be as flexible as you like, for example you can build them with defaults. 'If x is a bird then x flies' can live in harmony with 'If x is a penguin then x does not fly'. The secret is to build a reasoning system that keeps track of which rules override other rules. (Penguins are a kind of bird. So all rules about birds are overridden by any information you have about penguins.) You can also allow the formulation of rules about how to use rules ... rules about rules about rules etc.
D. Good News: Speed through Heuristics
In logic programming, the system blindly searches though every possible solution. In rule based systems it is a lot easier to formulate heuristics: rules of thumb that help shorten the search.
F. Good News: Rules Seem Ideal for Explaining Language
The grammatical structure of language can be expressed as a set of rules (although no one claims to have an account of the full grammar of English). Chomsky has argued that there must be a set of innate rules that govern language abilities called Universal Grammar. If this is right then rule based approaches correctly model a central cognitive accomplishment.
G. Bad News: Combinatorial Explosion
Dennett's account of R2-D1
H. Bad News: Common Sense is Elusive. Why Computers Can't Understand Language. (Terry Winograd's work on airline reservation systems shows how complex language can be. When someone asks for a flight a little closer to 7, they don't want one a little closer, they want one as close as possible to 7!)
I. Bad News: AI has no theory. What have I learned when I create a computer model that does something? All the AI researcher has done is transfer his intelligence to the computer in the form of a program. This may tell us something about what computers are capable of doing, but it certainly does not reveal the details or even the principles of human intelligence.

Further Reading
Winston, P. Artificial Intelligence A clear text on artificial intelligence with thorough accounts of such topics as representation and search.
Winograd, T. Language as a Cognitive Process Provides a good starting point for understanding the classical approach to natural language processing.
Forsyth, R. Expert Systems A collection describing a number of expert systems.
McCorduck, P. Machines Who Think A excellent survey of the history of AI.

1. But there are good reasons for cognitive science not to study individuals, and races.
   
2. One is that cognitive science is the study of features common to humanity.
   
3. Furthermore race, for example, is a superficial and subjective concept and not appropriate for science. Science requires the use of natural kinds, that is concepts that pick out essential features of the world around us. We should try to get rid of any variables that might be irrelevant or might cloud our view.
   

C. What are the fundamental questions in cognitive science. They are ones that we take for granted, but which on second thought are really quite mysterious.


1. How do people understand language?
   
2. How do they reason?
   
3. How do they recognize faces? etc.
   
4. Why doesn't John appear to have grown when he walks towards you? Why does the pen not seem to be deforming when you change its angle of view. We seem to perceive the world in terms of objects . This is a truly wonderous perceptual ability.
   
5. Why do only certain sounds (not, for example, the raspberry noise) of those that we can make with our mouths turn up in the world's languages?
   
6. How can we recognize which words are possible words of English ('blick' is, but 'tliop' is not , despite the fact that 'tl' is a legal phoneme: as in 'battl e'). We seem to know the rule that English words don't start with 'tl'.
   
7. How do we know that a thing that opens is an opener and that many of such things are openers not openser ?
   
8. How do we process structure? For example 'John climbed up the mountain' has a different form from 'John called up his mother'. The proof of this is that we can invert the 'up X' part in the first but not the second:
   Up the mountain John climbed.
   ?Up his mother John called.
   

This shows that 'up' goes with the phrase 'up the mountain', in the first, but 'up' goes with 'called up' in the second. We detect such structural facts without thinking.

1. How is it possible for a human to process an immense variety of English sentences? (There are at least 10 to the 20th power sentences 20 words long.)
   
2. In general: how do we manage to handle novelty in so many domains? In perception, we see something new everyday, but we know how to integrate it into an object in each case. How we handle novelty is one of the deep questions in cognitive science.
   

D. So lets look at the ability to perceive the world as objects. Is this innate or learned?


1. What does the concept of an object involve?
   a. We know the hidden parts are really there.
   b. Two objects can't be in the same place at the same time.
   c. Objects (generally) endure through time, and they tend to keep their properties (even if their apparent size or shape varies as the point of view varies.) 2. In the past, the generally accepted view was that learning explains how we go from a booming buzzing confusion in the perceptual world to a world of objects. 3. Piaget's theory (based on quite sound observation) pretty much accepts this view and describes the stages in the process. 4. But maybe Piaget's conclusions reflect more the nature of his experiments that the true nature of the child.
   
2. Recently cognitive scientists have discovered a new tool. We use infant habituation (boredom) to measure what the infant can and cannot distinguish. Surprise at novelty (versus "more of the same") can be measured by gaze, general posture, and by a sucking response.
   
3. In Elizabeth Spelke's pioneering experiment, a donkey and a horse on adjacent screens were shown bouncing at different rates. A boing sound in sync with one but not the other was presented. Infants attend more to the screen where the boing is in sync, which suggests that infants have a conception of an object as a unity of sight and sound.
   
4. Kellman and Spelke showed 4 month old infants and image of a rod mowing back and forth but partly hidden by a rectangle. In one condition, the rectangle is removed to show a rod, and in the other one sees two rod fragments in coordinated motion (with nothing where the rectangle used to be). The first condition excited very little attention, but the second was highly attended to, suggesting that the discovery that there was no unified occluded object was quite surprising.
   
5. Renee Baillargeon showed 5.5 month old infants a "drawbridge" apparatus. In the "possible" condition, the child sees the object and then the drawbridge occludes it and stops at the point where it would collide with the object. In the "impossible" condition, the bride closes 180 degrees as if the object behind it weren't there. There was much more attention to the second condition.
   
6. Baillargeon also showed infants short and tall bunnies occluded by a u-shaped mask. The short bunny was too short to appear in the window (between the two legs of the u). and went from left to right behind the mask. The tall bunny was tall enough to appear in the window, but when it moved from left to right, it did not appear in the window. Infants attended much more to the tall bunny.
   
7. All of this suggests that very young infants already perceive a world of objects, and do not learn to do so. Some of these experiments have been replicated with 1 or 2 day old infants
   

II. Concepts: What are They?
A. Philosophers have always analysed concepts: Plato: What is justice?
B. And wondered whether they are inborn.

1. Rationalists (Descartes, Leibniz) tend to think (some) inborn concepts are a prerequiste for knowledge. (Descartes thought the concept of God was innate.)
   
2. Empiricists (Locke Hume) tend to think that concepts are the product of learning and so needn't be inborn.
   

C. Most cog scientists are very interested in how concepts can be learned, assuming that most concepts are not inborn.
D. Classical accounts of concepts:


1. Semantic nets: what makes a node the concept for DOG is its position in a complex net - where it fits in the hierarchy (or web) of other concepts. Concepts are used in the process of exploring the semantic net where it is found.
   
2. Frames, Scenes, Scripts: what makes a frame the concept DOG is the complex set of (labeled) structures related to it. Concepts are used when slots in these structures are filled in so as to apply them to a specific task. (C p. 159 for a sample Frames; p.161 for a sample Script.)
   
3. The Theory-Theory. The ideas in 1 and 2 can be generalized: A concept is any symbolic structure defined both by its role in processing and by its place in a hierarchy of other concepts. This is sometimes called the theory-theory of concepts. Concepts are like the terms of a scientific theory. (Consider F=ma. What is mass? - The thing that plays a role in the theory of physics.)
   

E. Prototype theory of concepts: A concept is not fixed by a definition but by a prototype, a typical member of the class.
F. Connectionist accounts of concepts: A concept is a region in a space of possible features.
G. Philosophical Thoughts about Concepts (or Meanings)


1. Meanings of concepts are derived from meaning of other concepts: Eg. LOVE, UNITY (Compare: the meaning of a word is its definition, the meanings of the words in its definition defined by their definitions, etc.)
   
2. Meanings of concepts are derived from perceptual observations: DOG, MOTHER
   
3. Meanings of concepts are derived from their role: NOT, ALL
   

III. Puzzles about the Classical View of Concepts.
A. We never seem to be able to give the definition (rules) for any concept. Try: A tiger is a large, striped feline. But there are well known counterexamples to the definition: the albino tiger. It seems that what makes something a tiger is not the presence or absence of any one feature but something more like a cluster of features (Consider Wittgenstein's thoughts about family resemblance.)
B. Prototypes vs. Rules.

1. Some psychological evidence (though not all) suggests that protoptyes are a better model. (eg. 'A robin is a bird' is more quickly processed than 'a penguin is a bird'.) The definition approach does not explain our judgments of what is and is not typical.
   
2. Then again, concepts like 'is drunk' needn't be applied on the basis of similarity to a prototype, but via a process of inference based on rules. (This guy jumped in the pool with his clothes on. That's distorted judgement, and being drunk does that, So he's drunk.)
   

C. If having concepts is a prerequisite for thought, and thinking is needed for learning (for example evaluating alternative hypotheses about which concept is usefully applied to the world), then how could we ever acquire a new concept? The classical theory seems to have only one explanation for concept learning. New concepts are composed from combining old ones. But then it seems we could never learn a genuinely new atomic concept.

IV. Learning
A. Which concepts are innate or to what extent are they innate? An how do we explain the acquisition of ones that are not innate?.
B. Techniques for acquiring new concepts: a) combinations of old concepts b) combination of features c) extraction of patterns in raw data (like inductive generalizations) C. Learning is the acid test for AI models, for often good performance is simply due to the fact that the desired ability is simply programmed in. To display the flexibility of human intelligence, you need a model that can generalize from what it has already achieved to resolve novel problems
D. Classical learning techniques. A historical overview.

1. Samuel's CHECKER learns two ways. Rote memorization of good/bad board positions (boring) and adjusting the evaluation function for board positions on basis of their appearance in a sequence of moves that eventually won a game.
   
2. Winston's ARCH-LEARNER generates and tests usefulness of new concepts. Uses hits and near misses to refine concept boundaries. (C. p. 158 top a-d)
   
3. Quinlan's ID3 learns concepts from positive and negative instances by creating a decision tree for classifying objects. Algorithm used to order the questions in the tree (most broadly discriminating first).
   
4. Fisher's COBWEB learns concepts by clustering features of examples and calculating a classification tree. It is unsupervised, that is not told which are positive and negative instances of concepts. It just looks at the class of all instances and clusters them.
   
5. Newell's SOAR uses chunking of problem solving steps (knowledge compiling) to build a toolbox of useful problem solving methods.
   
6. Connectionist models: To be explained in a few weeks
   

E. Problems:


1. Getting the right atoms or features
   
2. Developing genuinely new concepts, (the new term problem)
   
   

1. -> Caf
2. -> Cbe
   
3. -> Cbc
   
4. -> Ccd
   
5. Cxy&Cyz -> Cxz
   

18. What difficulties surface in trying to represent in Predicate Logic such sentences as: 'John believes that Mary believes that philosophers are right on Thursdays under a full moon.'
19. Why is 'Tigger is an albino tiger' a problem for theories of intelligence based on predicate logic? How might the use of defaults or non-monotonic logic help resolve these problems?
20. Describe advantages and disadvantages of using predicate logic to explain cognition.
21. Cite psychological experiments that suggest that predicate logic is not a good model of how humans actually reason.
22. People do not perform very well on Wason's 4-card task. What factors tend to improve performance? What does this suggest about the nature and origin of human reasoning abilities?
23. Tversky and Kahneman's work shows that many people (given information that Mary was a leftist in college) will judge that it is more likely that Mary is a feminist and a bank teller than that she is a bank teller. This of course violates a fundamental rule of probability: A and B can never be more probable than A. What does this show about the nature of human reason? Does it demonstrate that probability theory is irrelevant to cognitive science? What would a classical theorist say about this?
24. What are some good reasons for abandoning logic programming and resorting to more general rules and representations in cognitive science?
25. Evaluate the predicate logic approach to cognitive science along the following 5 dimensions: representational power, computational power, psychological plausibility, neurological plausibility, and practical applicability.
26. What was Dennett's point in describing the robots R1, R1D1, R1D2?
27. Describe the abilities and limitations of the following AI programs: CHECKERS, SHRDLU, MYCIN, SOAR, HYPO, CYC.
28. How does MYCIN (or HYPO or SOAR) work?
29. Describe semantic nets and frames, citing points of similarity and difference. What are limitations of these forms of representation?
30. What is the difference between forward and backward chaining? Explain with an example.
31. What is heuristic search? Explain with an example.
32. What are strengths and weaknesses of the rules approach to artificial intelligence? Evaluate the approach along Thagard's 5 dimensions.
33. A major problem in artificial intelligence research is to provide methods for usefully storing vast amounts of relevant knowledge about the world. Explain this problem with an example.
34. Why would exploration of differences in innate abilities in the population be a misguided approach to the topic of innateness in cognitive science?
35. Cite some of the things that we all know and take for granted about language, which nevertheless correspond to very sophisticated cognitive abilities.
36. Describe some of the basic aspects of the notion of an object that have been explored by psychological experiments with infants.
37. Describe one of the experiments given in class which suggests that the concept of an object is innate. How does the data support the conclusion? Can you think of any criticisms of the experiment?
38. Piaget believed that the concept of an object was pretty much absent in infants for the first 9 months. His evidence was that infants do not display an ability to manually search in the right direction once an object is occluded. What alternative explanation of this behavior can you give?
39. What is the theory-theory of concepts? Explain with reference to how the concept of mass is defined in science by such formula as F=ma.
40. What reasons are there for rejecting the idea that concepts are like definitions in the dictionary?
41. What evidence supports the idea that concepts are prototypes? What evidence undermines the idea?
42. Explain the major problem classicists face in explaining how we learn genuinely new concepts.
43. Explain three different methods whereby cognition might acquire new concepts.
44. Give an account of the history of the development of artificial intelligence programs that learn, explaining the variety of methods that have been explored. You should cite the contributions of at least three of the following programs: CHECKER, ARCH-LEARNER, ID3, COBWEB, SOAR.
45. What are the hardest questions to resolve in developing artificial intelligence programs that can learn? Give an account of at least four different approaches to the problem.
46. Steven Pinker says that artificial intelligence reveals that the easy problems are hard and the hard problems are easy. Provide examples to illustrate his point.

1. The brain contains at least 10^11 Neurons. Each of these consists of a Soma (cell body) Dendrites (inputs) and an Axon that terminates in Synapses (Output) (p. 276).
   
2. The brain also contains 10^12 Glial Cells that cover axons with myelin (a fatty insulator), absorb neurotransmitters, dispose of dead cells, etc..
   

B. Forebrain,


1. Cerebral cortex (or Cortex for short)
   a. Approximately six layers spread over about one square yard
   b. Its all crumpled up in the shape of a walnut, with two sides joined by the corpus collussum.
   c. Top surface grey matter = cell bodies (soma) underneath is white matter = axons covered with myelin, which is white.
   
2. Limbic System (Concerned with emotion and motivation)
   

C. Midbrain


1. Thalamus Lateral Geniculate nucleus (Visual Way-station)
   

D. Hindbrain


1. Cerebellum (Fine tunes motor control - provides smooth coordination)
   

III. How Neurons Work
A. Ion pumps (Potassium Sodium) in the cell wall maintain a difference in charge (called a membrane potential) across the cell membrane, so that the inside is negative.
B . But there are channels that can open to let positive ions back into the cell (or let negative ions out) and cancel the negative inside charge near the channel. This is called depolarization of the cell wall.
C. When channels open at the root of the axon (the axon hillock), the reduction of the charge difference (membrane potential) causes neighbor channels to open as well. This causes a cascade of openings (depolarization) down the axon all the way down to its synapses.
D. At the synapse, the change in charge causes little sacs full of neurotransmitters (called synaptic vesicles) to open into the cell wall, exposing the cell wall of the neighboring neuron's receptor sites to the neurotransmitter. Depending on the neurotransmitter and receptor site, the presence of the neurotransmitter may inhibit or sensitize the neighbor neuron to possible future depolarization.
E. The effects at all the synapses of the neighbor neuron add together. If there is enough over all activity at its axon hillock, the channels there will depolarize and the neighbor cell will fire.

IV. Neural Plasticity
A. During early development neural structure is often formed by the elimination of excess neurons and synapses.
B. The development of structure depends on the stimulation the brain receives, and when it occurs. If a sighted child is blindfolded during the critical period for creation of sight structures, the ability to see will have great difficulty developing. The same sort of critical period appears in the case of the recognition of phonemes (language sounds) and the ability to process grammatical structure.
C. If a child loses cortex normally devoted to such functions before the critical period there is a good chance another region of cortex will take over. So the brain is plastic at an early age.
D. However after a critical period, lost of the relevant part of the brain means that the ability cannot be restored, or is restored with great difficulty.
V. Brain Regions and Topographic Maps
A. In a normal brain, there are standard locations in cortex of the basic functions (although there are some variations as well). Here is a crude picture of a left hemisphere:



B. Motor, sensory, auditory and visual cortex are all arranged in topographic maps. This means that regions in cortex correspond to regions of the body, the retina, or the cochlea. For example, parts of sensory cortex respond to stimulation of the palm, and nearby ones to the thumb etc. (See p. 299 for a sample map). In auditory cortex, some neurons are devoted to low pitch, and their neighbors to slightly higher pitch and their neighbors to pitches higher still etc.. An area such as visual cortex may have many different topographic maps devoted to different functions, such as general shape detection, motion, and color.
To some extent, the specific regions dedicated to a given sensory region vary depending on much stimulation is received there. So the brain is still somewhat plastic at the micro-level.
VI. Neural Representation
A. Is there a grandmother neuron, a neuron that fires when I see a grandmother? For that matter, is there a neuron that fires when I see a pure blue visual field? Almost certainly not.
B. Brain representations are distributed across many neurons. So the representation of my grandmother is no doubt the combination of many many neurons coding for lots and lots of features that make up my grandmother experience: color of hair, facial shape, gait, sound of voice, etc..
C. Neural representation often uses what is called coarse coding. We illustrate this in the case of color vision. You might think that there are neurons that are responsive to particular wavelengths of light, say neurons for 500 nanometers, for 510 nanometers, etc..
D. But color vision depends on the fact that we have 3 different kinds of cones (sensory neurons) (called S, M, L) that respond somewhat differently to color. These cones have a very large region of wavelength overlap so that for most colors, all 3 kinds of cones are active at least to some degree. The representation of the color red, for example, corresponds to a characteristic amount of activity on the S, M and L cones. (So there really aren't any red green or blue cones as some popularizations would have it .) Green has its own pattern of activity, and so on for the other colors. This means that a color sensation is represented as a triple of numbers indicating the activity of S, M, and L. This kind of coarse coding is surprisingly efficient.
E. A similar representation is used to code tastes, but here there are 4 not 3 styles of tasters neurons (roughly for salt, sweet, sour, bitter).
F . For another example, the direction of a target object is coarse coded as a collection of activities on various neurons. How can this information be used by the brain to control the arm to grab the object? Is it ever averaged together in one place in the brain? Probably not. We just send the raw collection of directions in parallel to motor output to control the position of the arm. The slightly contradictory muscle movements will average out in the arm, and you will get the job done.
VII. Neuropsychology
A. Neuropsychology is the psychological study of how the brain carries out specific cognitive functions. This is mostly unknown, but new methodologies offer hope, and some older ones have already given some information.

1. MRI (Magnetic resonance imaging) PET (positron emission tomography) and ERP (event-related potentials) scans can now give us an inside look at the brain during different cognitive activities. We can see regions light up when we ask a subject to imagine a visual scene, do math, or understand language.
   
2. Study of the loss of function in people with damaged brains (stroke, wounds, tumors, etc.) has contributed a lot. In the old days all we could do is autopsy people with damage after they died, but now we can also use a variety of scans to see where the damage is without opening the skull. Furthermore, in the case of operations for epileptic seizures, the skull has to be opened anyway, and then neurons can be stimulated, and the result reported by the patient. (It doesn't hurt because there are no pain sensors in the brain.)
   

B. Some results on memory. The study of the patient HM supports the theory that there is a distinction between long term and short term memory. (It seems that HM lost his ability to lay down new memories in long term memory.) Furthermore, since HM can learn new skills despite not remembering any training, his case supports the distinction between declarative memory (memories for facts) and procedural memory (memory for how to do things).
C. Some results on language. The neurological study of language his revealed that language is primarily processed on the left hemisphere (for right handed people and many lefties as well). Two areas (at least) are specialized for linguistic functioning. Broca's area (a specialization of motor cortex) seems to be involved in production of language. When it is damaged, patients can understand, but have great difficulty producing language. Wernicke's area (a specialization of auditory cortex) seems to be related to speech understanding. For patients will damage here, language can be produced, but it often makes no sense.
D. Left Brain, Right Brain. Sensory inputs are reversed so information from the left goes to the right hemisphere and vice versa. In humans the two hemispheres seem somewhat specialized for certain functions. Left brain: language, manual dexterity Right brain: visual and spatial understanding.
E. Split Brain Studies. If you cut the corpus collussum, the main pathway between the two hemispheres, the patient seems normal, but under special conditions, interesting defects become apparent. Localization of speech in the left hemisphere is confirmed. Bizarre results can occur that undermine our conception of these patients as having a single consciousness. Example: a snow scene is presented to the right-arm hemisphere and the patient is asked to select one of four pictures as the one most closely related. The right arm points to a picture of a snow shovel. The left arm hemisphere is presented with a picture of a chicken and did not see the snow scene at all. When asked why the right had has just selected a snow shovel, the verbal response is: to shovel out the chicken coop. The left arm hemisphere considers itself to be in control of the right arm, and manufactures an explanation for the decision which we know is a fabrication. (Is the idea that we are in control a fabrication even for normals?)

Further Reading

Mind and Brain Scientific American Special Issue, September, 1992. [An excellent collection of introductory articles on the brain. The papers on vision (p. 68) language (p. 88) and neural net learning (p. 144) are especially relevant.]

Churchland, P. The Engine of Reason, the Seat of the Soul [An enjoyable book. See especially Ch. 2 on sensory coding, and Ch. 7 on brain defects]

Hardin, C. Color for Philosophers [This is an excellent introduction to the surprising facts of color vision, with interesting philosophical morals as well.]

Kosslyn, S. and Koenig, O. Wet Mind [This is an excellent and accessible account of cognitive science with strong emphasis on neurlolgy and neuropsychology. Very good on brian lesions and corresponding cognitive deficits.

Kuffler, S. et. al. From Neuron to Brain [A fine textbook on neurology, though not much on cognition]

1. Processing is parallel and local. Local means that processing at one point of the image depends for the most part only on activity of nearby points.
   
2. Processing is modular. There are different maps of the visual system especially designed to resolve different parts of the problem. Examples: edge detection, motion, depth. Each of these may be further subdivided. For example, for depth information, we have several different systems based on different sources of depth information: stereopsis, the differences in the images on each eye; motion, as object recedes or advances the "size" of object changes.
   
3. Success of processing depends on certain regularities about the environment. For example, using apparent size to tell whether objects are advancing or receding won't work if objects can increase or decrease their sizes like balloons. Since so few objects that humans encounter do this, the visual system has come to depend on the assumption that objects stay pretty much the same size.
   

III. David Marr and the Primal Sketch
A. Edge Detection. On Marr's theory, the first job the vision system has to do is detect edges or object boundaries. Usually an object boundary corresponds to a quick change in intensity, however there can be intensity changes within the object boundary and sometimes boundaries do not correspond to a noticeable intensity change (green book on a green table). So we will need to supplement this idea in the long run.

1. For starters, intensity changes can be computed by taking differences between adjoining pixels in the image. (For math mavens, this is the first derivative.) If we look at changes in those changes (the second derivative) areas where there was a change in the original will correspond to zero activity (zero crossings) in the new image. The new image will look something like an outline version of the old one. (p. 472, image (b)). Since neurons calculate weighted (think synapse) sums (think axon hillock), it is not difficult to imagine arrays of neurons providing the second derivative map.
   
2. The Mexican Hat Trick. But how can we design clusters of neurons that will make use of the zero crossing representations to pick out lines of various orientations? For we need to do this if we are ever going to represent the contours of objects.
   

On-Center and Off-Center cells. Two useful kinds of cells (than can easily be created by adjusting the weights between the neurons in the right way) are called on-center and off-center cells. On-center cells receive signals from many adjacent cells in a roughly circular pattern from the zero crossing map. The degree to which they fire depends on what happens in their circular patch is described by the Mexican hat function (p. 471). Activity on the central portion of the patch causes the on-center cell to fire, but activity around the edges inhibits firing. So if there is even illumination across the whole patch, the response will be a standard firing rate. If the center only is active, the on-center cell increases its activity above normal, and if the ring around it only is active that cell is active less than normal. If activity is suppressed across an area on the brim of the hat but not the center, then the inhibition will be weakened and the cell will fire more. However if it is suppressed over the center and a portion of the brim, then the positive center will be inactive leaving the remaining inhibition on the brim to cause the cell to fire less than normal.

On-Center Cells

1. Detecting and oriented edge. Now imagine that whole rows of off-center and on-centers are lined up in rows. If a zero-crossing lines (line of active on one side and inactive on the other) up just right all the cells in that row will fire, and no other cells will. We have a perfect oriented line detector!
   

1. By using the same trick for larger or smaller sized patches we can determine how sharp the edge is that we are detecting. (Smaller patches detect sharper edges.)
   

IV. Marr's Theory of Higher Level Representation
A. (the task is to explain how the brain can go from information about:
Contrast, 2-D Velocity, Disparity, 2-D Orientation to a 3-D world of objects involving:
Color, Texture, 3-D Shape, Distance, 3-D Trajectory
B. Levels on the Theory

1. Grey level (raw output of the photoreceptors)
   
2. Zero-crossing Representation (provides edges)
   
3. Primal Sketch (provides oriented edges, which represent the boundary contours of any object)
   
4. Boundary Features (provides information about such features as blobs bars and ends)
   
5. 2.5 D Sketch (provides depth information)
   
6. 3D Sketch (provides a three dimensional model of the world, which according to Marr is done by representing the object as a nested set of more and more complex cylinders: Human: Head Body; Body: Trunk, Arms, Leg; Arm: Upper-Arm, Forearm, Hand; Hand: Palm, Fingers
   
   

V. Treisman's Theory of Attention and Primitive Processing
A. The thesis is that there are basic visual processes that are computed in parallel that feed information to higher level processes responsible for binding features together. These second processes are calculated in series by an attention mechanism.
B. We can develop evidence fro this theory by presenting images with target shapes surrounded by distractors. If we measure the reaction time for identifying the targets and discover that it is fast and does not depend on the number of distractors, then we assume it is a basic parallel process. If the reaction time grows with the number of distractors then we assume the process is serial and involves attending to one thing after another in the scene.
C. For example, the letters L and T have the same elements in the same orientation, and differ only in how the elements are conjoined to each other. Recognition of these targets depends on attention. The differences between them do not just obviously "pop out". However if you examine a field of |s and /s, where the only difference is the orientation, the difference is immediately and easily apparent.
D. Basic features include orientation, brightness, curvature. A discrimination that requires conjoining features (white triangles and black squares, vs. black triangles and white squares) is extremely difficult to discriminate and takes tedious one-by-one inspection.

VI. Biederman's Theory of Higher Level Processing
A. Biederman's theory describes the features of something like Marr's 2.5 D level, but the primitives he postulates (called geons) are more flexible and varied: cones, cylinders, cubical shapes, etc and distortions of these such as changing the length to width ratio and the shape of the center line.
B. Biederman believes that objects are identified by a process of recovery by components. Each of the components is recognized, and their identity and arrangement allows us to tell what kind of object we have.
C. But how are the components recognized? By the boundaries between them, which are typically concave and rather sharply sloped inwards. (Consider the "joints" of the Michelin man, for an exaggerated version of the idea.)
D. Some evidence for these ideas is found in how well we recognize line drawings where parts of the scene are obscured. If the points of connection between parts (apexes of triangles for example) are removed, the scene is hard to recognize. If other parts are removed, but the connecting parts left, recognition is fairly easy.
E. The fact that objects viewed from strange angles where the "joints" are obscured are hard to recognize (p. 499) is a variant of the same line of evidence.

VII. Top-Down vs. Bottom-Up
A. Marr and many other researchers have tried to create theories of vision where the processing from retina to brain does not require higher-level information to identify the object. (For example, concepts like animals have 4 legs, or that the sky is above us and is blue or grey, etc.)
B. Clearly there are instance where higher level information is required to resolve the ambiguities in a scene. For example the same shape (N) can be read horizontally as an en, and vertically as a zee.
C. But to what extent does vision rely on top-down processing? Consider the Kaniza Triangle. Here we see an image of a triangle hovering above the scene, but there is no luminance differences on the two sides to allow us to pick out the boundary. Why do we perceive the edge? Perhaps conceptual information about how other things blot out other shapes helps us. However, there is some evidence that this phenomenon is very low level. For example we have evidence that the "edges" are already processed in V2, the next center down from the earliest: V1. So maybe a bottom up explanation is going to be more likely.
D. Another piece of evidence that low level processing is doing most of the work is the fact that a zero-crossing representation of a set of letter Bs obscured by blobs allows us to recognize the Bs much better than a blob representation. Perhaps the zero-crossing is really doing most of the representational work.

VIII. Further Topics on Vision
A. The Up-down axis is used as a major default assumption about how objects are aligned. Violations of this alignment cause errors in identification. (A problem faced by NASA designers of space stations, where expectations about up-down are more often violated.)
B. The alignment approach to object recognition is a competitor with theories based on recovery by components. One basic idea is that objects have conspicuous alignment points. These can be used to rotate and rescale the image to see if it matches stored representations from a "standard" point of view.
C. There is a major division in visual pathways between the what system (object recognition) and the where system (object location). These correspond to different topographic maps on cortex. It is possible that the recognition system is narrower view-window that the whole visual field, so that an attention mechanism must be used to move access to this processing from one element to another in the scene. We typically do not recognize all objects in a scene at once.
D. Eye motions (saccades) that flit from one spot to another in the scene are essential to effective vision. The detection of the motion of the visual scene across the retinal array is suppressed during a saccade. This is one form of visual attention. There is another form that operates even if our direction of gaze is fixed. This serial process may be involved with binding properties together in the same object. Binding errors can be induce by presenting scenes very quickly.

1. Implementational connectionists will view their role as explaining how symbolic processing is implemented in the brain. They pretty much accept the classical account, and attempt to explain how the processing described by classicists could be carried out in the brain's neural nets.
   
2. Radical connectionists view their theories as competitors to classical ones. The idea is that classicists have an incorrect theory about what cognition is like, and that connectionism can replace it with a much more adequate view. Naturally, radical connectionists and classicists have engaged in hot debate.
   
3. Hybrid connectionists think that connectionism best describes only some of our cognitive abilities, notably those in perception, pattern recognition, and motor control. Classical theories are needed to explain other abilities such as reasoning and language. So hybrid connectionists are radical for some abilities and implentational for others.
   

II. The Basics
A. Connectionist models are known by many names: (artificial) neural nets, parallel distributed architectures, subsymbolic models.
B. Units. Connectionist models are connected networks of simple processors called units. The units are supposed to model the basic behavior of neurons.
C. Weights. The synapses which regulate signals between neurons are modeled by values called weights. Weights can be positive (indicating that activity at the synapse encourages the neighbor neuron to fire) or negative (indicating that activity at the synapse inhibits firing by the neighbor neuron).
D. Activation Function. It is assumed that all units calculate the same very simple function. The fundamental idea is that the unit i sums the signals it receives from each of the neurons connected to it. The signal aj coming from each unit j connected to i is multiplied by the weight wij between i and j. The sum of these values for each connected unit is calculated. This value might be any positive or negative number. But a neuron's activity is best modeled as a number between 0 (inactive) and 1 (maximum firing rate). So we adjust this sum so that it lies between 0 and 1 with sig, the logistic (or sigmoid) function. (See p. 69 (b) for its graph.) Putting these ideas together, we get the basic activation function for units.
ai = sig(Sj wij aj) where sig is the function: sig(n) = (1+e-n)-1
This says that the activity of unit i is the result of multiplying the activity aj of each input neuron by the weight connecting it to i, and then applying the sigmoid function to this sum. Connectionists assume that all cognitive processing results from the behavior of many units all of which compute this function or a minor variant of it. Note that any possible arrangement of connections of such units can be expressed by simply setting wij to zero for any two units that are not connected. Therefore the architecture and behavior of the neural net is defined entirely by the weights between the units.
III. Standard Feed-Forward Architecture
A. Many connectionist models conform to a standard configuration called feed-forward nets. There is a bank of input units which contain the signals coming into the system, a bank of output units, recording the system's response, and usually one or more banks of hidden units that are waystations in the processing. In a connectionist model of a whole brain, the input units model the sensory neurons, the hidden units the interneurons, and the output units the motor neurons.
B. The astonishing thought behind this model is that all the brain does is simply the result of massively many units calculating the activation function according to the settings of the weights (the synaptic connections). Could such a simple-minded calculation really do the job? There is a lot of intriguing evidence that it may.
IV. Recurrence
A. Feed-forward Architectures are limited in what they can do. The signal flows directly from input to output. However we know that the brain contains recurrent pathways, that is, pathways that loop back to earlier levels.
B. Winner Take All Arrays. One use of recurrence in connectionist models is to provide for mutually inhibiting banks of neurons. Each bank sends inhibiting connections to the other bank, with the result that only one of the banks can be active. These arrays can be applied to problems that involve parallel constraint satisfaction. Such nets can model decisions between incompatible alternatives (M. p. 116), for example, the two ways of viewing the Necker cube. Marr and Poggio have used the idea to model how the brain matches up images from the two eyes to facilitate stereoscopic vision. The same kind of models can be used to understand decision making, planning, and explanation (M p. 115-117). C. Simple Recurrent Architectures. In simple recurrent architectures, information on the hidden units is sent back to the input level, so that information about the hidden units at time t-1 is available at the inputs at time t. This provides for a kind of short term memory, and is essential for processing where the net needs to respond to the history of the inputs. Such nets have been shown to be capable of simple grammatical processing.
V. Learning
A. The success or failure of a neural net model depends on the selection of the right weights. But how can we determine which weights we need to accomplish a certain task? One solution to the problem is to let the net figure it out. Let the presentation of the input and its response adjust the weights. There are two basic styles of learning in connectionist models: unsupervised, where the net simply adjusts the weights on the basis of the inputs it receives, and supervised learning where the adjustment is done. Descriptions of the most famous unsupervised (Hebbian) and supervised (Backpropagation) learning methods follows.
A. Hebbian Learning. The idea goes back to Donald Hebb. Put information at the input units, and calculate the activity of all the units. Then increase the weights between active units, and decrease those between inactive units. Do this for all the inputs that the net will encounter. This process will cause the net to classify regularities found in the input. For example, imagine that the inputs code for different features of animals: fur/feathers, 2/4 legs, forward/sideways facing eyes, sharp/blunt teeth, wings/no wings, carnivore/herbivore. Now train the net with features found in animals at the zoo. Weights between such features as carnivore, forward facing eyes, sharp teeth, will get strengthened. Also those between feathers, 2 legs and wings. The net has "discovered" the concepts "bird" and "predator". When features for a new animal are presented it will activate the units that represent the closest category to which those features belong. It is almost as if the net has extracted some prototypes from the data which it can apply to novel inputs.
B. Backpropagation. Backpropagation is the most popular form of supervised learning. We will illustrate with the example of a net trained to pronounce English words. The spelling of a word is put on the inputs, and a code for its correct pronunciation is to be presented on the output. This task is hard because of the irregularities of English pronunciation: 'have' does not rhyme with 'came' and 'same'; 'though' does not rhyme with 'rough' and even 'tough'. The training set will consist of a list of words together with their correct pronunciation codes. Training proceeds as follows. Start with random weights. Now present the first word in the training set, and calculate the activities of all the units. The output units will almost certainly not match the desired code for that word. For each output unit, trace the source of the error back through the network. Adjust weights (slightly) in the direction that will correct the error. Now do the same thing for the next item in the training set, and so on.
VI. Connectionist Representation
A. In local representation, single units are devoted to recording a concept. (Think grandmother neuron.)
B. In distributed representation, the representation of an item consists of a pattern of activity across all of the units. Nets trained with backpropagation and Hebbian learning spontaneously generate distributed representations of concepts they are learning. For example, a cluster analysis of the activation patterns on the hidden units of NETtalk shows a hierarchy of clusters and subclusters corresponding to phonetic distinctions. There is a main clustering into two: vowel, consonant. And within the consonants subclusters for voice or unvoiced, etc.. In learning the task, the network has acquired the concepts that it needs to process the inputs correctly.
C. Distributed representations in connections models correspond to extremely complex arrays of values across many units. Therefore the representation for a concept like [cat] can code of lots of features of the concept such as mammal, pet, furry, aloof, stalks-mice, and other features (like how it looks) that we would be hard pressed to describe in language. This so called subsymbolic form of representation allows the symbol to carry its own information about what it is about. The symbol is not arbitrary and atomic the way a word in a language is. By analysing the symbol, you can find out what it "means".
VII. Famous Connectionist Models
A. Connectionist models have been used for such divergent tasks as recognizing submarines, deciding bank loans, and predicting protein folding, to name just a few. What follows are a few of the better known connectionist models trained by backpropagation.
B. TRACE: Rummelhart and McClelland (1986) Predicting Past Tense of English Verbs
*Input: Phonetic code of present tense verb (sing)
*Desired Output: Phonetic code of the past tense of that verb (sang)
*Architecture: Feedforward net without hidden units
*Training Set: Phonetic codes of present and past tense of 460 English verbs
*Results: The net learned he past tenses of the 460 verbs in 200 rounds of training, and it generalized fairly well to new verbs, with good appreciation of "regularities" to be found among the irregular verbs (send / sent, build / built; blow / blew, fly / flew). During learning as the system was acquiring more regular verbs, it overregularized: (break / broked). This was corrected with more training. Children are known to exhibit the same tendency to overregularize. Whether this is a good model of how humans process verb endings is a matter of hot debate (Pinker & Price 1988).
C. NETtalk: Sejnowski and Rosenberg (1987) Pronouncing Written Text
*Input: 7 letters of the text (including space) in a moving window
*Desired Output: Phonetic code for the center few letters, which is sent to a speech synthesizer
*Architecture: Standard 3 layer feed-forward net. (80 hidden units)
*Learning: A large training set of text coupled with its phonetic transcription.
*Results: During learning the system goes through stages of babbling, double-talk, and finally intelligible speech, (with some accent). Generalization to novel text is good. Statistical analysis shows that hidden units use a distributed representation of basic phonological features.
D. Elman (1991)
* Input: Words drawn from a small set of English words (23 words plus End-of-Sentence) coded in 1s and 0s.
* Output: One output unit for each each word in the set.
* Architecture: Simple Recurrent Net
*Training Set: Grammatical sentences of from this vocabulary for a brand of English restricted to a small subset of its grammatical rules. The grammar did, however, provide for a hard test of grammatical awareness: subject-verb agreement across arbitrarily long relative clauses:
Any man who hates women who hate men .. also hates feminists.
*Desired Output: When a word from the sentence is applied to inputs, the desired output is the next word in the sentence. (Of course the net can't possibly succeed at this task.)
*Results: Nets were trained to be extremely accurate in the following sense, on the presentation of a sequence of words, all and only words that would be legal continuations at that point are active beyond a certain threshold at the output. When a word is presented that violates the rules of grammar no words reach threshold at the output. The trained net came very close to this desired performance.
VII. Attractions of Connectionist Models
A. Biological Plausibility. Neural net models "look like" the processing that we find in a brain, especially when we look at the processing we know about: sensory input and motor output. There is evidence for Hebbian learning at synapses. The 100 step rule would suggest that the brain's processing, unlike the usual classical models, is highly parallel.
B. Soft Constraints. Nets can learn to appreciate subtle statistical patterns that would be very hard to express as hard and fast rules. This allows them to avoid the brittleness displayed by classical models.
C. Fast Processing of Multiple Constraints. Nets can quickly resolve in parallel the complex set of conflicting forces to make a decision.
D. Graceful Degradation. When units are lost, the net behaves almost as well. In classical systems the loss of a circuit typically causes a fatal processing error.
F. Flexible Response to Noise. When the inputs are noisy (if part of the input is inaccurate or obscured by some other signal) nets respond appropriately (though somewhat less accurately).
G. Vector Representation. There is evidence that the brain is deeply committed to representations in the form of vectors (arrays of values). For example, coding for color and taste are both by vectors of 3 and 4 values. Neural net architectures are perfectly designed to handle vector processing.
H. Prototypes. Connectionist classification methods remind us of the idea of matching inputs to prototypes. So evidence for prototypes helps sway us toward connectionism.
I. Unified Theory of Learning. Classical accounts employ a variety of different learning techniques. Connectionists have a simple and fairly unified theory of learning based on backpropagation and Hebbian processes.
J. No Programming. We do not need to hire programmers to make a system do some complex task. Just build a training set for the desired behavior and let the model learn what to do.
VIII. Weaknesses of Connectionist Models
A. Biological Implausibility

1. Neural nets are too simple to capture the brain's processing. For example, they leave out neurotransmitters, and the spikiness of neural firing.
   
2. Backward Connections. Backpropagation requires signals be sent both forward and backward in a neural net. But there seem to be few if any backward neural connections. (However, if units represent groups of neurons, the backward pathways are compatible with what we know about neurology.)
   
3. Slow learning. Connectionist learning is slow, requiring hundreds of thousands of presentations. But people learn some things from a single example.
   

B. Processing Limitations


1. It is wrong to think that classical models cannot express fuzziness, or resolve multiple constraints quickly. Although may classical models are serial ones, this reflects more the nature of the computers available to the modelers, not their intentions about what is being modeled. Massively parallel systems are perfectly compatible with the rules and representations approach. However, classicists have argued that there are serious limitations to what neural nets can do if they do not merely implement classical techniques. The limitations tend to concern higher level processing such as reasoning and thinking
   
2. Systematic Processing. It would seem that symbolic processing is required to carry out certain kinds of reasoning operations. For example, to reason generally from A&B to A, I need to represent A&B as containing A, &, and B, so that I can drop off the A part. This suggests that at some level or other the brain must represent the constituents A, &, and B of A&B, and apply a rule that is sensitive to these parts. (In this case the rule would be: take the left part and drop the others.) Classical computers have an easy time of this, but it is not clear that connectionist models can do so without already implementing the representations and rules strategy. For implementational connectionists that will be fine. But radical connectionists, who believe they can explain cognition without classical structures, may face serious problems in what they can explain. (I should note however, that this issue is a matter of furious debate.)
   
3. Generalization. A related complaint against neural nets is that they do not generalize to novel inputs in human ways. For example, neural nets trained to produce the past tense of English verbs, that are given such odd and nonsense verbs as 'biznack', don't give what English speakers provide: biznacked. So the net does not seem to generalize properly to what we all know about past tense formation: when in doubt add 'ed'. What the model lacks is the appreciation of a rule, which suggests that only classical models can handle this problem properly. Again the matter is a topic of hot debate.
   

Further Reading
Clark, A. (1993) Associative Engines MIT Press [Provides a nice review of connectionist work with special emphasis on the problem of representation in language.]
Elman, J. L. (1991) "Distributed Representations, Simple Recurrent Networks, and Grammatical Structure," in Touretzky, D. Connectionist Approaches to Language Learning , Kluwer, Dordrecht, 91-122. [The classic experiment on teaching nets to learn syntactic structure]
Pinker, S. and Prince, A. (1988) "On Language and Connectionism: Analysis of a Parallel Distributed Processing Model of Language Acquisition," Cognition , 23, 73-193. [A through criticism of Rummelhart, D. and McClelland (1986). Undermines claims that connectionist models correctly simulate the process of language learning, especially the the acquisition of regular verbs.]
Rummelhart, D. and McClelland, J. (1986) "On Learning the Past Tenses of English Verbs," in Parallel Distributed Processing , vol. I, MIT Press, Ch. 18, pp. 216-271. [Classical article describing TRACE.]
Sejnowski and Rosenberg (1987) "Parallel Networks that Learn to Pronounce English Text", Complex Systems , 1, 145-68. [For more on NETtalk.]

Cognitive Science Notes Week 12 (Cognitive Psychology - Imagination)
(CS 2.1-2.3; M Ch. 6; CS 2.7)
I. Imagination in Cognitive Psychology
A. Imagination: the Historical Context

1. It has only been fairly recently that cognitive scientists have been concerned about imagination again.
   
2. To help set the stage, we voted on a question raised long ago by John Locke. Would a man born blind whose eyesight is restored be able to see? The class split about 50-50 on whether the man would be able to see.
   
3. William James' Principles of Psychology set the stage in America for what psychology was all about. This work took imagination and our phenomenal life seriously, as did the German psychologists (such as Wundt) who exerted influence over America at the time.
   
4. But WWI put an end to the German influence over American psychology, and a new tradition emerged in the 20s and 30s called Behaviorism. This school was suspicious of any concepts that could not be defined by public observation. The result was that work on imagination was not publishable.
   
5. WWII brought new influences into the intellectual scene. Psychologists dedicated to the war effort had new experience with information theory, targeting control, human factors, and patients suffering from head injuries. These and other factors eventually lead to a reaction against behaviorism called cognitive psychology. The birth of the school can be dated by a conference at MIT on September 11 1956. There is a new interest in trying to understand the internal states of the brain on analogy with the states of computers.
   
6. One might expect research on imagination to have followed quickly. However the reaction against behaviorism gave such topics as learning a bad name. It was not until the 80s that a second revolution (connectionism) challenged some of the basic articles of faith of the classical approach to cognitive science. Textbooks as late as 1988 talk very little about learning and imagery, and treat connectionism as an odd new fad. Since we are immersed in it now, it may be hard to understand how revolutionary the intellectual climate is at the moment. But ideas and concepts in cognitive science are changing rapidly.
   
7. In particular, the new movements have challenged the classical computer-based view that adopted a top-down methodology, and considered thought to be a matter of symbolic processing. In the classical context, there would seem to be very little room for research on, or even mention of, imagination. In the new climate, however, bottom up approaches to understanding the mind have become more popular. With this has come a deeper interest in and appreciation of learning and perception.
   

B. Why imagination is interesting


1. Consider a fly. It does a very good job of not getting caught. It has a very sophisticated visual system. Can we really hope to explain what a fly is doing (or even a human!) without bringing in the images that the fly's eye delivers to the fly's brain? Does it seem plausible at all that such cognition could be symbolic?
   
2. Imagine an elephant. Note that different people imagined one from different points of view (though none from the rear). Then we were to imagine a fly on the trunk and to "notice how many legs were on the fly". Did we still imagine the elephant? No. It was clear that to imagine the fly we zoomed in on it, leaving the elephant out of the picture. This is just one of the many interesting features of our ability to imagine: to zoom in when we want to furnish details. Cognitive science should help us understand such abilities.
   
3. Note that we have imagination in all of the senses: we can imagine pain, music (Louie, Louie), tastes (pepperoni pizza) motor routines (slam dunk), etc. and we can do so while at the same time receiving sensations from the external world.
   
4. So there are two basic ways that the brain is capable of processing objects: through our perceptions of the world, and by imagining them. There is every reason to believe that this second ability is as important to our ability to think, reason and navigate the world as is our ability at language.
   
5. Well, could the blind man with eyesight restored see? No. There is a window of opportunity when the basic ability to recognize a world of objects gets set down in the brain. Kittens reared with their eyelids sewn shut never learn to see properly even when their eyes are opened in later life.
   

II. The Imagery Debate

1. Imagination presents an interesting alternative to the classical approach to cognitive science. Since the classical view takes cognition to be the result of symbolic processing similar to what goes on in a computer, either imagination must be understood as a form of symbolic processing, or imagination must be considered to play no interesting cognitive role.
   
2. The classical theory is that cognitive science's job is to discover what is common to the human cognitive architecture. This amounts to something like discovering the functional structure of a computer. Humans are remarkably alike in their basic cognitive abilities, so it is a good guess that there is a structure here that we all have in common.
   
3. The basic picture of this architecture is to divide it in three: Sensory systems; Motor systems; and Central systems. The central systems include thinking attention memory learning and language. The idea is that sensory systems are modular. They do their jobs of delivering information to the central system independently from each other and from direction by the central system. The central system receives information from the senses and controls motor output, but it is not driven by sensation. A lot of thought and reasoning can proceed without help at all from sensation. (In fact sensation may hinder thought; why else do we stare at the ceiling when we think hard?) A computer processing away without the help of any new input is a good model of this feature.
   
4. Classicists presume that the central system is highly dependent on symbolic processing. There are several reasons for believing this.
   a. Turing machines are a model of computation, and known to be capable of carrying out any function that can be specified by a set of rules, including presumably all possible tasks in thinking, reasoning, attention, memory, learning and language. Turing machines are the simplest kind of symbolic processors (computers). Although the brain almost certainly doesn't implement such a simple and cumbersome machine, variants of the same basic design (von Neumann architectures) have been proven to be extremely useful and powerful information processing devices. What better model of intelligence could we find?
   b. Abilities in reasoning and language appear to require processing that is sensitive to the structure of symbol strings. For example, our language and thinking processing abilities are productive , which means that we are capable of recognizing and understanding an unlimited variety of sentences. The computational model explains how we are capable of this amazing feat. We compute language meaning much the way that a calculator computes arithmetic - not by storing up large quantities of information (the answers), but by following simple rules (similar to routines you learned in grade school to add subtract multiple and divide). Furthermore, our abilities in language and thought are systematic, which means that we use the same processing methods to deal with structures that have the same form. Humans that can understand 'John loves Mary' also understand 'Mary loves John' and every other sentence of the form: Name Verb Name. Humans do not learn the meanings of sentences piecemeal. They compute meanings on the basis of a sentence's structure. The symbolic processing hypothesis would explain this fact. A calculator deals with 3x(5+7) using fundamentally the same operations as it does to calculate 7x(5+3), for it too bases the calculation in the same symbolic structure in each case.
   

III. The Advantages of Imagination

1. Now let us turn to the facts of imagery. Kosslyn has championed the view that imagery is a separate cognitive ability that involves feeedback to the visual system. The idea that imagery is important to cognition provides an alternative to the view that central processing is essentially symbolic. Brains might contain a special graphic processor along with a symbolic processor. What advantages would this graphic processor bring?
   a. One important idea is that visual imagery carries much more information that symbolic representations are capable of. A picture is worth a thousand words.
   b. Consulting an image makes things obvious that we would otherwise have to think out, (eg. the chair is close to the couch in figure 6.2 M p. 96)
   c. There are a number of skills such as finding things, planning errands, trying out ways of building things such as bridges, explaining continental drift etc. where an ability to imagine the various objects, actions and likely outcomes is extremely helpful. We can literally see in our mind's eye the things we should avoid doing when we imagine a course of events. Imagination gives us foresight. It also allows us to adapt ahead of time. For example, just by imagining a task, the athlete can train herself to improve.
   d. There is excellent evidence that much of language understanding is based on metaphors which are in turn founded on visual imagery . For example, top, up ,etc. mean better, stronger (top of his game) down, bottom mean worse, weaker etc. (in the pits).
   e. We also know that imagery is useful in solving problems (the architect's diagrams) and improving memory. There is even some new interest in imagery in artificial intelligence research.
   

IV. Experiments on Imagery

1. Koslyn had people imagine moving attention from one point to another on a map. The time to move attention was proportional to the distance on the map suggesting that attention actually "moves" from point to point in an imaging space.
   
2. Mental rotation experiments support the same basic idea. Subjects asked whether two images matched apparently used a mental rotation technique to solve the task, for the time it took for solution depended on the angle through which the image would have to be rotated to align the two. However work of Pylyshyn shows that angle of rotation is not the only feature that effects speed on this task.
   
3. Experiments with PET and rCBF scanning show that visual imagination and other cognitive tasks differ in that the former involve activation of visual areas of the brain. Furthermore work with brain-damaged patients shows that brain damage can selectively impair abilities at mental rotation.
   
   

1. UG (universal grammar) is the set of principles that are presumed to be common to all grammars of a language humans could learn. The idea is that UG helps the child by narrowing down the set of possible hypotheses about what the structure is of the language to be learned. UG is presumably part of our genetic endowment.
   
2. An alternative view would be that language learning is brought about by mechanisms that are useful more generally and are not specific to a specialized language learning module. Evidence for the alternative view includes the fact that we have no evidence at all for the storage of UG information in specific genes. Nor are there good stories about how UG principles might have been the product of evolution. There is some evidence also that abilities that serve language (such as categorical perception) also serve other abilities in nonhumans.
   

B. Principles and Parameters Theory


1. What might some of these principles be? One problem is that languages vary quite widely along a number of different dimensions, for example, syllable structure, case assignment, and the placement of modifiers. To resolve this problem linguistics have proposed a basic system of linguistic principles, that still allow options to be selected for each language (parameters). For example, modifiers can be placed before or after what they modify, but not (say) 5 words away. So a child learning a given language need only learn the parameter setting (before or after) for modifiers. [The same idea seems to be at work in the colors of mammals and butterflies.] 2. Phonological Universals. Example. Do you allow consonants at the end of syllables? English: Yes, Italian: No. Still all languages adhere to the principle that if you allow consonants to end syllables, there are always some syllables that do not.
   
2. Theories that decide what a pronoun may refer to (binding theories) follow the same principle in all languages; it is just the nature of the grammatical chunk that varies (English: clause, Icelandic: tensed clause). Similar points can be made for bounding theory, which describes how items may be moved, and theta theory that tells us how phrases are assigned roles such as agent and patient.
   
3. X-bar theory is an attempt to describe the principles and parameters for possible phrase structure grammars. One important idea is that phrases contain a head (NounPhrases a head Noun, VerbPhrases a head Verb, PrepositionaPhrases a head Preposition, etc.). One interesting regularity is that in most languages the head of a phrase is always on the right, or the left. For example in Japanese, the verb and the preposition both end the phrase while in Hebrew they both lead it. (English is an exception. Class question: Do you see why?)
   
   

1. Strict behaviorism. We are not supposed to ask people what they mean. Just look at verbal behavior. (Bloomfield does talk a bit about meaning, but he characterizes phrase meaning as the influence it has on others behavior when uttered.)
   
2. Interest in languages outside Europe, especially American Indian languages.
   
3. Spoken language is emphasized over text. Language is seen as a stream of sound.
   
4. To understand a language you need to understand a corpus (set of utterances) used in a culture. Linguistics is the study of "sonic behavior".
   
5. All higher level structure in language is determined by phonetic features (bottom up), so for example...
   
6. Words are merely the collecting together of sounds that often are uttered together.
   
7. Harris was interested in discourse analysis (the analysis of whole groups of sentences). Another influential idea of his was to idea that sentences could be reduced to normal or kernel form. For example passive sentences are converted to active, and relative clauses into separate sentences: 'The visible world was created by invisible God.' ==> 'The world is visible'. 'God is invisible'. 'God created the world'. This inspired the theory of transformations.
   

III. Chomsky's Work
A. Chomsky has had an immense influence. He is the most cited living author. Chomsky proof-read Harris' book and with the support of Nelson Goodman went off to Harvard. There he created a massive work The Logical Structure of Linguistic Theory which laid the groundwork for a whole new approach to linguistics. A piece of this called Syntactic Structures (1957) had a strong influence on linguistics, and can be thought of as a major influence on the birth of cognitive science.
B. Chomsky borrowed ideas from the theory of formal systems (logic), and applied them to language. In such a formal theory we define what we mean by a well-formed formula (wff) using a set of recursive rules. Like this: p, q, r are wffs. If X is a wff, then so is ~X. If X and Y are both wffs, then so are (X&Y), (X->Y), (XvY), and (X<->Y). These allow the creation of a potentially infinite class of wffs: p, q, ~p, (~p&q), ~(~p&q), (~p&q)->~p, etc... Chomsky's idea was that the same was true of language. There is a set of rules that can be used to generate all (and only) the well-formed sentences of English.
C. Chomsky also borrowed the notion of a transformation from Harris, an idea that is also inspired by the notion of the application of rules to a set of sentences to create good reasoning. One such rule is Modus Ponens: from X and X->Y, deduce Y. Correct logical rules have the feature that they preserve truth. (If the premises are true then so must be the conclusion.) This logical idea is the analogue of the Katz-Postal Hypothesis that transformations preserve meaning .
D. On Chomsky's view the goal of linguistics is to produce a grammar, i.e a set of recursive rules that generate the whole host of possible sentences of a language. Language is productive and it is essential to capture this fact in linguistics. The linguist's role is not to study a corpus (set) of actual utterances. Instead it studies human competence to produce an unlimited variety of well formed sentences.
E. So methodology in linguistics may be very different from methodology in psychology where we must do experiments on many subjects. In linguistics, the linguistic can use his own linguistic competence to make judgements about what is or is not a sentence of her language. Since this competence is presumably shared, there is no need to constuct experiments across different speakers. (This methodology makes psychologists very uneasy.)
E. The grammar is what the baby learns when it learns language, so the linguists job is to learn explicitly what is learned unconsciously by the child.
F. By studying grammars we study the rules that children acquire when they learn language. Since the language heard by children does not contain enough clues to determine the grammar, there must be some innate mechanism that helps the child learn the linguistic rules.
G. The goal is to provide formal theories of language at three possible levels (the higher the better). 1. Observational Adequacy: (Weak generative capacity) Rules that generate all and only the sentences of the language.

1. Descriptive Adequacy: (Strong generative capacity) Rules that assign syntactic structure to strings of words.
   
2. Explanatory Adequacy: A theory of the mechanisms that explain exactly how the hearer processes language.
   

IV. The Hierarchy of Grammars
A. Finite State Machines. These are devices that merely move from one box or node to another, making a selection from each box and continuing along any arrow from a box. Chomsky showed that no finite state machine can generate all and only strings of the form: aaaa...bbbb... where the number of as and bs is the same. But language has structures with similar complexity, notably wherever there are rules of agreement (say) between noun phrase and verb phrase: 'John loves Mary', but 'Men love Mary'.
B. Phrase Structure Grammars. Phrase structure grammars allow the introduction of rewrite rules with variables referring to grammatical types. This vastly improves the power of the grammar to account for the structure of language. Rules of language must be expressed not by how one transitions from one 'box' to another, but by the understanding of grammatical categories: NP, VP, Auxiliary Verb,. How else did Justin's daughter generalize to the sentence : 'I am going am'nt I?' She "knew" the rule that tag negation works with auxiliary verbs 'I have it haven't I' (but not regulars '*I walk, walkn't I') so this construction makes sense. This kind of over-generalization is a basic feature of language learning: children learn irregular plurals: mice, but as they become more familiar with the regular plural rule, they go back to mistakes: mouses. Click experiments also suggest that we are aware of the phrase structure of sentences that we hear, For the time at which clicks are heard subjectively moves towards the major grammatical boundaries.
C. Transformational Grammars. These introduce yet another innovation. Rules that transform phrase structures into alternative forms. Transformations provide especially economical explanations for the formation of questions, and passive voice, but also in accounting for deletions ('John and Mary like Jill' instead of 'John likes Jill and Mary Likes Jill') that we may be using to help memory chunking that helps overcome the 7 plus or minus 2 constraint on short term memory.

1. Dualism. The mind is a radically different sort of substance or property outside the physical world.
   
2. Materialism (Physicalism, Naturalism) The Mind is a part or feature of the physical world.
   a. Reductive materialism. Mental states just are physical states of the brain.
   b. Functionalism. Mental states are computational states (or more abstract higher level states).
   c. Eliminative materialism. Mental states are myths that science will do away with.
   d. Instrumentalism. The notion of a mental state may be theoretically useful, but like vectors and centers of mass, it does not refer to something that actually exists in the brain.
   

B. If Cognitive Science seeks to provide a complete account of the mind, then dualism is to be avoided, for it puts the mind beyond the reach of the natural sciences. Most cognitive scientists (where they have a philosophical view) believe in some form of functionalism. This fits rather nicely with the classical computational approach (CRUM)

II. Some Problems for Functionalism and Nearby Brands of Materialism
A. Emotions. It does not seem that emotions could be computational states of anything. Suppose a computer-robot were to mimic my brain's computational states exactly. Still it wouldn't feel anything. There seems to be an important divide between thought and rationality on one hand and the emotions. Perhaps functionalism is a good theory of the former but it seems much less able to account for the latter. Here are some responses to the challenge.

1. Denial. Emotions just aren't part of cognitive life anyway, so forget them. But Damasio's work challenges this response. He describes patients whose loss of emotional response (due to brain injury) severely compromises their ability to make reasonable decisions or interact socially.
   
2. Provide a Functionalist theory of emotions. The idea is that emotions are a computational aspect of the goal setting routines (e-motion) . Emotions are devices that monitor and focus attention on parts of the task that really matter to the organism's success. They provide a quick appraisal of overall status of the problem solving. (Far from goal: sad, near: happy, fear: survival goals are threatened.) Emotions may also play a crucial role in social thought. Empathy allows the emoter to reason about others on analogy with ones own states of happiness and sadness, etc..This kind of thinking promotes cooperative social effort.
   
3. Provide a psychophysical theory of emotions. OK so emotions are not computational states. They are explained by physio-chemical states of the brain. The amygdala is essential emotional response, and it in turn is effected by certain neurotransmitters, notably serotonin, and dopamine. So a theory of emotions is not computational, it is physiological. We might frame a computational theory of Martian thought, but we need a human body to frame a theory of human emotions.
   
4. Still there seems a nagging objection. Emotions are ways we feel, and these private subjective feelings just are not to be explained by the presence of computational routines, neural activity in parts of the brain or the presence of different chemicals. We are conscious of our emotions, and this conscious seems beyond the range of any science.
   

B. Consciousness and Qualia


1. 'Qualia' (plural, the singular is 'quale' pronounced: quahlay) is a philosophers term for ways things feel to us: the experience of redness, or the taste of almonds, or the way it feels when you are mad. Computational theories and even imagery theories do not seem to even mention our qualia. It seems that our conscious feelings lie outside the range of any computational of physiological theory. If so, cognitive science is essentially incomplete. It will never yield a full account of our mental lives. (These ideas are naturally aligned with some brand of dualism.) Here are some responses to the challenge
   
2. Denial. Say that cognitive science just isn't about the way things feel. That feelings don't matter. Although cognitive science has as a matter of fact tended to ignore feelings, this orientation seems excessively harsh. Qualia are, after all, what makes life interesting. Shouldn't a theory of our cognitive life account for them in some way?
   
3. Provide a Functionalist theory of qualia and consciousness. The idea is to claim that qualia just are certain computational states of the brain. The feeling of pain, for example just is the state with certain typical relations to inputs (damage to the body), outputs (winching, saying "Ow") and other mental states (fear, distress, avoidance). Any state in any system that set up this set of relationships would count as pain. Consciousness can be explained as an attention mechanism on analogy with an operating system.
   
4. Provide a psychophysical theory of qualia and consciousness. Qualia may be accounted for via the particular sensory vectors that are provided by the sensory equipment. (Even emotions might be the sensation of our own body states.) Consciousness can be explained via the synchronization between neural groups controlled by the thalamus. This synchrony is useful in the attentional system, and in binding information together to obtain a unified view of the world. Consciousness is also related to short-term memory.
   

C. Absent Qualia


1. There seems to be a fundamental objection to the ideas outlined in A. and B. We can imagine the Chinese nation executing exactly the commands of a computer program functionally identical to your brain's while you are experiencing a nice cool beer on a sunny patio in Houston. But the Chinese nation could not be experiencing anything of the kind! Functional identity is not enough to insure identity of experience, so the functional account must be wrong. Adding in neurophysiology to the theory of qualia does not seem to help very much. Imagine now that we have a robot, (or even a brainless cadaver) that is radio controlled by the Chinese nation. The sensory physiology that delivers qualia would be the same as in me. But that wouldn't make it any more plausible that the system has the experience of enjoying a cool beer.
   
2. The objection is one example of a basic worry: we can't imagine how conscious experiences could be the result of neural activities of any kind. (That marvelous first-slug-of-a-cool-beer-on-a-hot-day feeling seems far from explained by noting there is a certain pattern of activity on my cortex!) But should be take such arguments "from a lack of imagination" seriously? In the past people have not be able to imagine that life could be the product of molecular interactions, but biochemistry has now provided enough information for us to see how this could be. The same might also happen with consciousness.
   
3. Furthermore, imagine that we have a thing before us that in every conceivable way acts (inside and outside) as a perfect functional analog of a human being. What evidence could one use to deny consciousness to such a thing? Doesn't the right functional behavior serve as adequate evidence of consciousness in other humans, such as our friends and family? So why deny consciousness to this thing? Is it even coherent to deny this thing conscious states, and to say that despite its proper behavior, there is, nevertheless, "nothing going on inside". Call a being functionally identical to you that lacks consciousness a zombie. Given that it's functionally identical, what conceivable test could you use to come to know it is a zombie? Note: zombies will claim they have consciousness and talk a good story about their internal states, etc..
   
4. One answer to this is to say that we can just plain imagine a being functionally identical to ourselves that lacks consciousness, or even a physical universe identical to out universe where none of the beings are conscious. But then consciousness becomes an internal property of things that cannot be investigated by any empirical science. There would be no scientific way to distinguish ersatz (seeming) consciousness from the real stuff. This would be bad for anyone who hopes for a functional explanation of consciousness. So functionalists must find reasons to reject this tactic. People with a more biological or physiologically oriented account of consciousness must face the same problem. They must say that a being biologically identical to me that lacked consciousness makes no sense, despite the fact that we can imagine such a being.
   
   

Further Reading on Consciousness and Qualia

Chalmers, D. The Conscious Mind (Presents a dualist theory. Has a good bibliography)
Chalmers, D. "The Puzzle of Conscious Experience," Scientific American 273, (1995) pp. 80-86 (A very accesible review of Chalmer's dualism)
Churchland, Paul "The Rediscovery of Light" Journal of Philosophy , 1995
Churchlands, Paul and Patricia "Could a Machine Think?" (Jan. 1990) Scientific American pp. 32-37
Crick, F. and Koch, C. "Towards a Neurobiological Theory of Consciousness," Seminars in Neurosciences 2 (1990) pp. 263-275 (A neurphysiological view: consciousness is a certain synchronized activity of neural bundles oscillating at about 40 hertz).
Damasio, A. Descartes' Error (Argues that emotion is central to cognition)
Dennett, D. Consciousnes Explained (An instrumentalist account of consciousness. Fun reading but challenging noetheless.)
Searle, J. "Is the Brain's Mind a Computer Program?" Scientific American (Jan. 1990) pp. 26-31 (An easy entry to some of the arguments against functional accounts of qualia)

1. Whether top-down or bottom-up theories of cognition are more apt.
   
2. What is the nature of reasoning?
   
3. Whether a symbolic processing model of cognition is adequate.
   
4. What are concepts?
   
5. What is consciousness?
   
6. How do we understand language?
   
7. How are objects recognized?
   
8. What are perceptions, for example, the sensation of seeing the color red?
   
9. What features of cognition are innate?
   
10. How does the brain learn?
    
11. What is human memory?