Face processing: studies in a series of prosopagnosic patients

Face processing: studies in a series of prosopagnosic patients Louvain

Queen Square, November 2006

Trevor Bass Andrea Butler Mariya V Cherkasova Shaunak Deepak Chris Fox Rebecca L Hefter James M Intriligator Julian P Keenan George Malcolm So Young Moon Numaan Malik Galit Yovel Jingher Zhao

Jason JS Barton Olivier Felician Beatrice de Gelder Nouchine Hadjikhani Todd Handy Nancy Kanwisher Dara S Manoach Margaret O’Connor Dan Z Press Max Riesenhuber

Overview: I. Starting points II

Prosopagnosia 1. face perception • A. facial spatial relations • B. non-facial perception • C. between- and within-object spatial processing • D. global/local processing 2. face detection 3. face imagery III. Summary IV. Future directions…

STARTING POINT #1

Cognitive models of face processing propose a branching hierarchy of stages

Occipital Face Area

Superior Temporal Sulcus

Fusiform Face Area

STARTING POINT #2. Neuroimaging data show a network of face processing regions in human cortex.

STARTING POINT #3

A variety of lesions cause prosopagnosia. 8 adult-onset lesions: 5 bilateral 1 bilateral anterior temporal trauma 2 bilateral occipito-temporal trauma, gunshot 2 right anterior temporal/occipital small left temporal lesions trauma, encephalitis 3 unilateral 3 right occipito-temporal stroke (2), tumor 3 childhood-onset lesions: head trauma occipital polymicrogyria respiratory arrest, posterior gyral atrophy

Is there a relation between cognitive models and anatomic data

Can prosopagnosia tell us something about structure-function correlations?

occipital face area?

superior temporal sulcus?

fusiform face area?

anterior temporal?

I. PROSOPAGNOSIA

“The inability to recognize previously seen faces, with relative sparing of other perceptual, cognitive and memory functions.” Broad Subtypes: 1. Apperceptive - failure to generate a sufficiently accurate percept to allow a successful match to stores of previously seen faces. 2. Associative - accurate percept, but failure to match because of loss of facial memory stores or disconnection from them. Family of subtypes corresponding to defects at different stages in a cognitive model of face processing.

1. PERCEPTION IN PROSOPAGNOSIA Q. What kind of perceptual deficit can

affect recognition of individual faces yet spare basic-level object recognition ? object recognition theories: a. ‘entry-level’ recognition, satisfied by coarse categorical structural descriptions (i.e. “recognition by component” theory, geons, edge-based) b. precise metrical (coordinate) analysis for exemplar recognition

coordinate change

categorical change

• Faces all share the same basic structure, with the same features

(two eyes, nose, mouth) in the same arrangement (e.g. eyes above nose) • What distinguishes one face from another are subtler variations in the precise metrical structure and arrangement of these elements.

Coordinate (2nd order spatial relation) change

Categorical (1st order spatial relation) change

1.A) Perception of spatial relations in prosopagnosia What is the

perceptual defect in prosopagnosia ? •Perceiving precise structural coordinates (e.g. second-order spatial relations) may be important in recognizing specific exemplars of complex 3-D structures. •Normal subjects have difficulty recognizing inverted faces, and studies show that this is accompanied by difficulty perceiving the spatial relations of facial features. Barton JJS, Keenan J, Bass T. Discrimination of spatial relations and features in faces: effects of inversion and viewing duration. Brit J Psychol 2001;92: 527. Barton JJS, Deepak S, Malik N. Attending to faces: change detection, familiarization, and inversion effects. Perception 2003; 32: 15.

?

1.A) Perception of spatial relations in prosopagnosia What is the

perceptual defect in prosopagnosia ? •Perceiving precise structural coordinates (e.g. second-order spatial relations) may be important in recognizing specific exemplars of complex 3-D structures. •Normal subjects have difficulty recognizing inverted faces, and studies show that this is accompanied by difficulty perceiving the spatial relations of facial features. Q. Are prosopagnosics impaired at perceiving ‘coordinate’ spatial relations? Barton JJS, Keenan J, Bass T. Discrimination of spatial relations and features in faces: effects of inversion and viewing duration. Brit J Psychol 2001;92: 527. Barton JJS, Deepak S, Malik N. Attending to faces: change detection, familiarization, and inversion effects. Perception 2003; 32: 15.

Results: perception of spatial relations:

Barton JJS, Press DZ, Keenan JP, O’Connor M. Lesions of the fusiform face area impair perception of facial configuration in prosopagnosia. Neurology 2002; 58: 71.

1a

The “geometry effect”: Integrating spatial relations across the face: S.008, with anterior temporal lesions, does relatively well at discriminating single spatial relations (mouth or eye) in a target face. Q. How does she integrate spatial information across the whole face? Consider ‘aspect ratio’ of eyes to mouth. Combined moves of eyes and mouth can preserve the aspect ratio or distort it.

(more distorting - less distorting)

“GEOMETRY EFFECT”: Results: Normal subjects are faster and more accurate at discriminating more distorting combinations than less distorting ones in upright faces, BUT NOT in inverted faces. S.008 does not show a significant ‘geometry effect’. Hence she perceives spatial changes but does not integrate this information across the face.

upright

inverted

1.B) Non-facial perception in apperceptive prosopagnosia:

Neuropsychology • visual object and space perception battery • Ghent overlapping figures • Benton line orientation Within-category • vegetable identification

“Low level” perception • static contrast sensitivity • luminance • saturation • spatial resolution • dot displacement • curvature discrimination

• Most patients had difficulty with fruit/vegetable identification

normal 0-3 errors patients 3-20 errors • Most consistent defects in occipitotemporal lesions were: - reduced mid-high spatial freq. (4.2 - 8.4 cycles/°) contrast sensitivity - reduced perception of dot displacement • Bilateral lesions reduced thresholds for saturation perception, scores for overlapping figures. • Curvature and line orientation impaired in S.006 only, who had the most difficulty with basic-level object recognition.

1.B) Non-facial perception in apperceptive prosopagnosia

Barton JJS, Cherkasova, MV, Press DZ, Intriligator JM, O’Connor M. Perceptual functions in prosopagnosia. Perception 2004; 33: 939.

1.C) ‘Within-object’ versus ‘between-object’ spatial representations:

Prosopagnosics fail at the “coordinate” spatial relations in faces or abstract dot patterns. What about other “visuospatial” functions? Within-object and between-object spatial representations may be functionally and anatomically distinct: • Between-object perception can show the opposite direction of neglect to within-object neglect. Humphreys G. Phil Trans R Soc Lond B 1998;353:1341 • Patient with bi-parietal lesions (dorsal pathway) has normal within-object but abnormal between-object spatial judgments. Cooper A, Humphreys G. Neuropsychologia 2000;38:723 Q. Do our patients (with ventral pathway lesions) have the opposite dissociation: abnormal within-object but normal between-object spatial perception?

Within-face spatial processing

Between-face spatial processing

Which face is furthest away from the other two?

Which face is different?

Results:

within

within - object

between - object

Conclusion: prosopagnosic subjects have good between-object perception.

Barton JJS, Cherkasova MV. Impaired spatial coding within-objects but not between-objects in prosopagnosia. Neurology 2005; 65: 270.

Q. When does a between-object relation become a within-object one

(A ‘competition’ between spatial representations)

Task 1: which apex is further away from the other two? Task 2 (B?lint’s patient): is this triangle symmetric or asymmetric?

control

unilateral

bilateral

1.D) Global/local perception: If there is a ‘holistic’ deficit in face

perception, some postulate that this should also be revealed as a failure to perceive ‘global’ level information. Q. Do they fail to show effects of the global level of hierarchical Navon letters?

Global/local 8 apperceptive prosopagnosic patients (1 developmental)

Longer RT in general, but normal pattern of global influences.

2. FACE DETECTION IN PROSOPAGNOSIA - de Gelder postulated five

separate face processing mechanisms, in which face detection and face recognition are distinct.. Riesenhuber’s computational models suggest that detection and recognition are merely points on a perceptual continuum. Are prosopagnosic subjects normal at detecting faces? RUBIN’S VASE EXPERIMENT:

How do we diagnose associative prosopagnosia

Indirect conclusion - “if perception seems normal, must be associative” • What we need is a method of assessing the status of facial memory stores, without having to go through a perceptual route: ………... imagery. • What kind of face imagery ? Is there a distinction between local features and ‘holistic’ or overall facial configuration?

3. FACE IMAGERY IN PROSOPAGNOSIA

Face imagery questionnaire

37 questions, AFC 2 choice. 31 controls, not at ceiling (93% correct) 19 feature-related imagery Who has the more prominent nose? Meryl Streep Nicole Kidman Who has a moustache? Martin Luther King Jr Louis Armstrong 18 ‘global’ face shape imagery Who has the narrower face? Cher Martha Stewart Who has the more angular face? George Washington Abraham Lincoln calculate binomial limits of chance (does it exist?), and 95% prediction interval for normal imagery (is it impaired?)

Imagery results

Summary - imagery results: • bilateral occipitotemporal lesions impair

but do not abolish face imagery. • unilateral right-sided occipitotemporal lesions impair imagery for global facial structure more than imagery for features. • anterior temporal lesions abolish face imagery. Barton JJS, Cherkasova M. Face imagery and its relation to perception and covert recognition in prosopagnosia. Neurology 2003; 61: 220-225.

III

SUMMARY (a) Tentative structure-function correlations: • bilateral occipitotemporal lesions (in the region of the FFA): have a severe failure to perceive “coordinate” spatial relations, are only mildly impaired face imagery, consistent with an apperceptive prosopagnosia. • right occipitotemporal lesions: are similar to bilateral ones, except for preserved imagery for facial features, and less severe prosopagnosia (d’ measures). • anterior temporal lesions: markedly impair face imagery, with only subtle (integrative?) effects on perceiving spatial relations, consistent with an associative prosopagnosia.

III

SUMMARY (b) Characteristics of the apperceptive defect: • This defect in processing structure affects other ‘non-face’ stimuli, and may be one mechanism by which recognition of exemplars of other object classes is impaired. • This defect is specific for ‘within-object’ spatial representations. • Reduced high spatial frequency contrast sensitivity may play a role. • Global/local processing is not affected. • Figure-ground assignment is impaired and more difficult with faces.

Do these conclusions have predictive power

RC: 25 year old man HSV Encephalitis 2 years ago. BEHAVIOURAL TESTING: Face familiarity: impaired Famous faces: d’ = 0.61 (N>2.9) Relatives’ faces: d’ = 0.87 Perception of faces - good Benton Face recognition test = 45/54 Spatial relations : 2 second; unlimited time - eye 80% (N> 73); 93% (N> 68) - mouth 70% (N> 72); 89% (N> 55) Morphed faces - same view = 77.8% (N>78) - varying view = 62.5% (N>57)

Short term memory for faces - poor Warrington Recognition Memory test faces = 27/50 ; words = 45/50

RC’s behavioural data are most consistent with an associative

prosopagnosia: Prediction: RC’s neuroimaging will show: 1. Bilateral anterior temporal lesions 2. Sparing of FFA….

….BUT is FFA spared

PLAN: do an fMRI experiment on RC

fMRI confirms that it is:

Result: RC does have an FFA, AND it shows adaptation effects to identity. This supports the hypothesis that the FFA is involved in perception of facial structure, and that its loss leads to apperceptive prosopagnosia.

fMRI confirms that it is:

Event-related potentials in RC also show a face-selective N170 response in his right occipitotemporal region, and possibly a distinction between familiar and anonymous faces in later responses

Left occipitotemporal

Right occipitotemporal

N170

familiar faces

anonymous faces

objects

Future directions: Perception of other facial structural data

Future directions: Why are some right PCA patients prosopagnosic and

others not?

P.001 non-prosopagnosic

S.009 prosopagnosia

“I was at a conference and I saw this other person walking

I thought it was probably someone I knew and so I smiled at them and started to greet them, only to realize that it was my own face and I was looking at a mirrored wall.” (LH - subject 011)

http://www.neuroophthalmology.ca/UBCNeuroOp/JBarton/FrJBhome.html

NORMAL PERCEPTUAL STUDIES Barton JJS, Keenan JP, Bass T

Discrimination of spatial relations and features in faces: effects of inversion and viewing duration. Brit J Psychol 2001; 92: 527-49. Barton JJS, Deepak S, Malik N. Attending to faces: change detection, familiarisation and inversion effects. Perception 2003; 32: 15-28. Malcolm GL, Leung C, Barton JJS. Regional variation in the inversion effect for faces: different patterns for feature shape, spatial relations, and external contour. Perception 2004; 33: 1221-31. Barton JJS, Radcliffe N, Cherkasova MV, Edelman J, Intriligator JM. Information processing during face recognition: the effects of familiarity, inversion and morphing on scanning fixations. Perception 2006: 353:1089-105 Fox CJ, Barton JJS. What is adapted in face adaptation? The neural representations of expression in the human visual system. Brain Res, in press PROSOPAGNOSIC STUDIES Barton JJS, Cherkasova M, O’Connor M. Covert and overt recognition in acquired and developmental prosopagnosia. Neurology 2001; 57: 1161-7. Barton JJS, Press DZ, Keenan, JP, O’Connor M. Lesions of the fusiform face area impair perception of facial configuration in prosopagnosia. Neurology 2002; 58: 71-8. Barton JJS, Cherkasova M, Press DZ, Intriligator J, O’Connor M. Developmental prosopagnosia: a study of three patients. Brain and Cognition 2003; 51: 12-30. Barton JJS, Cherkasova M. Face imagery and its relation to perception and covert recognition in prosopagnosia. Neurology 2003; 61: 220-225. Barton JJS, Zhao J, Keenan JP. Perception of global facial geometry in the inversion effect and prosopagnosia. Neuropsychologia 2003; 41: 1703-1711. Joubert S, Felician O, Barbeau E, Sontheimer A, Barton JJS, Ceccaldi M, Poncet M. Impaired configurational processing in a case of progressive prosopagnosia associated with right temporal lobe atrophy. Brain 2003; 126: 2537-50. de Gelder B, Frissen I, Barton J, Hadjikhani N. A modulatory role for facial expressions in prosopagnosia. Proc Nat Acad Sci 2003: 100: 13105-10. Barton JJS, Cherkasova MV, Hefter R. The covert priming effect of faces in prosopagnosia. Neurology 2004; 63: 2062-8. Barton JJS, Cherkasova M, Press DZ, Intriligator J, O’Connor M. Perceptual function in prosopagnosia. Perception 2004; 33: 939-56. Barton JJS, Cherkasova MV. Impaired spatial coding within-objects but not between-objects in prosopagnosia. Neurology 2005; 65: 270-4. Barton JJS. Disorders of face perception and recognition. Neurologic Clinics 2003; 21: 521-548. AUTISM SPECTRUM STUDIES Barton JJS, Cherkasova MV, Hefter R, Cox TA, O’Connor M, Manoach DS. Are patients with social developmental disorders prosopagnosic? Perceptual heterogeneity in the Asperger and socio-emotional processing disorders. Brain 2004; 127: 1706-16. Hefter R, Manoach DS, Barton JJS. Perception of facial expression and facial identity in patients with developmental social processing disorders. Neurology 2005; 65: 1620-5.