Behavioral and neuropsychological evidence suggests that during the visual processing of words, adult readers compute an abstract representation of the letters and words, that is, a representation that encodes letter identities (whether it is an Aa or a Bb…) and abstracts away from the specific visual forms (uppercase vs. lowercase, script vs. print…).  How could such a seemingly non-visual representation (after all, the representation abstracts away from what the input looks like) develop in the brain’s visual system?

We used a variety of techniques to make progress on this issue.  First, we used fMRI to confirm that there are areas of extrastriate visual cortex that respond to abstract rather than visual word forms.  Specifically, these visual areas responded to aLtErNaTiNg CaSe words that were visually unfamiliar (Polk & Farah, 2002 ACP_PDF 2_file_document).

Second, we developed a computational model that demonstrated how a neural network could self-organize to develop abstract letter representations when exposed to words from a beginning reading corpus. The model is based on the idea that different forms of the same letter tend to appear in similar contexts (in the same words written in different ways). The modeling work demonstrated how this assumption could interact with correlation-based synaptic plasticity (Hebbian learning) to lead to the development of abstract letter representations (Polk & Farah, 1997 ACP_PDF 2_file_document):

Screen Shot 2016-02-04 at 4.22.29 PM

Third, we ran two behavioral experiments confirming predictions of the hypothesis implemented in the neural network model.  Specifically, we showed that repeated exposure to novel stimuli (Japanese characters) in similar contexts led those stimuli to become significantly more confusable, as the similar contexts hypothesis would predict (Polk et al., 2009 ACP_PDF 2_file_document):

Screen Shot 2016-02-04 at 4.23.06 PM

We also found the first direct, neural evidence for the well-known dual route model of word processing. Specifically, we showed that complex words that are decomposed into parts produce unique activation in Broca’s area and the basal ganglia relative to complex words that are not thought to be decomposed (Vannest et al., 2005 ACP_PDF 2_file_document).