Where did language come from? Precursor mechanisms in nonhuman primates

Auditory cortex: Decoding of vocalizations and words

An ‘auditory word form area’ in the aSTG of humans? A gateway to semantics

Organization in terms of auditory linguistic information in humans also follows an antero-ventral trajectory: A meta-analysis of a large number of auditory imaging studies with PET and MRI found that phonemes are encoded antero-laterally from Heschl’s gyrus[29]. Word representations are found anterior to that[29–31] (Fig. 2A). The resulting “word patch” has occasionally been referred to as an auditory word form area[30] (AWFA), by analogy to the visual word form area (VWFA) for reading, which is found in ventral occipito-temporal cortex[32].

The imaging data supporting an auditory word representation in the aSTG (see also[33,34] for earlier pioneering studies) have found independent confirmation by results of auditory single- and multi-unit responses to whole words in a human patient[35] and by electrostimulation mapping of word comprehension[36] (Fig. 2B). The results also chime with voxel-based lesion-symptom mapping of stroke patients with semantic aphasia[37,38], and with theories based on studies of patients with primary progressive (semantic) dementia[39]. Both semantic patient groups show significant anterior superior temporal involvement, which is located even more anterior than the ‘word-form’ area[40]. Taken together, all of these data point to a hierarchically organized, anterior-directed system in the aSTG (extending laterally into the middle temporal region in humans[41]), which links auditory word representations with meaning[42–44].

A seminal discovery more than three decades ago has shown that vervet monkeys also have the ability to map complex sounds to meaning[45]: Alarm calls referring to different dangers result in differential behavior. The same is true for the monkeys’ own calls, which carry meaning about food types and social rank[46]. It will be interesting to ultimately compare the loci and extent of these auditory-semantic networks in both species.

The auditory ventral stream for auditory object recognition and categorization

The functional-anatomical pathway for encoding auditory object information, including short utterances, continues into ventrolateral prefrontal cortex (VLPFC) in monkeys[47] and ventral inferior frontal cortex (IFC) in humans[3], by way of the uncinate fasciculus and the extreme capsule fiber system (EmC). Together with the anterior part of the STG it has been referred to as the auditory ventral stream (AVS) in both species[2,3]. In monkeys, an antero-ventral network of vocalization-specific activation (without clear hemispheric asymmetry) is apparent in functional MRI studies[48–51] (Fig. 2C). There is good reason to believe that back-projections from frontal regions to aSTG in both species lead to categorization of sounds as auditory objects[44].

The auditory dorsal stream for sensorimotor control and sequence analysis

As words are concatenated into longer sequences in humans and the rules of syntax have to be used to make sense out of whole sentences, speech turns into language and a process specific to humans comes into play. Most of this analysis is accomplished by the auditory dorsal stream (ADS) and its associated motor structures[52]. It is in this pathway where some authors have sought to find fundamental differences between monkeys and humans[53–55], but an auditory dorsal pathway with direct connections from posterior STG to prefrontal cortex clearly exists in both species[47,56]. It has been argued that even mere quantitative changes can lead to qualitative differences, as in nuclear physics or semiconductor electronics, once they surpass a certain threshold[57], but the search for the fundamental difference between monkeys and humans is still on, and concentrates now on the nature of the interaction between ventral and dorsal pathways[57–59].

Motor control of vocal production: larynx and nucleus ambiguus

Much has been said about the apparent changes during evolution that may have occurred in the innervation of the larynx[9,61–63]. In particular, it has been claimed that the neural control center of the larynx in the lower brainstem, the nucleus ambiguus, receives a direct projection from primary motor cortex M1 in humans but not in monkeys[61,63]. With recent advances in neuroanatomical tracing techniques[64] as well as diffusion tensor imaging[65], this has proven inaccurate, however: It turns out that at least one of the laryngeal muscles, the cricothyroid, which participates in the control of vocal pitch, receives direct input from several motor cortical regions in the macaque, but its single most prominent cortical input comes from M1 [106]. In addition, diffusion tractography in both monkeys and humans shows that the laryngeal motor cortex (LMC) network is largely comparable in both species, albeit with more extensive LMC connections to somatosensory and parietal cortex in humans[65]. Thus, there again seems to be only a quantitative difference in LMC connections between monkeys and humans.

Furthermore, the nucleus ambiguus is only one of several brainstem nuclei controlling the >40 muscles of the vocal apparatus, and its function is restricted to the control of only 6 of them[62]. As such, it does have some control over pitch changes and over the length of the vocal tract, as necessary for the production of different vowels[66] (“phonation”). Together with improved voluntary control of respiration and coordinated breathing, this may have led to a first simple system for volitional phonation in monkeys[67,68] and, perhaps, to the evolution of singing and yodeling in humans. Thus, speech and language may have in fact evolved from song and music, as some (including Darwin) have argued[69,70].

Supralaryngeal control of vocal production

While the larynx is able to produce relatively simple vocalizations with varying pitch and spectral content, most of the information content of human speech is generated by the supralaryngeal components of the vocal tract, like tongue, lips, and lower jaw. These articulators are under the control of three cranial nerves (hypoglossal, facial and trigeminal) and their corresponding brainstem nuclei (Fig. 3A). Surprisingly little is known about the cortical innervation of these nuclei in monkeys[71], but detailed studies with transneuronal tracers are currently under way (P. Strick, personal communication). Initial results demonstrate converging input from two locations in primary motor cortex: from a ventral location in the orofacial region and a more dorsal location representing the diaphragm for breathing (Helou et al., Soc. Neurosci., 2017;406.03).

It was also found recently that posterior parietal cortex (PPC) provides surprisingly prominent input to the hand in rhesus monkeys, forming a higher-order “command apparatus” for fine motor control of complex hand movements[72], which might have enabled the evolution of tool use in primates. A similar command center in the PPC may have evolved for vocal control in humans but not in monkeys.

The supralaryngeal components of the human vocal tract enable the human vocal system to produce hundreds of discernable phonemes across the languages of the world and carry the bulk of the information content of human speech. Whereas the larynx itself can only produce temporal modulations of the voice through glottal closure and via pulsatile exhalation (c.f. the barks, grunts and screams of macaques), the human vocal tract can produce subtle closures and constrictions of the airflow that each carry their own acoustic-phonetic signatures depending on place or manner of articulation (bilabial, dental, fricative etc.). Combining the various phonemes leads to the combinatorial explosion that is language, and each language uses only a small subset of possible combinations.

The tongue carries much of the load for the production of these “information-bearing elements”[73], and it is for good reason that many languages use the same word for “tongue” and “language” (lingua, langue). The supralaryngeal part of the vocal tract is the real reason for the enormous diversity of the vocal repertoire in humans and the multitude (but also confines) of speech sounds they can produce by combining coordinated movements of the various articulators[60]. The tongue alone (innervated by the n. hypoglossus) consists of eight individual muscles, which allow to move its tip in three spatial dimensions and change its width and length as well as curl, protrude, and retract it (using the hyoid bone as a lever). Tongue action is supplemented by coordinated movements of the lips and the mandible, creating multiple points of contact with the gums and different points of closure of the airways responsible for the production of various types of sounds with distinct vocal signatures, including virtually all consonants. Consonants (in accord with their rarer occurrence) provide most of the information content in speech. In fact, a number of languages, such as Hebrew and Arabic, use consonantal alphabets (“abjads”) to transcribe their spoken words with individual signs for consonants, but not for vowels.

Observation of speech movements in real time

Visualization of the intricate and speedy movements of the supralaryngeal elements of the vocal apparatus during singing and speaking has become possible through real-time magnetic resonance imaging[74,75] (rtMRI; Fig. 3B; (https://www.facebook.com/quartznews/videos/1176008672432833/). Watching these videos should convince anyone that speech is very likely the most sophisticated motor behavior in the human body, rivaled only by the movements of the hand and fingers, and undoubtedly many times more complex than in any other animal species that produces communication sounds. It appears highly unlikely, therefore, that monkeys, lacking most of the necessary articulators, could, in principle, produce human speech with their own vocal tracts, even if they had the neural wherewithal to do it[76]. Simple observation of this miraculous machine in action should be reason enough to at least consider the evolution of human speech production as one of the key events driving the evolution of speech and language.

Talking birds

Previous theories emphasizing the crucial importance of speech production for the evolution of language[77] have been dismissed by making the (perhaps facetious) argument that even parrots can learn to speak. The most publicized of such cases, an African grey parrot named Alex, was reported to have a vocabulary of 150 words, close to that of a 2-year old human infant[78]. Other bird species, such as mynahs and mockingbirds, which can also learn to imitate human speech sounds, have been reported to imitate artificial sounds, like car and machine sounds as well, so the primary purpose of these abilities is unlikely for communication.

Although parrots may be able to produce imitations of human words and utterances, these observations tell us more about human speech perception than about vocal learning in animals: Most imitations are acoustically highly degraded and can only be understood as speech by humans because their own recipient structure (the auditory cortex) is tuned to the expected targets and fills in the rest[79] (just as we can understand sinusoidal speech or speech sampled through a cochlear implant).

Imitation in human infants

How then does imitation work? How do we know what muscles to contract and which articulators to activate, in order to produce the speech sound we have in mind? The sounds stored in auditory cortex during early exposure to speech serve as targets to guide motor production. Auditory feedback is obviously necessary to assure that the produced copy at least approximates the original; the resulting error is then minimized over many iterations. In fact, it may take children several years to perfect their pronunciation. This error minimization is a critical part of the model[3,80]; it doesn’t have to be conscious, but is more likely an implicit learning process. Once an acceptable transformation from perception to production is established by learning, the result is fixed (with an occasional refresher?) and becomes the “forward model”, which can be accessed quickly without sensory feedback (“ballistically”, as it were).

Imitation also works for visual gestures and has given rise to the mirror neuron theory, in which neurons in the premotor cortex of monkeys are activated not only during a certain sensory-guided motor act, like reaching or grasping, but also when the monkey observes another monkey performing the same action[81]. Thus, the mirror system is able to predict the sensory consequences of actions, and this is not restricted to the visual domain.

Motor theories of speech perception

The most prominent version of a motor theory of speech perception was put forward by Liberman and colleagues (1967). What started out as an ingenious explanation for acoustic invariance in speech production by Liberman and by others[82], turned into a radical theory claiming “perception of phonetic structure without translation from preliminary auditory impressions”[83]. In its final instantiation[84], Liberman’s theory “…takes the speech elements to be articulatory gestures that are the primary objects of both production and perception. Those gestures form a natural class that serves a linguistic function and no other. Therefore, their representations are immediately linguistic, requiring no cognitive intervention to make them appropriate for use by the other components of the language system”[84].

Does it follow then that we can understand only sounds that we can produce? More generally, are doable sounds represented differently in the brain? Is the “special” aspect of speech the fact that doable sounds depend on sensorimotor (not just sensory) information? The control experiment would be to test sounds that cannot be imitated without extensive training, such as animal sounds[44].

In the extreme, this would beg the question whether it is actually necessary that the motor system is intact for someone to learn and understand speech? Although it is well known that Broca’s area (as well as other motor-related regions) is active during speech perception[85,86], studies of Broca aphasics, who are unable to produce speech but can understand it, seem to argue against it[87]. However, more studies are needed, as the outcome may depend on the exact site of the lesion (BA44 or 45?) and on the level of speech processing. Clearly, the motor system is of great importance for the initial programming of internal models during language acquisition (as is an intact sense of hearing)[88,89].

Convergence of ventral and dorsal streams

According to the dual-stream model of Rauschecker and Scott (2009) (Fig. 1; Fig. 4), the ventral and dorsal streams converge in Broca’s area in the inferior frontal cortex. The dorsal stream targets BA 44, while the ventral stream ends in BA 45. It has been conjectured that Broca’s region plays an important role in the transformation of highly processed auditory-sensory information into motor-articulatory signals and vice versa[80,90]. The dorsal stream in itself is organized as a system for sensorimotor integration and control and is programmed by sensory input from the ventral stream. The inferior parietal cortex measures the error signal and functions to minimize it.