Imaging
Brain Imaging Group
- Hyperscanning-MEG system Social interaction is supposed to seriously affect child development, and simultaneous multisubject recording is expected to address the issue how multiple brains interact each other. This technique is called hyperscanning. We built the first hyperscanning system to examine brain-to-brain interactions between a mother and her child using two magnetoencephalography (MEG) scanners in collaboration with Kanazawa University and Yokogawa Electric Corporation. Figure 3 left shows the hyperscanning-MEG system equipped with video projection devices. Now, we are examining brain activities during their interaction, such as face-to-face communication (Figure 1 right) and mutual coordination.
- Atypical auditory processing systems in children who have difficulty in social interaction Rapid temporal processing of speech signals is important in daily colloquial environments and enhances the basic ability for social interaction (Figure 2 left). The earliest cortical component of the auditory evoked field (AEF) (i.e., the P1m) is a prominent component in 1- to 10-year-old children. Recently, a child-customized MEG device has facilitated the acquisition of bi-hemispheric recordings even in young children. Using this new device, we demonstrated that children with autism spectrum disorder (ASD) exhibited significantly less leftward lateralization in their P1m intensity compared with the typically developing (TD) children (2 to 5 years old) (Figure 2 right) and that the P1m intensity is linearly increasing during 3 to 7 years old periods in TD (Figure 3 left). In addition, we reported that language-related performance was reflected in the dipole intensity of the P1m in TD young children (2 to 5 years old) even after controlling for confounding factors (e.g., age). Our recent study with wider age range group (3 to 10 years old) demonstrated an inverted U-shaped growth curve for the P1m dipole intensity in the left hemisphere in TD children (Figure 3 right), whereas more diversified age-related distribution was observed in children with ASD. These our results from MEG studies contribute to our understanding of diversified pathophysiological mechanisms in the central nervous systems in young children who have difficulty in social interaction.
- Atypical brain spontaneous dynamics in children who have difficulty in social interaction It is necessary to study the pathophysiology of ASD in young children because aberrant white matter pathway development may occur during infancy; however, it is challenging to measure brain connectivity using MRI in conscious young children with ASD because they are not always cooperative or patient. On the other hand, MEG has advantages over other neuroimaging techniques for use in young children, including fewer constraints and an absence of unpleasant sounds. In addition, most of the signal recorded by an MEG sensor reflects cortical activity located around the sensor (within a short distance). This spatial property of MEG makes it particularly advantageous in studies investigating brain laterality, local dynamics or the long-range connectivity of neural oscillations, even for sensor level analysis. Using a custom child-sized MEG system, our sensor-level study demonstrated significantly reduced connectivity between left-anterior and right-posterior areas in 3- to 7-year-old children with ASD (Figure 4 left). The TD group displayed significantly rightward lateralized brain oscillations in the theta-1 frequency band power compared to the ASD group. On the other hand, unusual brain lateralization was observed in young children with ASD (Figure 4 middle).. Multiscale entropy (MSE) analysis can characterize the complexity inherent in brain signal dynamics over multiple temporal scales in the dynamics of neural networks. Using this method, we revealed an age-related increase of brain signal variability in a specific timescale in TD children, whereas atypical age-related alteration was observed in the ASD group (Figure 4 right).
- Parent-child interaction in daily life We demonstrate that the social skills and communication subscales in the autism spectrum quotient (AQ) are more sensitive as autism traits in a Japanese sample and to demonstrate that some autistic traits in mothers are specifically associated with disturbances in the social ability of their young children with ASD, as measured by the Social Responsiveness Scale (SRS) score (Figure 5 left). In addition, as shown in Figure 5 (right), we demonstrate that autistic symptom mitigation in ASD children was associated with increased empathy levels measured by EQ score in their parents. Future studies can include interventional procedures to further our understanding of this topic.
- How the human brain develops self-consciousness and matures synchronous/asynchronous behavioral skills One's own face represents oneself, and thus recognizing the face as one’s own is fundamental for the formation of self-identity. We have shown using self-face recognition task in adults that inferior fronto-parietal network mediated by the inferior branch of the superior longitudinal fasciculus fiber tract (SLF III) in the right hemisphere dominantly involves self-face recognition (Morita et al. 2015). Surprisingly, the same neuronal substrates in the right hemisphere were commonly shared by corporeal awareness task (cf. Naito et al. 2016; Amemiya & Naito, 2016). The series of studies have revealed a lateralized function of the right ‘non-dominant’ hemisphere in the adult’s brain by showing that corporeal awareness and self-identification share neuronal substrates in the right inferior fronto-parietal network connected by the SLF III (Fig.1). We further showed that the right-side dominance of inferior fronto-parietal activity for self-face recognition emerges after junior high school age but not at elementary school age, and that the degree of right-sided dominance correlates with individual self-consciousness score (Morita et al. 2016). Thus, the development of one’s self-consciousness leading to self-identity seems to be somehow associated with the emergence of the right-lateralized activity in the inferior fronto-parietal network mediated by the SLF III.
Figure 1. The hyperscanning MEG system and preliminary experiment (Hirata et al. 2014 Front Hum Neurosci.)
Figure 2. Left: Brain response to the human voice (Yoshimura et al. 2013 PlosOne). Right: Autism spectrum disorder exhibited significantly less leftward lateralization in their P1m intensity compared with the typically developing children (Yoshimura et al. 2013, Mol Autism)
Figure 3. Left: Trajectories of the P1m amplitude in the left hemisphere in typically developing children (Yoshimura et al. 2014 NeuroImage). Right: an inverted U-shaped growth curve during 3 to 10 years old for the P1m dipole intensity in typically developing children (Yoshimura et al. in press , Autism Research)
Figure 4. Reduced connectivity between left-anterior and right-posterior areas (red solid line) in 3- to 7-year-old children with ASD (Kikuchi et al., 2015 Social cognitive and affective neuroscience). Middle: unusual brain lateralization in theta band power was observed in young children with ASD (Hiraishi et al, 2015, Psychiatry Clin Neurosci). Right: an age-related increase of brain signal variability analyzed by multiscale entropy in a specific timescale in TD children, whereas atypical age-related alteration was observed in the ASD group (Takahashi et al, in press, Human brain mapping).
Figure 5. Left: Broader autism phenotype in mothers predicts social responsiveness in young children with autism spectrum disorder (Hasegawa et al., 2015, Psychiatry Clin Neurosci). Right: Autistic symptom mitigation in ASD children was associated with increased empathy levels in their parents (Hasegawa et al., 2015 Psychiatry Res).
We believe that an ability of controlling synchronous and asynchronous actions to external cues is a precursor to develop socially interactive skills with others. In order to understand its developmental processes, fMRI study was conducted at CiNet NICT in the campus of Osaka University on child and adult participants. They were asked to control synchronous and asynchronous hand actions to periodic sounds (rhythm). We found that control of these hand actions in children (aged 8-11) is in still immature state of development. They are not able to keep controlling these actions at precise timings required in the task, and its neural basis was immature functional connectivity between the cerebral cortex and the cerebellum (Naito et al. 2016), which seems to be corroborated by the immature anatomical cerebro-cerebellar connections.
Psychological and Behavioral Experiment Group
- Cognitive development based on sensorimotor predictive learning Human infants acquire various cognitive abilities in the first few years of life. Although behavioral changes in infants have been closely investigated, the underlying mechanisms for the development are not yet completely understood. We proposed a theory of cognitive development based on predictive learning of sensorimotor information (Figure 6). Our robot experiments demonstrated that various cognitive functions such as self-other cognition, imitation, goal-directed action, and prosocial behavior can be acquired through sensorimotor predictive learning, whereas traditional studies have modeled them using different architectures (Nagai & Asada, 2015).
- Modeling of the emergence of mirror neuron system The mirror neuron system (MNS) is a group of neurons that discharge both when people are executing an action and when they are observing the same action performed by other individuals. Such "mirroring" function enables people to better understand and anticipate the goal of others' actions and to imitate them. We developed a neural network model by which a robot acquires the MNS through sensorimotor predictive learning (left in Figure 7). The model learns the association between motor neurons and visual representations, while the visual acuity gradually improves as in infants. Our experiment demonstrated that the robot learned to recognize the self and other based on their predictabilities and that the MNS emerged as a byproduct of the development of self-other cognition (right in Figure 7) (Nagai et al., 2011; Kawai et al., 2012).
- Modeling of the emergence of prosocial behavior Although it is well known that infants tend to help other persons even without expected rewards, what motivates them to behave altruistically is an open question. We hypothesized that the minimization of prediction error is a potential mechanism for infants to show prosocial behaviors. Our hypothesis suggests that infants first learn to generate actions through sensorimotor predictive learning and adopt their predictor to recognize how the sensory state changes over time (left in Figure 8). Then, if other persons fail in achieving an action goal, infants detect prediction error, which triggers their own actions in order to minimize the prediction error. Our experiment demonstrated that a model based on the minimization of prediction error enabled a robot to produce prosocial behaviors when the robot detected actions in its action experiences (right in Figure 8) (Baraglia et al., 2014).
- Modeling of the emotional development through infant-caregiver tactile interaction Behavioral studies of infants have shown that pleasant/unpleasant state gradually differentiates into emotional states such as happiness, anger, and sadness over the first few years of life. We proposed a probabilistic neural network model, which hierarchically integrates multimodal information, to reproduce emotional development observed in infants (left in Figure 9). Emotional states are represented as the activation of neurons at the highest layer of the network. Our key idea is that tactile interaction between an infant and a caregiver plays an important role in emotional development because the tactile sense can inherently discriminate pleasant and unpleasant signals and has a better acuity than in other modalities in early infancy. Our experiment using actual sensory signals from interaction demonstrated that typical emotional categories developed only when the tactile sense was endowed with the above functions (right in Figure 9) (Horii et al., 2013).
Figure 6. Cognitive development in human infants (top) and predictive learning as an underlying mechanism for development (bottom)
Figure 7. Emergence of mirror neuron system through the development of self-other cognition based on predictive learning: a neural network model (left) and a robot experiment (right)
Figure 8. Emergence of prosocial behavior based on the minimization of prediction error: a probabilistic action tree working as a predictor (left) and a robot experiment (right)
Figure 9. Emotional development through infant-caregiver tactile interaction: a probabilistic neural network (left) and an acquired representation of the emotional states (right)
Developmental Brain Simulation Group
In order to understand the interactions between the body, brain and environment that generate versatile behaviors, we need to analyze the relationship between the emerged behaviors and the network structures of the neural system of the brain. We conducted a physical simulation using a snake-like robot with a nonlinear oscillator network and extracted the network structure based on transfer entropy for each different movement. Two kinds of network structures were found depending on the behavior (stable or unstable). We call them causality networks compared to the wired network which is physically fixed. The (un)stable motion is generated by (globally) locally connected (few) several sub-networks consisting of (more) less complex network through (strong) weak interaction with environment. Figure 10 shows these two causality networks which are dynamically transiting each other (left: stable, right: unstable). One speculation is that the unstable and stable states may correspond to the phenomena of artificial consciousness and unconscious- ness, respectively from the perspective of integrated information theory.
This network architecture can be the most fundamental one for CDS because any learning methods or value systems have not been applied to this network, yet. Two kinds of network structures emerge based on the transfer entropy, and therefore this can be regarded as temporal functional differentiation; one is for stable states and the other for unstable ones although Yamaguti and Tsuda showed spatial functional differentiation.
The physical and causality networks may correspond to an anatomical structure of real brain (physical connections of synapses observed in DTI) and functional one (e.g., active brain regions in fMRI under a specific task), respectively. Pesoa showed the functional diversity map of the brain which shows how the same brain regions are diversely utilized for different functions. For example, anterior insula and anterior cingulate cortex show higher functional diversity.
Two found causality networks (stable and unstable) utilize one physical network. Therefore, some parts of the physical network might be utilized in two different states (functions), these functions and networks are dynamically changing. We may find such situations in the following studies.
Figure 10. Two causality networks dynamically transiting each other
Robot Development Group
-
Facial and gestural expressions are a very important and indispensable part of artificial empathy. We need realistic research platforms to enable to realize emotional interaction with caregivers. Figure 11 shows Affetto we have been developing as a research platform with realistic appearance of a 1- to 2-year-old child. The pictures from the left to the right indicate facial expressions of the first generation, physical interaction and internal structure of the upper torso of the second generation. The internal structure was redesigned so that Affetto can absorb physical force both inside (protection) and outside (avoidance of injuring humans). The fourth figure from the left shows a CAD image of upper body: divisions of mechanisms and approximate sizes of the third (current) generation. Final whole body with two legs is shown in the rightmost picture.
Figure 11. Affetto in three generations: facial expressions and physical interaction, the internal structure of upper torso and a whole body with two legs
For the vocal interaction, we have been developing Infant-like Vocal Robot “Lingua” which can generate human-infant-like utterance based on the mechanically designed vocal system with vocal cords, tongue, lips, vocal tract and so on. Figure 12 shows the current status of Lingua. From the left, a figure of the whole appearance, total degree of freedom of the vocal system, vocal cords and tongue mechanism. The right graph indicates utterable voices in the formant space by Lingua (blue dots) and the previous vocal robot (black dots). The former is much better than the latter because Lingua covers the almost range of human infant.
Figure 12. Lingua: a whole appearance, vocal cords and tongue, utterance range in the formant space
PAGE TOP