The Effect of Reading Direction on Emotion Recognition: A Pilot Study

Hemispheric Processing of Words and Sentences

The role and function of visual fields (especially the foveal region) are important and well-studied topics in vision research of reading. Although it is an accepted fact that parafoveal regions project contralaterally, there is an ongoing debate about whether the foveal region functions as a whole in which information is projected to both hemispheres or whether information is processed laterally similar to parafoveal vision [3, 4, 5].

On the sentence level, contralateral projection to the hemispheres is well supported by evidence found by Pollatsek, Bolozky, Well, and Rayner (1981) [6] who arrived to the conclusion that English readers’ perceptual span is asymmetric in a way that information falling to the right of the fixated word is more extensive (15–16 characters) than that falling to the left side (3–4 characters), thus occupying more area in the right visual field (RVF) than the left visual field (LVF). (The effect is reversed for readers reading Hebrew text, which is read from right-to-left.) On the word (foveal) level, however, the picture is not so clear. There are two competing theories about hemispheric projection linked to the foveal region. The bilateral projection theory (BPT) posits that the information from the fovea region will project both to the left and right hemispheres regardless the point of fixation [4]. The split fovea theory (SFT) argues that the fovea is split in the middle, with information to the right of the center of the fovea projecting to the left hemisphere (LH) and information to the left of the center of the fovea projecting to the right hemisphere (RH) [3].

Reading and Hemispheric Processing

When we read, our fixation typically falls to the initial one-third of the word, called the preferred viewing location (PVL). This also is where fastest and most accurate word recognition occurs and is the optimal viewing point (OVP) for word identification [7, 8, 9]. If the BPT is correct, then word-level information processing is done by both hemispheres equally, while sentence-level processing is done mostly by the LH. If the SFT is the valid theory, however, then processing is mostly done by the LH on both the word and sentence level. Whichever the case, the RVF and the LH are used more in reading than the LVF and the RH; therefore, we consider the direction of reading to be possibly an asymmetric physical stimulation in the case of (normal, everyday) left-to-right reading.

RVF Superiority in Reading

The RVF superiority in reading found in the previous experiments is attributed mainly to the hemispheric specialization of the LH for processing language-related stimuli (i.e., the Broca area and the visual word form [VWF] area), but growing evidence suggests that it is, in part, a result of perceptual learning. In a series of experiments, Nazir, Ben-Boutayab, Decoppet, Deutsch, and Frost (2004) [10] reinforced this supplementary explanation. They argued that because of the time people spend “practicing” discriminating orthographic material and because of the regular use of the close-to-center landing site, reading improves in a way such that people will process more and more information in the visual field identical to the direction of visual scanning (i.e., in left-to- right reading the right visual field). Because of that, those who read in a left-to-right direction for a long time will use their RVF more and more compared to their LVF, thus making their LH more active compared to their RH while reading.

Similarly, reinforcing the previous assumption Wong and Hsiao (2012) [11] found that the OVP effect for reading musical notes was a consequence of long-term reading practice of musical notes. Novice, intermediate, and expert musicians read three-note sequences in the experiment. Their fixation point fell more and more to the left of the center of the sequence as their level of experience increased. From these results, it seems likely then that the LH’s superiority in processing written material directed in a left-to-right direction is partly due to a learning process, namely to our habit of reading in this direction. In this case the scale of the asymmetry created by left-to-right reading is the product of a learning mechanism instead of a pure, innate brain activation mechanism. Now, we take a small detour to familiarize with emotion processing theories then we move on to make our hypothesis based on our assumptions.

Hemispheric Processing of Emotions

Hemispheric lateralization of emotions remains a debated theory in the field of cognitive neuroscience despite several studies that used varied methods in an attempt to support cerebral asymmetries. Currently two competing theories attempt to explain these findings: the valence hypothesis (VH) and the right hemisphere hypothesis (RHH).

According to the VH, unilateral stimulations activate the contralateral hemisphere, which then affects emotional experience (e.g., mood, performance). During a left-sided stimulation, the right hemisphere, which is specialized in negative emotion processing, will be activated. During a right-sided stimulation, the left hemisphere, which is specialized in positive emotion processing, will be activated. The RHH, however, suggests that the right hemisphere is responsible for all emotion processing, regardless of valence.

Researchers commonly try to deliberate between these two theories by using visual field (VF) experiments in which activation is attributed to the hemisphere opposite the VF (i.e., if the stimulus is presented in the RVF, then its processing is attributed to the LH).

Despite the many research studies conducted in this area, the results are still inconclusive. On one hand, there are findings supporting the theory that positive emotions are better and faster recognized when presented in the RVF (i.e., the left hemisphere processes positive emotions) and vice versa for negative emotions [12, 13, 14]. On the other hand, in a divided visual field experiment, when the stimuli of facial expressions are simultaneously presented in both the right and left visual fields, participants tend to recognize the target expression better and faster when it is presented in the LVF [15, 16, 17]. The findings of these studies have been interpreted in favor of either the VH or the RHH, so the debate remains unresolved.

Physical Stimulations and Emotions

Schiff and Lamon conducted a series of experiments in which emotional processing was not tested by stimuli presentations to visual fields but by using physical stimulations to generate emotional experience. They were among the first researchers who found a laterality effect between physical activity and emotional experience. Small unilateral muscle contractions of the face [1] and the hand [2], unilateral nostril breathing [18], and unilateral vibrotactile stimulations [19] all influenced emotional experience, depending on the side of contraction. Stimulations on the left side resulted in negative states of emotion while stimulations on the right side resulted in positive states of emotion. The authors explained their results on the basis of the VH.

Present Research

The goal of this research was twofold. First, based on our theoretical background that VF use is unequal during reading, we used the reading process as a physical stimulation to address the inconsistent findings of lateralized emotional processing, and at the same time prove that direction of reading can function as a unilateral stimulation. We expected that the act of reading would generate distinguishable emotional processing reaffirming its unilateral stimulating nature and that it would reaffirm or deny the competing models of hemispheric lateralization of emotions.

Considering the SFT theory and the results of Pollatsek et al. (1981) [6], we hypothesized that the asymmetric use of VFs during reading would generate a lateralized hemispheric activation that could be captured through emotion processing as in the Schiff and Lamon experiments.

Schiff and Lamon (1989, 1994) [1, 2] induced stimulations of both the right and left side and then compared the effects. In the current study, however, it was difficult to set up a similar inverse condition (i.e., left- to right [LR] reading vs. right-to left [RL] reading) because the capacity to read in a different direction was lacking. Incapable of constructing an experimental situation where LR and RL reading conditions could be compared on the same sample in a within-subject design, we chose to compare the conditions of LR and top-to-bottom (TB) reading. Japanese can be read in both LR and TB

directions, using the same graphemes. Moreover, in this setting, the sample shared the same lingual and cultural background contrary to settings in which two languages are read by the same person [6] or in which different languages are read by different people. Kajii and Osaka (2000) [20] found that the OVP effect disappeared during TB reading in a Japanese sample, which means that while reading in a TB direction both VFs are used equally for information processing, preventing the hemispheric lateralization from occurring.

For testing, we chose a facial expression recognition task because it is easy to measure and the results are simple to interpret. We made the following predictions based on information from competing emotion- processing models such as the VH, RHH, and the approach/withdrawal system (AWS) theory, and from studies by Demaree, Everhart, Youngstrom, and Harrison (2005), Fusar-Poli et al. (2009), and Harmon-Jones, Vaughn-Scott, Mohr, Sigelman, and Harmon-Jones (2004) [21, 22, 23, 24, 25].

First, due to the hemispheric lateralization created by reading, we expected that the left-weighted asymmetric use of the LH during LR reading would result either in superior and faster recognition of emotions that are attributed to the LH (e.g., anger1, happiness, surprise) or inferior emotion recognition in general in concordance with the RHH.

Second, in the TB reading condition where the visual fields are used evenly for information processing, we expected either better performance and faster recognition of emotions attributed to the RH (e.g., sadness, disgust, fear) or superior emotion recognition in general in accordance with the RHH.

Participants

Thirty Japanese people (17 men and 13 women) participated in the experiment. Their mean age was 30.8 (SD ± 6.228). They were all residents of Tokyo and most of them were studying at Hitotsubashi University. Recruitment was done both through advertisement and

1We attributed anger to the LH based on studies conducted by Harmon-

Jones and Allen (1998), Harmon-Jones, Sigelman, Bohling, and Harmon-Jones (2003), and Harmon-Jones et al. (2004).

personal inquiry. Those who signed up for the experiment via advertisement received 1500¥ per session.

Materials

The stimulus material consisted of two standardized test batteries that measure basic emotion recognition. The first was the Japanese Female Facial Expression (JAFFE) measure by Lyons, Akamatsu, Kamachi, and Gyoba (1998) [26]. The second was the Asian Reading the Mind in the Eyes (ARME) instrument by Adams et al. (2010) [27], which was used to measure more complex emotion and state recognition. Image material (Stimuli): We selected 48 pictures from the JAFFE database: six faces for each basic emotion and 12 neutral faces. The JAFFE images are of female actors showing facial expressions of the six basic emotions and neutral expressions.

We administered all 36 pictures from the ARME, which is an adaptation of the Reading the Mind in the Eyes (RME) test originally developed by Baron-Cohen, Wheelwright, Hill, Raste, and Plumb (2001) [28] to measure mind reading capabilities (considered an advanced theory of mind test of typically developed adults and adults with an autism spectrum disorder. We decided to use ARME pictures as a general emotion recognition variable because they present complex emotional and mental states [29] and their valence is less confined than basic emotions, thus allowing a less valence-specific processing. Participants were presented a series of images of the eye region of the face of an equal number of female and male actors. Four complex mental state descriptors were presented around each image, one at each corner, with one being the correct target word describing the mental state of the actor. Participants were instructed to choose the word that best described what the person in the photo was thinking or feeling.

The images for both batteries were presented on a 13- inch laptop screen, administered by either DMDX (for Windows) or Psyscope (for Mac) software. The instructions and the emotion labels used during the presentation were written in the same direction as the text the participants read during the reading task.

The pictures were grouped into two image sets in order to be able to measure the performance of the same test participant on both reading conditions. Each set contained three JAFFE pictures of each of the six basic emotions, six JAFFE pictures of neutral faces, and 18 ARME pictures.

The tests were scored by totaling the number of items (images) for which the participant correctly identified the mental state and basic emotion portrayed. Reaction times also were registered for each image task. Reading material (conditions): In the experiment, we used four reading texts that differed in two dimensions: their content (Text A or Text B) and their direction of reading (left-to-right or top-to-bottom). This created the following conditions: (a) Text A read from left to right, (b) Text A read from top to bottom, (c) Text B read from left to right, and (d) Text B read from top to bottom. The texts were 24–25 pages long and were put together from randomly selected tales from a Japanese website. All tales appeared in only one of the texts and were written in Japanese Hiragana.

Procedure

Each participant was tested individually in a soundproof room during two sessions. During the first session, participants listened to Sergei Rachmaninoff’s Allegro Vivace for 3 minutes to ensure a similar mood for each person before starting the experiment. Then the participants were given one of the compilations of Japanese tales to read for 30 minutes. In the second session, participants completed the facial expression recognition task involving 24 pictures from the JAFFE database and 18 pictures from the ARME test. The experiment was repeated a week later. The order of the conditions (including image sets, starting condition, and reading materials) was randomized and counterbalanced.

Statistical Method

Response times from trials with errors were excluded, and those response times greater than two standard deviations above the mean were ignored as outliers. We ran a repeated measures ANOVA for both performance and response time data of the ARME and JAFFE pictures. For the ARME pictures, we used a mixed 2x2 design, in which the within-subject variable was reading direction and the between-subject variable was gender. For the Interestingly, we found an unexpected interaction between gender and reading direction in the performance scores for happy faces (F[1, 28] = 6.283, µ2 = .183, p < .05) (see Figure 3). The interaction between the two variables is significant. Although men are better at recognizing happy faces in the LR reading condition, the same is true for women in the TB reading condition.

JAFFE pictures, we added emotion as a third dimension, resulting in a 2x2x7 design. We also ran separate repeated measures ANOVA for each emotion, using the same mixed 2x2 design with reading direction as the within-subject variable and gender as the between- subject variable.