The effect of the spatial positioning of items on the reliability of questionnaires measuring affect

A largely unexplored area within the field of questionnaire design is the influence of the spatial positioning (layout) of items on the reliability of questionnaires. This is partly because a change in previously disparate thinking around the role of the body in cognition only occurred in the late 1990s and early 2000s (Mirolli & Parisi, 2009; Wilson, 2002). The work of Lakoff and Johnson (1980, 1999) reconceptualised the Western ontological view of the mind’s relation to the human body. They described the mind as being embodied, meaning that thought, although being abstract, is rooted viscerally in bodily experience. This field is now commonly referred to as embodied cognition (Wilson, 2002). Although Lakoff and Johnson’s (1980, 1999) conceptualisation of embodied cognition has its origins in philosophy and linguistics, it does not rule out practical implications and applications. Their ideas on embodied cognition, for example, provoked several experimental studies which investigated the relationship between affect and spatial positioning (layout) of questionnaires (Meier & Robinson, 2004).


Introduction
A largely unexplored area within the field of questionnaire design is the influence of the spatial positioning (layout) of items on the reliability of questionnaires.This is partly because a change in previously disparate thinking around the role of the body in cognition only occurred in the late 1990s and early 2000s (Mirolli & Parisi, 2009;Wilson, 2002).The work of Lakoff andJohnson (1980, 1999) reconceptualised the Western ontological view of the mind's relation to the human body.They described the mind as being embodied, meaning that thought, although being abstract, is rooted viscerally in bodily experience.This field is now commonly referred to as embodied cognition (Wilson, 2002).Although Lakoff andJohnson's (1980, 1999) conceptualisation of embodied cognition has its origins in philosophy and linguistics, it does not rule out practical implications and applications.Their ideas on embodied cognition, for example, provoked several experimental studies which investigated the relationship between affect and spatial positioning (layout) of questionnaires (Meier & Robinson, 2004).
The traditional layout of questionnaires measuring affect negates the sensory motor or visceral aspects that influence the conceptual thoughts of respondents, mainly due to the fact that responding to a questionnaire is predominantly perceived as a mental activity.However, from the embodied cognition perspective, thinking can be seen as rooted in bodily experience (cf.Damasio, 1994;Foster et al., 2008;Lakoff & Johnson, 1980, 1999;Pinker, 2007).In particular, affect is related to the spatial orientation of verticality (Lakoff & Johnson, 1980, 1999;Meier & Robinson, 2004, 2005;Pinker, 2007;Poole & Langston, 2008).In other words, experimental findings on embodied cognition indicate that a person is spatially orientated to position what is positive at the top and what is negative at the bottom.To a lesser extent, there is evidence that affect is also related to the spatial orientation of a horizontal relationship (Foster et al., 2008;Niedenthal, Barsalou, Winkielman, Krauth-Gruber & Ric, 2005).In this case, a person is spatially orientated to position what is positive on the left and what is negative on the right; or (to a lesser extent) to position what is positive near oneself and what is negative far from oneself.

Research purpose and objectives
The relationship between affect and spatial orientation suggests spatial bias: positive is up and negative is down, or positive is right and negative is left.Negating the embodied cognition perspective in the design of questionnaires measuring affect may have a negative effect on the reliability of the questionnaire.Consequently, the two experiments presented in this article sought to ascertain the extent to which congruence between a respondent's spatial orientation (related to affect) and the layout of the questionnaire (spatial positioning of the questionnaire items) may impact the reliability of a questionnaire (measuring affect).

Embodied cognition
Embodied cognition holds that cognitive processes are deeply rooted in the body's interactions with the world and therefore that the mind must be understood in the context of its relationship to a physical body that interacts with the world (Wilson, 2002).Put more simply that, 'states of the body modify states of the mind' (Wilson & Golonka, 2013, p. 58).Theories of embodied cognition thus claim that cognitive activity is fundamentally grounded in a physical context.Consequently, cognition is both supported and constrained by the architecture of our bodies and brains ( Van den Bergh, Schmitt & Warlop, 2011).In this study, embodied cognition provides the epistemological means from which new insights into spatial positioning and affect can be investigated.
Embodied cognition -orientational metaphors: Lakoff and Johnson (1999) believed that reason is metaphorical as well as embodied and therefore has its origins in bodily experiences and neural structures.They therefore viewed the mind as being intrinsically embodied and abstract concepts as being largely metaphorical (Lakoff & Johnson, 1999).Accordingly, they believed that metaphors play a significant role in conceptual thinking and therefore are pervasive not only in language but also in thoughts, experiences and actions (Lakoff & Johnson, 2003).Metaphors are used to organise 'a whole system of concepts with respect to one another' (Lakoff & Johnson, 2003, p. 14).Certain concepts are associated with specific spatial positions or orientations by means of orientational metaphors.Metaphors related to spatial orientation can be referred to as orientational metaphors and will be the focus of the two studies reported here.For example, an upright posture is associated with happiness or positive affect, whereas a slouched posture is associated with unhappiness or negative affect.This indicates that there is a relationship between an individual's feeling state or affect and the individual's posture (Lakoff & Johnson, 1999).Therefore, according to Crous (2007) 'up' may be the 'core metaphor underlying the positive' (p.22), whereas down is associated with what is negative.Consequently, the positive is embodied to be up and the negative is embodied to be down.
Piaget's perspective on how children develop provides one explanation for the relationship between affect and, for instance, vertical physical representation (Meier & Robinson, 2004).Piaget and Inhelder (1969) refer to the period in a child's development, where language is still absent, as the sensorimotor period.Within this period the child's representational functions are not yet developed and the child thus lacks the ability to recall objects that are not present, due to the absence of representations of these objects.These objects are then represented by actions performed by the body and therefore knowledge of these actions can be described as being embodied (Gibbs, 2006).With age the child develops a more abstract method of thinking that is based on the representations in the sensorimotor stage of development (Meier & Robinson, 2004).For instance, the upright posture of body is used to express happiness.This association between representations and affect gives rise to embodied metaphors.As a result, in the English language, metaphors expressing 'happiness' are mostly related to space and orientation (e.g. the upright posture of the body), thus the conceptual metaphor 'happy is up' is formed (e.g.we had to cheer him up) (Kovecses, 1991, p. 33;Lakoff & Johnson, 1980) -and these metaphors, in turn, can affect evaluations we make (Meier & Robinson, 2004).
Embodied cognition -The relationship between affect and spatial orientation: Studies have shown a clear relationship between spatial orientation and affect (cf.Foster et al., 2008;Meier & Robinson, 2004;Poole & Langston, 2008;Wapner, Werner & Krus, 1957).In an early study conducted by Wapner et al. (1957) it was found that success, and therefore also positive affect, was associated with an upward position in space while failure, and therefore also negative affect, was associated with a downward position in space.Later, in a series of studies, Meier and Robinson (2004) demonstrated also the association between affect and vertical spatial positioning.They found that the participants evaluated positive words faster when they were presented closer to the top of the desktop (as opposed to the bottom), whereas negative words were evaluated faster when they were closer to the bottom of the desktop (as opposed to the top).In a second study, Meier and Robinson (2005) discovered that the shift in the respondents' visual space was consistent with their orientational metaphors (i.e. the upper region of the desktop was associated with what is positive and the bottom region was associated with what is negative).
In addition to the studies demonstrating the vertical relationship between affect and spatial orientation, recent studies have provided researchers with a reason to believe that there is a relationship between the horizontal spatial orientation of individuals and affect (Poole & Langston, 2008).Foster et al. (2008) conducted a study focusing on the horizontal relationship between affect and spatial orientation.Participants were required to place six different pegs, three representing positive affect (happy, joyful and surprised) and three representing negative affect (afraid, sad and disgusted), on a pegboard in front of them.According to the results of the study, the participants placed the pegs associated with positive emotions on the left side of the board and the pegs associated with negative emotions on the right side of the board.Furthermore, Foster et al. (2008) found that positive emotions are mediated by the left hemisphere of the brain while negative emotions are mediated by the right hemisphere.Therefore, according to the study positive emotions were found to be associated with a leftward directional bias and negative emotions with a rightward directional bias (Foster et al., 2008).On the other hand, Poole and Langston (2008) replicated the studies conducted by Meier and Robinson (2004) focusing, instead, on the association between horizontal positioning and affect but could not find evidence of this particular association.
Given the lack of consistent findings in the literature, an opportunity exists for further investigation into the relationship between spatial location and affect, and the extent to which it may have 'subtle but pervasive effects on evaluation' (cf.Meier & Robinson, 2004, p. 243).We proposed that these findings may have implications for the reliability of questionnaires presented on a two-dimensional space developed for the measurement of affect.This may include scales developed for the measurement of attitudes.The implication is that the way in which a questionnaire is approached may have a physical basis, which in turn may have implications for the reliability of the questionnaire.

The importance of measurement in organisational settings
Questionnaires are the most commonly used method of data collection in the field of organisational behaviour (Stone, 1978).In the past several decades, a plethora of scales has been developed to assess various attitudes, perceptions or opinions of organisational members (Hinkin, 1995).This information is often used to inform decisions about selection, advancement, training and culture, among other organisational functions.However, the adequate measurement of abstract constructs is perhaps the greatest challenge in conducting research in organisations (Hinkin, 1998).Measurement problems (i.e.problems with the reliability and validity of measures) continue to lead to difficulties in interpreting the results of abstract measures (Hinkin, 1995(Hinkin, , 1998)).When the results of such scales cannot be accurately interpreted, this impedes the ability of practitioners to make informed decisions.Thus, it is crucial that the measures on these survey instruments adequately represent the constructs under examination (Hinkin, 1998).In particular, Hinkin (1995) identified low internal consistency reliability as one of several factors which continues to threaten our understanding of organisational phenomena.

Reliability
Reliability refers to the consistency of a measure (Gregory, 2007) and is a necessary condition for validity (Kerlinger, 1986).In the social sciences, near perfect consistency in measurement is difficult to achieve but is crucial for ensuring the validity and measurement precision of a questionnaire (Muchinsky, Kriek & Schreuder, 2005).Reliability is best described as a continuum that ranges from minimal repeatability or inconsistency to near ideal consistency of results.Psychological measurements are typically positioned between the extremes of the continuum (Gregory, 2007).In this regard, careful consideration needs to be exercised in order to secure optimal reliability in all forms of testing.There are many factors which affect reliability; these factors may be categorised into five groups: 1) factors inherent to the individual, 2) the social/cultural setting, 3) the environment, 4) the questionnaire itself and 5) the nature of the sample.Some of these factors are interrelated (Ercan, Yazici, Ocakoglu, Sigirli & Kan, 2007).
In the literature, the notion of embodiment has not been acknowledged as a probable factor affecting reliability.However, the experimental findings which show that a person is spatially oriented to position what is positive at the top and what is negative at the bottom, and also to position what is positive to the left and what is negative to the right, imply that this may be the case.Based on the background and the proposed problem statement, the following research question was formulated: Does congruence between a respondent's spatial orientation and the spatial positioning of a questionnaire lead to higher reliability of a questionnaire measuring affect?We hypothesised that when there is congruence between a respondent's spatial orientation (related to affect) and the spatial positioning (layout) of a questionnaire, the reliability will be higher than in the case of incongruence.The principal objective of the two experiments reported here was, thus, to ascertain the extent to which congruence between a respondent's spatial orientation (related to affect) and the layout (spatial positioning of a questionnaire) may impact on the reliability of a questionnaire measuring affect.

Research approach
The research comprised a controlled experimental research design, where the reliability of the affect scale was the dependent variable and the spatial positioning of the items in the scale was the independent variable.The spatial positioning of the items was manipulated in different layouts of the affect questionnaire, and the differences in the reliability in respect to the different layouts were investigated.

Research participants
A total of 1825 participants from a metropolitan university in Johannesburg completed the self-report questionnaire online in computer laboratories.In return for their participation participants were offered the opportunity to be entered into a competition whereby they could win a small prize.The mean age of the participants was 21.2 years (SD = 4.08).There were 1045 women (57.3%).

Measuring instruments
Six different layouts of a questionnaire measuring affect were designed to obtain information with regard to the relationship between the spatial positioning of items (as related to the spatial orientation of the participants) and the reliability of a questionnaire measuring affect.The first section of each questionnaire pertained to biographical information.The second section consisted of 30 bipolar items that tapped into self-affect (Experiment 1) and consumer affect or attitude (Experiment 2).The questionnaire scale was created using carefully constructed sets of bipolar pairs of adjectives, anchored at the ends of the continuum.The same items and order of presentation of the items were used throughout all six versions of the questionnaire.In Experiment 1, the participants were required to indicate, on a five-point scale the extent to which the adjective on either side of the scale represented how they felt about themselves (see Table 1).
In Experiment 2, the participants were required to indicate, on a five-point scale the extent to which the adjective on either side of the scale represented how they felt towards a popular consumer sportswear brand (see Table 2).
The six different layouts of the affect scale consisted of three horizontal layouts (representing horizontal spatial orientation; see Table 3) and three vertical layouts (representing vertical spatial orientation; see Table 4).Each item in the three horizontal layouts of the scale consisted of two bipolar adjectives, one positioned on the left and the other on the right of the continuum.In the positive to negative horizontal layout (horizontal layout 1), the positive adjective was positioned on the left of the continuum and the negative adjective was placed on the right side of the continuum.For the negative to positive horizontal layout (horizontal layout 2), the positioning was reversed with the negative adjective on the left and the positive adjective on the right of the continuum.A combination of the positioning of the items in the aforementioned layouts was used in the mixed horizontal layout (horizontal layout 3).
The bipolar adjectives used in the horizontal layouts of the scale remained the same for the vertical layouts of the questionnaire.However, one adjective was positioned at the top of the continuum and the other adjective at the bottom of the continuum.Each item in the vertical positive to negative layout (vertical layout 1) consisted of a positive adjective at  the top of the continuum and a negative adjective at the bottom of the continuum.For the negative to positive vertical layout (vertical layout 2), the negative adjectives were positioned at the top of the continuum and the positive adjectives were positioned at the bottom of the continuum.A mixture of the item positioning in these two layouts was used in the mixed vertical layout (vertical layout 3).

Research procedure
The questionnaire was administered online in computer laboratories.Students on university campus have access to more than 500 computers in computer-based laboratories.
As students entered these laboratories they were approached to complete the questionnaire for research purposes.
Participants were randomly assigned to one of the six different layouts of the questionnaire.Each condition had roughly 300 participants.In terms of group equivalence, the sample was distributed relatively equally across the six layouts of the questionnaire, for education level, gender mix and age distribution.

Results
Experiment 1

Scale structure
A preliminary factor analysis revealed that the scale measured a strong general affect factor underlying all 30 items for four of the six layouts.However, for the vertical layout 3 and horizontal layout 3, the results suggested two factors each.This was attributed to the fact that the first factor of both layouts consisted of predominantly negative adjectives, positioned on the left or on the top of their respective scales.
The second factor for both mixed layouts, conversely, consisted of predominantly positive adjectives that were placed on the left or on the top of their respective scales.This suggested that the items were grouped into factors based on the spatial positioning of the emotionally charged adjectives.These results are consistent with the findings of Spector, Van Katwyk, Brannick and Chen (1997).They showed that affect scales that contain items reflecting both negative and positive emotions typically form factors based on the direction of item wording.However, while the loadings on the second factor with regard to horizontal layout 3 and vertical layout 3 indicated the presence of a second factor, there was no strong evidence indicating that the second factors were substantive.

Comparison of reliability
A pairwise comparison of the Cronbach's alpha reliability coefficients were done by means of Feldt's test of differences in independent reliability coefficients (Charter & Feldt, 1996; see Table 5).Only two comparisons were statistically nonsignificant (p > 0.05), namely, that of the horizontal layout 1 (a = 0.927) and horizontal layout 2 (a = 0.924), and that of horizontal layout 3 (a = 0.884) and vertical layout 3 (a = 0.864).This suggests that differences in the reliability coefficients between these layouts may not be due to the spatial positioning of emotionally charged adjectives.

Discussion
Experiment 1 confirmed that when there is congruency between the spatial orientation of affect and the spatial positioning or layout of a questionnaire, its reliability was higher.What is significant about the statistical analysis is that the reliability of the questionnaire seemed to be influenced by the layout rather than the content.The results indicated that the highest reliability for the vertical scales was that of vertical layout 1 (positive to negative).The difference in reliability between the three vertical layouts was also found to be statistically significant, indicating that congruence between vertical spatial orientation (related to affect, i.e. that up is associated with the positive and down with the negative.)and the spatial positioning (i.e. the vertical layout) of the questionnaire measuring affect had an influence on the reliability of the questionnaire.These findings are consistent with those of Fisher (1964), Meier and Robinson (2004), Meier, Robinson and Caven (2008), Poole and Langston (2008), Setic and Domijan (2007), Stepper and Strack (1993) and Wapner et al. (1957).The results further indicated that there was statistically no difference between the reliability coefficients of the horizontal layouts, excluding the mixed version, which corroborates the findings of Poole and Langston (2008), who could not determine congruency between affect and the horizontal spatial position.In other words, in this experiment no significant difference was found between the horizontal positive to negative layout and the horizontal negative to positive layout, indicating that congruence between an individual's horizontal spatial orientation (related to affect, i.e. that left is associated with the positive and right with negative) and the spatial positioning (i.e.horizontal layout) of the questionnaire had little influence on the reliability of the questionnaire.Therefore, although horizontal layout 1 had a higher reliability, the difference between this reliability and that of horizontal layout 2 was not significant and therefore not meaningful.
Finally, given that theory on embodiment and evidence from several empirical studies suggests that people are spatially oriented to vertically position what is positive at the top and negative at the bottom, and less so to horizontally position what is positive on the left and what is negative on the right, one would have expected the vertical positive to negative layout to have the highest reliability.However, this was not the case.A possible explanation may be due to the familiarity of a horizontal scale among the participants, who may have been conditioned to answer a scale in a horizontal layout rather than the unusual vertical layout.

Experiment 2
Experiment 2 was primarily designed to replicate central aspects of Experiment 1.However, Experiment 2 focuses on the participants' disposition towards a popular consumer brand (external to the self).A popular sportswear brand was used as it is one of the most recognisable brands among consumers in South Africa (Doke, 2008;Naidoo, 2008).

Scale structure
A preliminary factor analysis revealed that the scale measured a strong general attitude factor underlying all 30 items for four of the six layouts.However, for the vertical layout 3, and horizontal layout 3, the results suggested two factors each, as in Experiment 1.
In other words, in this experiment there were no significant differences between the three vertical layouts, indicating that congruence between an individual's vertical spatial orientation (related to affect, i.e. that up is associated with the positive and down with the negative) and the spatial positioning (i.e.vertical layout) of the questionnaire had little influence on the reliability of the questionnaire.A possible explanation as to why the results for Experiment 2 differed from those in Experiment 1, may be that participants in Experiment 1 focused internally (on the self), whereas participants in Experiment 2 focused on a consumer brand external to the self.We postulate that focusing internally (on the self) is perceived as more important and that this results in participants giving more attention to self-evaluations.The conceptual metaphor 'more is up' as spoken about by Lakoff and Johnson (1980), provides an example of another spatial orientation associated with verticality that may account for the fact that the differences in reliability for the vertical layouts, in Experiment 2, were found to be insignificant.In respect of the horizontal layouts, the results were similar to those found in Experiment 1.

Outline of results
The main aim of these studies was to determine the impact of the spatial positioning of the items, on the reliability of a questionnaire measuring affect.Both Experiment 1 and Experiment 2 found evidence to support the hypothesis, namely, that greater congruence between a respondent's spatial orientation (related to affect) and the spatial positioning (layout) of a questionnaire leads to higher reliability on the questionnaire.Furthermore, the results of Experiment 2 found that spatial-related conceptual processing was more likely to be triggered when participants were asked to evaluate their feelings towards themselves (internal/near), as opposed evaluating their feeling towards an object (external/far).

Practical implications
Overall, this research demonstrates that congruence between the spatial positioning of items and the spatial orientation of the respondent can impact the reliability of questionnaires measuring affect.These findings may serve to create awareness of the influence of the spatial positioning of items as a confounding variable in the questionnaire design.Specifically, the results indicated that items should consistently be positioned from either positive to negative adjectives or negative to positive adjectives throughout the scale.A mix of items ranging from positive to negative adjectives and negative to positive adjectives may be counterintuitive and confuse participants, consequently reducing the reliability of the questionnaire.This is in line with Barnette's (2000) findings, which showed that the use of a mix of items positioning is probably much too confusing for many respondents and is therefore not a recommended procedure.

Limitations and recommendations
A replication of this study is suggested to confirm the results of these experiments.Furthermore, research suggests that a replication study should consider the use of negatively charged adjectives.Barnette (2000), Cordery and Sevastos (1993); Schriesheim and Eisenbach (1995); and Schriesheim and Hill (1981) indicate the use of negatively worded items can often negatively affect the reliability of measures and can cause the appearance of artificial factors.Given the negative implications of using negatively worded items, it may be beneficial to consider a replication of this study which excludes the use of the negatively charged adjectives -and then compares the results.While this research has provided some initial evidence for the hypothesised account for the influence of spatial positioning on questionnaires measuring affect, consideration can be extended to other types of questionnaires (e.g.personality questionnaires) and/or other constructs.In particular, the findings suggest that future research should further explore the relationship between a respondent's spatial orientation and the spatial positioning of the construct (internal versus external).

Conclusion
In conclusion, this research complements previous studies (Meier & Robinson, 2004;Meier et al., 2008;Poole and Langston, 2008;Setic & Domijan, 2007) by confirming the metaphorical representation of affect and enhances our The significance of the difference in the reliability between the six layouts of the affect scale.

Compared layouts Significance of the difference in Cronbach's alpha
Horizontal positive to negative and vertical positive to negative Significant differences Horizontal negative to positive and vertical negative to positive Significant differences Horizontal mixed and vertical mixed Non-significant differences Horizontal positive to negative and horizontal negative to positive Non-significant differences Horizontal positive to negative and horizontal mixed Significant differences Horizontal negative to positive and horizontal mixed Significant differences Vertical positive to negative and vertical negative to positive Non-significant differences Vertical positive to negative and vertical mixed Non-significant differences Vertical negative to positive and vertical mixed Non-significant differences Horizontal positive to negative and vertical negative to positive Significant differences Horizontal positive to negative and vertical mixed Significant differences Horizontal negative to positive and vertical positive to negative Significant differences Horizontal negative to positive and vertical mixed Significant differences understanding of embodiment-related conceptual processing and its subsequent influence on self-evaluations versus external evaluations on an non-conscious level, specifically in relation to measuring affect.Further research is necessary to confirm these effects, expanding to include the relationship between congruence between a respondent's spatial orientation (related to affect) and the spatial positioning of the construct (internal/near versus external/far).

TABLE 1 :
Bipolar adjectives utilised in the affect scale (internal).

TABLE 2 :
Bipolar adjectives utilised in the affect scale (external).

TABLE 3 :
The positioning of the items in respect of the horizontal layouts of the affect scale.

TABLE 4 :
The positioning of the items in respect of the vertical layouts of the affect scale.