About the Author(s)


Leigh Leo Email
Department of Industrial Psychology and People Management, University of Johannesburg, South Africa

Sebastian Kolsch
Department of Industrial Psychology and People Management, University of Johannesburg, South Africa

Azaria Beukes
Department of Industrial Psychology and People Management, University of Johannesburg, South Africa

Freddie Crous
Department of Industrial Psychology and People Management, University of Johannesburg, South Africa

Johann Scheepers
Department of Industrial Psychology and People Management, University of Johannesburg, South Africa

Citation


Leo, L., Kolsch, S., Beukes, A., Crous, F., & Scheepers, J. (2016). The effect of the spatial positioning of items on the reliability of questionnaires measuring affect. SA Journal of Industrial Psychology/SA Tydskrif vir Bedryfsielkunde, 42(1), a1303. http://dx.doi.org/10.4102/sajip.v42i1.1303

Original Research

The effect of the spatial positioning of items on the reliability of questionnaires measuring affect

Leigh Leo, Sebastian Kolsch, Azaria Beukes, Freddie Crous, Johann Scheepers

Received: 02 Sept. 2015; Accepted: 24 Feb. 2016; Published: 31 Aug. 2016

Copyright: © 2016. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Orientation: Extant research has shown that the relationship between spatial location and affect may have pervasive effects on evaluation. In particular, experimental findings on embodied cognition indicate that a person is spatially orientated to position what is positive at the top and what is negative at the bottom (vertical spatial orientation), and to a lesser extent, to position what is positive on the left and what is negative on the right (horizontal spatial orientation). It is therefore hypothesised, that when there is congruence between a respondent’s spatial orientation (related to affect) and the spatial positioning (layout) of a questionnaire, the reliability will be higher than in the case of incongruence.

Research purpose: The principal objective of the two studies reported here was to ascertain the extent to which congruence between a respondent’s spatial orientation (related to affect) and the layout of the questionnaire (spatial positioning of questionnaire items) may impact on the reliability of a questionnaire measuring affect.

Motivation for the study: The spatial position of items on a questionnaire measuring affect may indirectly impact on the reliability of the questionnaire.

Research approach, design and method: In both studies, a controlled experimental research design was conducted using a sample of university students (n = 1825).

Major findings: In both experiments, evidence was found to support the hypothesis that greater congruence between a respondent’s spatial orientation (related to affect) and the spatial positioning (layout) of a questionnaire leads to higher reliability on a questionnaire measuring affect.

Practical implications: These findings may serve to create awareness of the influence of the spatial positioning of items as a confounding variable in questionnaire design.

Contribution/value-add: Overall, this research complements previous studies by confirming the metaphorical representation of affect and enhances our understanding of embodiment-related conceptual processing and its subsequent influence on self-evaluations versus external evaluations on an unconscious level, specifically in relation to measuring affect.

Introduction

A largely unexplored area within the field of questionnaire design is the influence of the spatial positioning (layout) of items on the reliability of questionnaires. This is partly because a change in previously disparate thinking around the role of the body in cognition only occurred in the late 1990s and early 2000s (Mirolli & Parisi, 2009; Wilson, 2002). The work of Lakoff and Johnson (1980, 1999) reconceptualised the Western ontological view of the mind’s relation to the human body. They described the mind as being embodied, meaning that thought, although being abstract, is rooted viscerally in bodily experience. This field is now commonly referred to as embodied cognition (Wilson, 2002). Although Lakoff and Johnson’s (1980, 1999) conceptualisation of embodied cognition has its origins in philosophy and linguistics, it does not rule out practical implications and applications. Their ideas on embodied cognition, for example, provoked several experimental studies which investigated the relationship between affect and spatial positioning (layout) of questionnaires (Meier & Robinson, 2004).

The traditional layout of questionnaires measuring affect negates the sensory motor or visceral aspects that influence the conceptual thoughts of respondents, mainly due to the fact that responding to a questionnaire is predominantly perceived as a mental activity. However, from the embodied cognition perspective, thinking can be seen as rooted in bodily experience (cf. Damasio, 1994; Foster et al., 2008; Lakoff & Johnson, 1980, 1999; Pinker, 2007). In particular, affect is related to the spatial orientation of verticality (Lakoff & Johnson, 1980, 1999; Meier & Robinson, 2004, 2005; Pinker, 2007; Poole & Langston, 2008). In other words, experimental findings on embodied cognition indicate that a person is spatially orientated to position what is positive at the top and what is negative at the bottom. To a lesser extent, there is evidence that affect is also related to the spatial orientation of a horizontal relationship (Foster et al., 2008; Niedenthal, Barsalou, Winkielman, Krauth-Gruber & Ric, 2005). In this case, a person is spatially orientated to position what is positive on the left and what is negative on the right; or (to a lesser extent) to position what is positive near oneself and what is negative far from oneself.

Research purpose and objectives

The relationship between affect and spatial orientation suggests spatial bias: positive is up and negative is down, or positive is right and negative is left. Negating the embodied cognition perspective in the design of questionnaires measuring affect may have a negative effect on the reliability of the questionnaire. Consequently, the two experiments presented in this article sought to ascertain the extent to which congruence between a respondent’s spatial orientation (related to affect) and the layout of the questionnaire (spatial positioning of the questionnaire items) may impact the reliability of a questionnaire (measuring affect).

Literature review
Embodied cognition

Embodied cognition holds that cognitive processes are deeply rooted in the body’s interactions with the world and therefore that the mind must be understood in the context of its relationship to a physical body that interacts with the world (Wilson, 2002). Put more simply that, ‘states of the body modify states of the mind’ (Wilson & Golonka, 2013, p. 58). Theories of embodied cognition thus claim that cognitive activity is fundamentally grounded in a physical context. Consequently, cognition is both supported and constrained by the architecture of our bodies and brains (Van den Bergh, Schmitt & Warlop, 2011). In this study, embodied cognition provides the epistemological means from which new insights into spatial positioning and affect can be investigated.

Embodied cognition – orientational metaphors: Lakoff and Johnson (1999) believed that reason is metaphorical as well as embodied and therefore has its origins in bodily experiences and neural structures. They therefore viewed the mind as being intrinsically embodied and abstract concepts as being largely metaphorical (Lakoff & Johnson, 1999). Accordingly, they believed that metaphors play a significant role in conceptual thinking and therefore are pervasive not only in language but also in thoughts, experiences and actions (Lakoff & Johnson, 2003). Metaphors are used to organise ‘a whole system of concepts with respect to one another’ (Lakoff & Johnson, 2003, p. 14). Certain concepts are associated with specific spatial positions or orientations by means of orientational metaphors. Metaphors related to spatial orientation can be referred to as orientational metaphors and will be the focus of the two studies reported here. For example, an upright posture is associated with happiness or positive affect, whereas a slouched posture is associated with unhappiness or negative affect. This indicates that there is a relationship between an individual’s feeling state or affect and the individual’s posture (Lakoff & Johnson, 1999). Therefore, according to Crous (2007) ‘up’ may be the ‘core metaphor underlying the positive’ (p. 22), whereas down is associated with what is negative. Consequently, the positive is embodied to be up and the negative is embodied to be down.

Piaget’s perspective on how children develop provides one explanation for the relationship between affect and, for instance, vertical physical representation (Meier & Robinson, 2004). Piaget and Inhelder (1969) refer to the period in a child’s development, where language is still absent, as the sensorimotor period. Within this period the child’s representational functions are not yet developed and the child thus lacks the ability to recall objects that are not present, due to the absence of representations of these objects. These objects are then represented by actions performed by the body and therefore knowledge of these actions can be described as being embodied (Gibbs, 2006). With age the child develops a more abstract method of thinking that is based on the representations in the sensorimotor stage of development (Meier & Robinson, 2004). For instance, the upright posture of body is used to express happiness. This association between representations and affect gives rise to embodied metaphors. As a result, in the English language, metaphors expressing ‘happiness’ are mostly related to space and orientation (e.g. the upright posture of the body), thus the conceptual metaphor ‘happy is up’ is formed (e.g. we had to cheer him up) (Kovecses, 1991, p. 33; Lakoff & Johnson, 1980) – and these metaphors, in turn, can affect evaluations we make (Meier & Robinson, 2004).

Embodied cognition – The relationship between affect and spatial orientation: Studies have shown a clear relationship between spatial orientation and affect (cf. Foster et al., 2008; Meier & Robinson, 2004; Poole & Langston, 2008; Wapner, Werner & Krus, 1957). In an early study conducted by Wapner et al. (1957) it was found that success, and therefore also positive affect, was associated with an upward position in space while failure, and therefore also negative affect, was associated with a downward position in space. Later, in a series of studies, Meier and Robinson (2004) demonstrated also the association between affect and vertical spatial positioning. They found that the participants evaluated positive words faster when they were presented closer to the top of the desktop (as opposed to the bottom), whereas negative words were evaluated faster when they were closer to the bottom of the desktop (as opposed to the top). In a second study, Meier and Robinson (2005) discovered that the shift in the respondents’ visual space was consistent with their orientational metaphors (i.e. the upper region of the desktop was associated with what is positive and the bottom region was associated with what is negative).

In addition to the studies demonstrating the vertical relationship between affect and spatial orientation, recent studies have provided researchers with a reason to believe that there is a relationship between the horizontal spatial orientation of individuals and affect (Poole & Langston, 2008). Foster et al. (2008) conducted a study focusing on the horizontal relationship between affect and spatial orientation. Participants were required to place six different pegs, three representing positive affect (happy, joyful and surprised) and three representing negative affect (afraid, sad and disgusted), on a pegboard in front of them. According to the results of the study, the participants placed the pegs associated with positive emotions on the left side of the board and the pegs associated with negative emotions on the right side of the board. Furthermore, Foster et al. (2008) found that positive emotions are mediated by the left hemisphere of the brain while negative emotions are mediated by the right hemisphere. Therefore, according to the study positive emotions were found to be associated with a leftward directional bias and negative emotions with a rightward directional bias (Foster et al., 2008). On the other hand, Poole and Langston (2008) replicated the studies conducted by Meier and Robinson (2004) focusing, instead, on the association between horizontal positioning and affect but could not find evidence of this particular association.

Given the lack of consistent findings in the literature, an opportunity exists for further investigation into the relationship between spatial location and affect, and the extent to which it may have ‘subtle but pervasive effects on evaluation’ (cf. Meier & Robinson, 2004, p. 243). We proposed that these findings may have implications for the reliability of questionnaires presented on a two-dimensional space developed for the measurement of affect. This may include scales developed for the measurement of attitudes. The implication is that the way in which a questionnaire is approached may have a physical basis, which in turn may have implications for the reliability of the questionnaire.

The importance of measurement in organisational settings

Questionnaires are the most commonly used method of data collection in the field of organisational behaviour (Stone, 1978). In the past several decades, a plethora of scales has been developed to assess various attitudes, perceptions or opinions of organisational members (Hinkin, 1995). This information is often used to inform decisions about selection, advancement, training and culture, among other organisational functions. However, the adequate measurement of abstract constructs is perhaps the greatest challenge in conducting research in organisations (Hinkin, 1998). Measurement problems (i.e. problems with the reliability and validity of measures) continue to lead to difficulties in interpreting the results of abstract measures (Hinkin, 1995, 1998). When the results of such scales cannot be accurately interpreted, this impedes the ability of practitioners to make informed decisions. Thus, it is crucial that the measures on these survey instruments adequately represent the constructs under examination (Hinkin, 1998). In particular, Hinkin (1995) identified low internal consistency reliability as one of several factors which continues to threaten our understanding of organisational phenomena.

Reliability

Reliability refers to the consistency of a measure (Gregory, 2007) and is a necessary condition for validity (Kerlinger, 1986). In the social sciences, near perfect consistency in measurement is difficult to achieve but is crucial for ensuring the validity and measurement precision of a questionnaire (Muchinsky, Kriek & Schreuder, 2005). Reliability is best described as a continuum that ranges from minimal repeatability or inconsistency to near ideal consistency of results. Psychological measurements are typically positioned between the extremes of the continuum (Gregory, 2007). In this regard, careful consideration needs to be exercised in order to secure optimal reliability in all forms of testing. There are many factors which affect reliability; these factors may be categorised into five groups: 1) factors inherent to the individual, 2) the social/cultural setting, 3) the environment, 4) the questionnaire itself and 5) the nature of the sample. Some of these factors are interrelated (Ercan, Yazici, Ocakoglu, Sigirli & Kan, 2007).

In the literature, the notion of embodiment has not been acknowledged as a probable factor affecting reliability. However, the experimental findings which show that a person is spatially oriented to position what is positive at the top and what is negative at the bottom, and also to position what is positive to the left and what is negative to the right, imply that this may be the case. Based on the background and the proposed problem statement, the following research question was formulated: Does congruence between a respondent’s spatial orientation and the spatial positioning of a questionnaire lead to higher reliability of a questionnaire measuring affect? We hypothesised that when there is congruence between a respondent’s spatial orientation (related to affect) and the spatial positioning (layout) of a questionnaire, the reliability will be higher than in the case of incongruence. The principal objective of the two experiments reported here was, thus, to ascertain the extent to which congruence between a respondent’s spatial orientation (related to affect) and the layout (spatial positioning of a questionnaire) may impact on the reliability of a questionnaire measuring affect.

Research design

Research approach

The research comprised a controlled experimental research design, where the reliability of the affect scale was the dependent variable and the spatial positioning of the items in the scale was the independent variable. The spatial positioning of the items was manipulated in different layouts of the affect questionnaire, and the differences in the reliability in respect to the different layouts were investigated.

Research method
Research participants

A total of 1825 participants from a metropolitan university in Johannesburg completed the self-report questionnaire online in computer laboratories. In return for their participation participants were offered the opportunity to be entered into a competition whereby they could win a small prize. The mean age of the participants was 21.2 years (SD = 4.08). There were 1045 women (57.3%).

Measuring instruments

Six different layouts of a questionnaire measuring affect were designed to obtain information with regard to the relationship between the spatial positioning of items (as related to the spatial orientation of the participants) and the reliability of a questionnaire measuring affect. The first section of each questionnaire pertained to biographical information. The second section consisted of 30 bipolar items that tapped into self-affect (Experiment 1) and consumer affect or attitude (Experiment 2). The questionnaire scale was created using carefully constructed sets of bipolar pairs of adjectives, anchored at the ends of the continuum. The same items and order of presentation of the items were used throughout all six versions of the questionnaire. In Experiment 1, the participants were required to indicate, on a five-point scale the extent to which the adjective on either side of the scale represented how they felt about themselves (see Table 1). In Experiment 2, the participants were required to indicate, on a five-point scale the extent to which the adjective on either side of the scale represented how they felt towards a popular consumer sportswear brand (see Table 2).

TABLE 1: Bipolar adjectives utilised in the affect scale (internal).
TABLE 2: Bipolar adjectives utilised in the affect scale (external).

The six different layouts of the affect scale consisted of three horizontal layouts (representing horizontal spatial orientation; see Table 3) and three vertical layouts (representing vertical spatial orientation; see Table 4). Each item in the three horizontal layouts of the scale consisted of two bipolar adjectives, one positioned on the left and the other on the right of the continuum. In the positive to negative horizontal layout (horizontal layout 1), the positive adjective was positioned on the left of the continuum and the negative adjective was placed on the right side of the continuum. For the negative to positive horizontal layout (horizontal layout 2), the positioning was reversed with the negative adjective on the left and the positive adjective on the right of the continuum. A combination of the positioning of the items in the aforementioned layouts was used in the mixed horizontal layout (horizontal layout 3).

TABLE 3: The positioning of the items in respect of the horizontal layouts of the affect scale.
TABLE 4: The positioning of the items in respect of the vertical layouts of the affect scale.

The bipolar adjectives used in the horizontal layouts of the scale remained the same for the vertical layouts of the questionnaire. However, one adjective was positioned at the top of the continuum and the other adjective at the bottom of the continuum. Each item in the vertical positive to negative layout (vertical layout 1) consisted of a positive adjective at the top of the continuum and a negative adjective at the bottom of the continuum. For the negative to positive vertical layout (vertical layout 2), the negative adjectives were positioned at the top of the continuum and the positive adjectives were positioned at the bottom of the continuum. A mixture of the item positioning in these two layouts was used in the mixed vertical layout (vertical layout 3).

Research procedure

The questionnaire was administered online in computer laboratories. Students on university campus have access to more than 500 computers in computer-based laboratories. As students entered these laboratories they were approached to complete the questionnaire for research purposes. Participants were randomly assigned to one of the six different layouts of the questionnaire. Each condition had roughly 300 participants. In terms of group equivalence, the sample was distributed relatively equally across the six layouts of the questionnaire, for education level, gender mix and age distribution.

Results

Experiment 1
Scale structure

A preliminary factor analysis revealed that the scale measured a strong general affect factor underlying all 30 items for four of the six layouts. However, for the vertical layout 3 and horizontal layout 3, the results suggested two factors each. This was attributed to the fact that the first factor of both layouts consisted of predominantly negative adjectives, positioned on the left or on the top of their respective scales. The second factor for both mixed layouts, conversely, consisted of predominantly positive adjectives that were placed on the left or on the top of their respective scales. This suggested that the items were grouped into factors based on the spatial positioning of the emotionally charged adjectives. These results are consistent with the findings of Spector, Van Katwyk, Brannick and Chen (1997). They showed that affect scales that contain items reflecting both negative and positive emotions typically form factors based on the direction of item wording. However, while the loadings on the second factor with regard to horizontal layout 3 and vertical layout 3 indicated the presence of a second factor, there was no strong evidence indicating that the second factors were substantive.

Descriptive statistics and reliabilities of the layouts

The descriptive statistics of the six layouts were as follows: the mean scores (with SD in parentheses) were 53.07 (15.80) for horizontal layout 1, 53.19 (15.73) for horizontal layout 2, 56.43 (13.52) for horizontal layout 3, 51.93 (13.88) for vertical layout 1, 51.50 (12.09) for vertical layout 2, and 56.58 (12.62) for vertical layout 3. The reliabilities of the layouts were as follows: horizontal layout 1 (a = 0.927), horizontal layout 2 (a = 0.924), horizontal layout 3 (a = 0.884), vertical layout 1 (a = 0.905), vertical layout 2 (a = 0.856), and vertical layout 3 (a = 0.864).

Comparison of reliability

A pairwise comparison of the Cronbach’s alpha reliability coefficients were done by means of Feldt’s test of differences in independent reliability coefficients (Charter & Feldt, 1996; see Table 5). Only two comparisons were statistically non-significant (p > 0.05), namely, that of the horizontal layout 1 (a = 0.927) and horizontal layout 2 (a = 0.924), and that of horizontal layout 3 (a = 0.884) and vertical layout 3 (a = 0.864). This suggests that differences in the reliability coefficients between these layouts may not be due to the spatial positioning of emotionally charged adjectives.

TABLE 5: The significance of the difference in the reliability between the six layouts of the affect scale.
Discussion

Experiment 1 confirmed that when there is congruency between the spatial orientation of affect and the spatial positioning or layout of a questionnaire, its reliability was higher. What is significant about the statistical analysis is that the reliability of the questionnaire seemed to be influenced by the layout rather than the content. The results indicated that the highest reliability for the vertical scales was that of vertical layout 1 (positive to negative). The difference in reliability between the three vertical layouts was also found to be statistically significant, indicating that congruence between vertical spatial orientation (related to affect, i.e. that up is associated with the positive and down with the negative.) and the spatial positioning (i.e. the vertical layout) of the questionnaire measuring affect had an influence on the reliability of the questionnaire. These findings are consistent with those of Fisher (1964), Meier and Robinson (2004), Meier, Robinson and Caven (2008), Poole and Langston (2008), Setic and Domijan (2007), Stepper and Strack (1993) and Wapner et al. (1957).

The results further indicated that there was statistically no difference between the reliability coefficients of the horizontal layouts, excluding the mixed version, which corroborates the findings of Poole and Langston (2008), who could not determine congruency between affect and the horizontal spatial position. In other words, in this experiment no significant difference was found between the horizontal positive to negative layout and the horizontal negative to positive layout, indicating that congruence between an individual’s horizontal spatial orientation (related to affect, i.e. that left is associated with the positive and right with negative) and the spatial positioning (i.e. horizontal layout) of the questionnaire had little influence on the reliability of the questionnaire. Therefore, although horizontal layout 1 had a higher reliability, the difference between this reliability and that of horizontal layout 2 was not significant and therefore not meaningful.

Finally, given that theory on embodiment and evidence from several empirical studies suggests that people are spatially oriented to vertically position what is positive at the top and negative at the bottom, and less so to horizontally position what is positive on the left and what is negative on the right, one would have expected the vertical positive to negative layout to have the highest reliability. However, this was not the case. A possible explanation may be due to the familiarity of a horizontal scale among the participants, who may have been conditioned to answer a scale in a horizontal layout rather than the unusual vertical layout.

Experiment 2

Experiment 2 was primarily designed to replicate central aspects of Experiment 1. However, Experiment 2 focuses on the participants’ disposition towards a popular consumer brand (external to the self). A popular sportswear brand was used as it is one of the most recognisable brands among consumers in South Africa (Doke, 2008; Naidoo, 2008).

Scale structure

A preliminary factor analysis revealed that the scale measured a strong general attitude factor underlying all 30 items for four of the six layouts. However, for the vertical layout 3, and horizontal layout 3, the results suggested two factors each, as in Experiment 1.

Descriptive statistics and reliabilities of the layouts

The descriptive statistics of the six layouts were as follows: the mean scores (with SD in parentheses) were 54.87 (20.22) for horizontal layout 1, 53.53 (20.28) for horizontal layout 2, 54.28 (17.51) for horizontal layout 3, 51.38 (17.25) for vertical layout 1, 49.88 (17.13) for vertical layout 2 and 55.41 (17.01) for vertical layout 3. The reliabilities of the layouts were as follows: horizontal layout 1 (a = 0.962), horizontal layout 2 (a = 0.959), horizontal layout 3 (a = 0.920), vertical layout 1 (a = 0.948), vertical layout 2 (a = 0.946), and vertical layout 3 (a = 0.912).

Comparison of reliability

A pairwise comparison of the Cronbach’s alpha reliability coefficients were done by means of Feldt’s test of differences in independent reliability coefficients (Charter & Feldt, 1996; see Table 6). Five comparisons were statistically non-significant (p > 0.05), namely, that of horizontal layout 1 (a = 0.962) and horizontal layout 2 (a = 0.959), and that of horizontal layout 3 (a = 0.920) and vertical layout 3 (a = 0.912), which is identical to what was found in Experiment 1. However, Experiment 2 also found that vertical layout 1 (a = 0.948) and vertical layout 2 (a = 0.946); and vertical layout 3 (a = 0.912), as well as that of vertical layout 2 (a = 0.946) and vertical layout 3 (a = 0.912), were insignificant. This suggests that differences in the reliability coefficients between these layouts may not be due to the spatial positioning of emotionally charged adjectives.

TABLE 6: The significance of the difference in the reliability between the six layouts of the affect scale.
Discussion

Experiment 2, like Experiment 1, confirmed that when there is congruency between the spatial orientation of affect and the spatial positioning or layout of a questionnaire, its reliability was higher. The results indicated that the highest reliability for the vertical scales was that of vertical layout 1 (positive to negative layout). However, in contrast to Experiment 1, Feldt’s test indicated that the differences in the reliability of three comparisons of the vertical layouts were non-significant and therefore not statistically meaningful. In other words, in this experiment there were no significant differences between the three vertical layouts, indicating that congruence between an individual’s vertical spatial orientation (related to affect, i.e. that up is associated with the positive and down with the negative) and the spatial positioning (i.e. vertical layout) of the questionnaire had little influence on the reliability of the questionnaire. A possible explanation as to why the results for Experiment 2 differed from those in Experiment 1, may be that participants in Experiment 1 focused internally (on the self), whereas participants in Experiment 2 focused on a consumer brand external to the self. We postulate that focusing internally (on the self) is perceived as more important and that this results in participants giving more attention to self-evaluations. The conceptual metaphor ‘more is up’ as spoken about by Lakoff and Johnson (1980), provides an example of another spatial orientation associated with verticality that may account for the fact that the differences in reliability for the vertical layouts, in Experiment 2, were found to be insignificant. In respect of the horizontal layouts, the results were similar to those found in Experiment 1.

Summary and concluding discussion

Outline of results

The main aim of these studies was to determine the impact of the spatial positioning of the items, on the reliability of a questionnaire measuring affect. Both Experiment 1 and Experiment 2 found evidence to support the hypothesis, namely, that greater congruence between a respondent’s spatial orientation (related to affect) and the spatial positioning (layout) of a questionnaire leads to higher reliability on the questionnaire. Furthermore, the results of Experiment 2 found that spatial-related conceptual processing was more likely to be triggered when participants were asked to evaluate their feelings towards themselves (internal/near), as opposed evaluating their feeling towards an object (external/far).

Practical implications

Overall, this research demonstrates that congruence between the spatial positioning of items and the spatial orientation of the respondent can impact the reliability of questionnaires measuring affect. These findings may serve to create awareness of the influence of the spatial positioning of items as a confounding variable in the questionnaire design. Specifically, the results indicated that items should consistently be positioned from either positive to negative adjectives or negative to positive adjectives throughout the scale. A mix of items ranging from positive to negative adjectives and negative to positive adjectives may be counter-intuitive and confuse participants, consequently reducing the reliability of the questionnaire. This is in line with Barnette’s (2000) findings, which showed that the use of a mix of items positioning is probably much too confusing for many respondents and is therefore not a recommended procedure.

Limitations and recommendations

A replication of this study is suggested to confirm the results of these experiments. Furthermore, research suggests that a replication study should consider the use of negatively charged adjectives. Barnette (2000), Cordery and Sevastos (1993); Schriesheim and Eisenbach (1995); and Schriesheim and Hill (1981) indicate the use of negatively worded items can often negatively affect the reliability of measures and can cause the appearance of artificial factors. Given the negative implications of using negatively worded items, it may be beneficial to consider a replication of this study which excludes the use of the negatively charged adjectives – and then compares the results. While this research has provided some initial evidence for the hypothesised account for the influence of spatial positioning on questionnaires measuring affect, consideration can be extended to other types of questionnaires (e.g. personality questionnaires) and/or other constructs. In particular, the findings suggest that future research should further explore the relationship between a respondent’s spatial orientation and the spatial positioning of the construct (internal versus external).

Conclusion

In conclusion, this research complements previous studies (Meier & Robinson, 2004; Meier et al., 2008; Poole and Langston, 2008; Setic & Domijan, 2007) by confirming the metaphorical representation of affect and enhances our understanding of embodiment-related conceptual processing and its subsequent influence on self-evaluations versus external evaluations on an non-conscious level, specifically in relation to measuring affect. Further research is necessary to confirm these effects, expanding to include the relationship between congruence between a respondent’s spatial orientation (related to affect) and the spatial positioning of the construct (internal/near versus external/far).

Acknowledgements

Competing interests

The authors declare that they have no financial or personal relationships which may have inappropriately influenced them in writing this article.

Authors’ contributions

This research formed part of an embodied cognition project initiated by F.C. He conceptualised the study which was executed by S.K. and A.B. under his supervision. S.K. was responsible for the data gathering and statistical analyses for Experiment 1. A.B. was responsible for the statistical analyses for Experiment 2. J.S. acted as a statistical consultant. L.L. integrated the two studies, wrote the literature and finalised the statistical analyses previously done by S.K. and A.B.

References

Barnette, J.J. (2000). Effects of stem and Likert response option reversals on survey internal consistency: If you feel the need, there is a better alternative to using those negatively worded stems. Educational and Psychological Measurement, 60(3), 361–370. http://dx.doi.org/10.1177/00131640021970592

Charter, R.A., & Feldt, L.S. (1996). Testing the equality of two alpha coefficients. Perceptual and Motor Skills, 82, 763–768. http://dx/doi.org/10.2466/pms.1996.82.3.763

Cordery, J.L., & Sevastos, P.P. (1993). Responses to the original and revised job diagnostic survey: Is education a factor in responses to negatively worded items? Journal of Applied Psychology, 78(1), 141. http://0-dx.doi.org.ujlink.uj.ac.za/10.1037/0021-9010.78.1.141

Crous, F. (2007). Branding the positive. Inaugural address. University of Johannesburg, South Africa. Retrieved August 4, 2015, from http://ujdigispace.uj.ac.za:8080/dspace/bitstream/10210/1633/1/Crous%20F.pdf

Damasio, A. R. (1994). Descartes’ error: Emotion, reason, and the human brain. New York:Grosset/Putnam.

Doke, L. (2008, 23 August). Home-grown products excel as favourite brands: Some welcome local surprises emerged among the winners in this category. Sunday Times. Retrieved March 10, 2010, from http://www.timeslive.co.za/multimedia/archive/00270/TopBrands2009_270389a.pdf

Ercan, I., Yazici, B., Ocakoglu, G., Sigirli, D., & Kan, I. (2007). Review of reliability and factors affecting the reliability. Unpublished manuscript.

Fisher, S. (1964). Depressive affect and perception of up-down. Journal of Psychiatric Research, 2, 25–30. http://dx.doi.org/10.1016/0022-3956(64)90026-3

Foster, P.S., Drago, V., Webster, D.G., Harrison, D.W., Crucian, G.P., & Heilman, K.M. (2008). Emotional influence on spatial attention. Journal of Neuropsychiatry, 22(1), 127–135. http://dx.doi.org/10.1037/0894-4105.22.1.127

Gibbs, R.W. (2006). Embodiment and cognitive science. New York: Cambridge University Press.

Gregory, R.J. (2007). Psychological Testing: History, principles, and applications. (5th edn.). Boston, MA: Pearson/Allyn and Bacon.

Hinkin, T.R. (1995). A review of scale development practices in the study of organizations. Journal of Management, 21(5), 967–988. http://dx.doi.org/10.1177/014920639502100509

Hinkin, T.R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 2(1), 104–121. http://dx.doi.org/10.1177/109442819800100106

Kerlinger, F. (1986). Foundations of behavioral research. (3rd edn.). New York: Holt, Rinehart & Winston.

Kovecses,Z. (1991). Happiness A definitional effort. Metaphor and Symbolic Activity, 6, 29–46. http://dx.doi.org/10.1207/s15327868ms0601_2

Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago, IL: Chicago University Press.

Lakoff, G., & Johnson, M. (1999). Philosophy in the flesh: The embodied mind and its challenge to western thought. New York: Basic Books.

Lakoff, G., & Johnson, M. (2003). Metaphors we live by. London: The University of Chicago Press.

Meier, B.P., & Robinson, M.D. (2004). Why the sunny side is up: Associations between affect and vertical position. Psychological Science, 15, 243– 247. http://dx.doi.org/10.1111/j.0956-7976.2004.00659.x

Meier, B.P., & Robinson, M.D. (2005). The metaphorical representation of affect. Metaphor and Symbol, 20, 239–257. http://dx.doi.org/10.1207/s15327868ms2004_1

Meier, B.P., Robinson, M.D., & Caven, A.J. (2008). Why a big Mac is a good Mac: Associations between affect and size. Basic and Applied Social Psychology, 30, 46–55. http://dx.doi.org/10.1080/01973530701866516

Mirolli, M., & Parisi, D. (2009). Language as a cognitive tool. Minds & Machines, 19, 517–528. http://dx.doi.org/10.1007/s11023-009-9174-2

Muchinsky, P.M., Kriek, H.J., & Schreuder, D. (2005). Predictors: Psychological assessments. In M. Coetzee & D. Schreuder (Eds.), Personnel psychology (pp. 77–123). Cape Town: Oxford University Press Southern Africa (Pty).

Naidoo, S. (2008, February 03). SA kids would gladly sell their souls for the right label: Teens believe top brands will help them fit in. Sunday Times. Retrieved March 10, 2010, from http://www.timeslive.co.za/sundaytimes/article89028.ece

Niedenthal, P.M., Barsalou, L.W., Winkielman, P., Krauth-Gruber, S., & Ric, F. (2005). Embodiment in attitudes, social perception, and emotion. Personality and Social Psychology Review, 9, 184–211. http://dx.doi.org/10.1207/s15327957pspr0903_1

Piaget, J., & Inhelder, B. (1969). The psychology of the child. New York: Basic Books.

Pinker, S. (2007). The stuff of thought. New York: Penguin.

Poole, B.D., & Langston, W.E. (2008). Affect and horizontal position. Paper presented at the 49th annual meeting of the Psychonomic Society, Chicago, IL. Retrieved November 28, 2009, from http://frank.mtsu.edu/~wlangsto/PsychonomicsF08.ppt

Schriesheim, C.A., & Eisenbach, R.J. (1995). An exploratory and confirmatory factor-analytic investigation of item wording effects on the obtained factor structures of survey questionnaire measures. Journal of Management, 21(6), 1177–1193. http://dx.doi.org/10.1177/014920639502100609

Schriesheim, C.A., & Hill, K.D. (1981). Controlling acquiescence response bias by item reversals: The effect on questionnaire validity. Educational and Psychological Measurement, 41(4), 1101–1114. http://dx.doi.org/10.1177/001316448104100420

Setic, M., & Domijan, D. (2007). The influence of vertical spatial orientation on property verification. Language and Cognitive Processes, 22, 297–312. http://dx.doi.org/10.1080/01690960600732430

Spector, P.E., Van Katwyk, P.T., Brannick, M.T., & Chen, P.Y. (1997). When two factors don’t reflect two constructs: How item characteristics can produce artifactual factors. Journal of Management, 23(5), 659–677. http://dx.doi.org/10.1177/014920639702300503

Stepper, S., & Strack, F. (1993). Proprioceptive determinants of emotional and nonemotiona feelings. Journal of Personality and Social Psychology, 64, 211–220. http://dx.doi.org/10.1037/0022-3514.64.2.211

Stone, E. (1978). Research methods in organizational behavior. Glenview, IL: Scott, Foresman.

Van den Bergh, B., Schmitt, J., & Warlop, L. (2011). Embodied myopia. Journal of Marketing Research, 48, 1033–1044. http://dx.doi.org/10.1509/jmr.09.0503

Wapner, S., Werner, H., & Krus, D.M. (1957). The effect of success and failure on space localization. Journal of Personality, 25, 752–756. http://dx.doi.org/10.1111/j.1467-6494.1957.tb01563.x

Wilson, A.D., & Golonka, S. (2013). Embodied cognition is not what you think it is. Frontiers in Psychology, 4, 58. http://dx.doi.org/10.3389/fpsyg.2013.00058

Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 9(4), 625–636. http://dx.doi.org/10.3758/BF03196322



Crossref Citations

No related citations found.