Abstract
Orientation: Entrepreneurial competence (EC) is important for entrepreneurial success. Situational judgement test (SJT) is a promising tool for assessing EC, but its construct validation often faces methodological challenges.
Research purpose: This study aims to re-examine the factor structure of the short version of the Indonesian Entrepreneurial Competence Inventory (IECI), an SJT for measuring EC in the Indonesian context.
Motivation for the study: Previous studies produced conflicting findings: exploratory factor analysis supported a three-dimensional model, while confirmatory factor analysis (CFA) indicated nine-dimensional model. This disparity most likely caused by the limitations of CFA in modelling complex SJT data, creating validity ambiguity.
Research approach/design and method: Data were collected from 498 entrepreneurs in the micro, small, and medium enterprises sector in Indonesia. The IECI factor structure was analysed using exploratory structural equation modelling (ESEM) and bifactor ESEM.
Main findings: The bifactor ESEM model provided the best fit, revealing one strong general factor (g-factor) and three weaker specific factors. These findings reconcile previous studies by demonstrating that EC has a hierarchical structure.
Practical/managerial implications: The total IECI score has been proven to be a highly reliable measure of general EC. However, the interpretation of sub-dimension scores must be done with caution because of their smaller variance contribution.
Contribution/value-add: This study resolved on the IECI’s dimensionality by applying more appropriate psychometric method. The findings advance the theoretical understanding of EC as a bifactor construct and strengthen the IECI’s validity.
Keywords: exploratory structural equation modelling; bifactor ESEM; factor structure; entrepreneurial competence; situational judgement test; Indonesia.
Introduction
Entrepreneurship is crucial in promoting economic growth, driving innovation, and creating job opportunities (Audretsch et al., 2001). The success of entrepreneurial ventures depends heavily on the capabilities of entrepreneurs themselves (Johnson, 1990; Shane & Venkataraman, 2000). Central to these capabilities is entrepreneurial competence (EC), a construct encompassing the knowledge, skills, and attributes essential for navigating the complexities of business world and achieving superior business performance (Bird, 2019; Mitchelmore & Rowley, 2010). Given its direct impact on business outcomes, the accurate measurement of EC is important, particularly for the development and evaluation of entrepreneurship training programmes.
A significant challenge in this area is the cultural specificity of most measurement tools. Instruments developed in Western context often lack suitability for different cultural settings, like Indonesia, where entrepreneurial behaviours and values may differ (Cronk, 2017). In response to the concern, Riyanti and Suwartono (2018) and Wulani et al. (2019) developed a specific EC instrument for the Indonesian context. While these instruments were found to be valid and reliable, they primarily assessed perceptions rather than actual competencies. To address this limitation, Suryani et al. (2021) developed a new instrument, known as the Indonesian Entrepreneurial Competence Inventory (IECI), using the situational judgement test (SJT) model.
Although the SJT model presents numerous advantages, including its ability to explain incremental variance in relevant criteria (Weekley & Ployhart, 2005; Whetzel et al., 2020), support training and evaluation (Fritzsche et al., 2006), offer greater efficiency than some traditional tools (Joiner, 2002; McDaniel et al., 2001), and elicit positive participant reactions owing to high face validity (Anderson, 2003; Hausknecht et al., 2004; Lievens et al., 2005), various validity and reliability concerns related to the model are not properly addressed (Christian et al., 2010; Weekley & Ployhart, 2005). Empirical evidence from numerous studies has indicated significant challenges in identifying factors measured by the SJT because of overlapping construct characteristics (Whetzel et al., 2020). Additionally, the model frequently lacks adequate consideration of the constructs it aims to assess (Schmitt & Chan, 2013). This limitation also pertains to the IECI SJT model, which exhibits validity and reliability concerns.
The validation process of the IECI revealed significant inconsistencies in terms of construct structure and reliability. These inconsistencies were clearly evident when comparing two key studies in its development. The initial development study by Suryani et al. (2021) used exploratory factor analysis (EFA) successfully identify a valid and reliable three-dimensional structure (Cronbach’s Alpha values ranging from 0.71 to 0.78). This resulted in a shortened version of the instrument with 22 items. However, a subsequent validation study by Riyanti et al. (2022) applied confirmatory factor analysis (CFA) to the long version of the instrument and found support for a valid nine-dimensional structure. While the nine-dimensional model showed acceptable model fit, its internal reliability was reported to be relatively low (Cronbach’s Alpha values per dimension ranged from 0.48 to 0.70). The clear difference between the EFA findings, which yielded a reliable three-factor structure, and the CFA findings, which supported a nine-factor structure with lower reliability, creates conceptual and psychometric ambiguity. This indicates that the latent structure of the IECI is unstable and highly dependent on the analytical method used.
This issue likely comes from the methods used in conventional factor analysis for instruments as complex as SJTs. Modern psychometric literature shows that CFA models, with their strict assumptions such as zero cross-loadings (each item measures only one-factor), are often too restrictive and unrealistic for modelling human response data, especially for scenario-based items (Marsh et al., 2014). Situational judgement test items are inherently multidimensional (Whetzel et al., 2020). This means that a single problem scenario can trigger various competencies at the same time in order to be solved. When you apply simple structures to complex data using CFA, you might get the wrong results. For example, you might have a model that does not fit or a solution that makes reality too simple.
To deal with these methodological limitations and resolve the IECI factor structure debate, a more flexible and advanced analytical approach is needed. Exploratory structural equation modelling (ESEM) and the bifactor ESEM model are better solutions (Morin et al., 2020; Reise, 2012). Exploratory structural equation modelling allows for cross-loadings between factors, enabling more accurate modelling of complex relationships between items and latent constructs. Meanwhile, bifactor models can test the existence of a general factor that explains the common variance across all items, as well as several specific factors that explain unique variance. This approach is very important for EC, which is believed to have one main skill and several more specific sub-skills. This study aims to take another look at the factor structure of the short-form IECI by using the ESEM and bifactor ESEM methods.
Specifically, this study will compare the fit of the three-factor ESEM model (one general factor and three specific factors) with the bifactor ESEM model to determine the most valid and theoretically accurate representation of the factor structure. This study has three contributions. Firstly, the study uses the latest standards for psychometric analysis, which are better for validating SJT instruments. Secondly, the study tries to resolve previous conflicting findings about the dimensionality of EC by testing a more comprehensive hierarchical model. Thirdly, the results of this study will provide a stronger foundation for the use of IECI as an assessment and evaluation tool in entrepreneurship development programmes in Indonesia.
Literature review
The concept of entrepreneurial competence
The study of EC has grown significantly since Boyatzis (1982) introduced the concept of competencies, emphasising the role of knowledge, skills, and abilities (KSA). In entrepreneurship, EC is broadly defined as the capacity to think, act, and adapt effectively in various business situations (Guion, 2011).
Key definitions from Bird (2019), Man et al. (2002) and Mitchelmore and Rowley (2010) converge on the idea that EC comprises fundamental characteristics-including knowledge, skills and personality trait that enable entrepreneurs to succeed. Despite variations in specific components, these three elements consistently form the core of the EC construct.
Measurement of entrepreneurial competence
The measurement of EC in various studies to date has varied considerably. Among the most frequently referenced is the model developed by Man et al. (2008). This instrument has played a central role in shaping EC research by offering a structured approach to assessing six core competency domains: opportunity, relationship, conceptual, organisational, strategic, and commitment. This framework was later expanded to include two supporting domains (Man et al., 2008). However, despite this broader structure, most studies tend to focus only on the six domains, often neglecting the supporting domains, which limit a full understanding of the construct. To enhance its comprehensiveness, Man et al. (2008) added additional rational competencies—such as ethical competence, familism, social responsibility, and technical competence—into the model. The foundational structure of this framework has been widely accepted by other researchers to develop instruments tailored to specific research contexts and cultural settings (Draksler & Sirec, 2021; Lans et al., 2011; Rahman et al., 2015; Wulani et al., 2019).
Beyond the Man et al. model, another significant theoretical foundation for EC assessment is the EntreComp framework developed by Bacigalupo et al. (2016). This framework is gaining popularity, but most assessment tools derived from it do not fully implement its original structure. Instead, researchers often select and adapt its dimensions and sub-dimensions to suit the specific context of their studies, making it difficult to assess their conceptual grounding. The original EntreComp document itself has been excluded from some academic reviews because it was not published in a peer-reviewed journal and lacked a formal psychometric analysis of its dimensions and reliability.
Despite this variation, most EC instruments demonstrate strong internal consistency, with reported reliability coefficients (e.g. Cronbach’s alpha or McDonald’s omega) meeting or exceeding accepted standards. In terms of validity, the evidence presented in various studies ranges from moderate to strong. However, theoretical and structural inconsistencies among these instruments suggest that, to date, there is no universally accepted best measure for assessing EC.
Challenges in measuring entrepreneurial competence using situational judgement test
The selection of the SJT to measure EC is based on its superiority in assessing an individual’s ability to apply knowledge and skills in scenarios relevant to real-life contexts. Unlike the Likert scale, which measures self-perception or beliefs (self-efficacy), SJT is designed to measure applied judgement. Participants are not asked ‘can you do it’; they are asked ‘what would you do’ in a given situation. This provides a more accurate picture of actual competence. However, this advantage also brings significant psychometric challenges. Scenario-based SJT items are inherently complex and multidimensional.
A single business problem scenario – for example, facing a new competitor – may simultaneously require competencies in strategic analysis, achievement orientation (ACH), and persuasion to be resolved effectively. Conventional analysis methods, such as CFA, which assume that each item measures only one-factor (no cross-loading), are no longer adequate. As Sorrel et al. (2016) have clearly demonstrated, approaches rooted in Classical Test Theory (CTT) often fail to capture the multidimensional nature of SJT scores. This explains why previous validation studies using CFA on IECI faced issues such as poor model fit and low convergent validity.
The dimensional structure of entrepreneurial competence: A transition from multidimensional to hierarchical
The main debate in the EC literature concerns its dimensional structure. The predominant perspective regarding EC is that it is multidimensional in nature. The framework developed by Man et al. (2002), for instance, categorises EC into six areas: opportunity, relationship, conceptual, organisational, strategic, and commitment. The framework developed by Spencer and Spencer (2008), which served as the basis for the development of the IECI instrument, identified 13 distinct competency dimensions. Early EFA studies of the IECI also supported this multidimensional view by identifying three distinct factors (Suryani et al., 2021).
However, the findings of Riyanti et al. (2022) suggest the possibility of a more complex structure, given their indication that a single-factor model is more suitable in CFA analysis. In response to this, editors and scholars in the field of modern literature have proposed the consideration of hierarchical models, such as second-order or bifactor models, as superior alternatives. This perspective is corroborated by numerous studies that demonstrate EC is most effectively conceptualised as a hierarchical structure. For instance, Man et al. (2008) posit the existence of intercompetency relationships that can be modelled hierarchically within their framework. Other studies, including those by Cárdenas-Gutiérrez et al. (2021), which sought to validate an entrepreneurship competency scale for secondary school students, have also found support for a more complex structure than that which can be explained by independent factors alone.
The bifactor model is of particular theoretical interest because of its assumption of a unifying EC factor (g-factor) that influences all items, as well as several specific factors that explain the unique variance of certain groups of items (Reise, 2012). This theoretical framework has the capacity to reconcile findings that initially appear contradictory. Specifically, the g-factor elucidates the rationale behind the emergence of a unidimensional model as a suitable solution, while distinct factors illuminate the factors that give rise to the identification of a multidimensional model. Consequently, the bifactor model does not compel researchers to opt for either unidimensional or multidimensional structures. Rather, it integrates both into a more comprehensive and realistic framework.
Research framework and hypotheses
Based on the above literature synthesis, this study adopts a conceptual framework that EC is a hierarchical construct. The disparity between the EFA and CFA findings in previous IECI studies is not seen as a contradiction, but rather as an artefact of the methodological limitations of CFA in modelling complex SJT data.
Therefore, this study hypothesises that the bifactor ESEM model will provide the most theoretically valid and accurate representation of the factor structure for the short-form IECI. Specifically, this model hypothesises the existence of:
- one general factor (g-EC) representing general EC
- three specific factors representing sub-dimensions: Opportunity Recognition, Achievement Orientation, and Strategic Influence, after accounting for variance from the common factor.
Testing these hypotheses will provide a deeper and more accurate understanding of what the IECI actually measures and how EC is structured in the context of micro, small, and medium enterprises (MSMEs) in Indonesia.
Research design
Instrument
The short version of the IECI covered three dimensions, comprising 22 items. These dimensions included (1) strategic influence (INF), with a total of 7 items, (2) ACH, with a total of 7 items, and (3) opportunity recognition (OP), with a total of 8 items. The questions were presented as cases, which describe situations experienced by entrepreneurs, followed by answer choices in the form of response-based constructs, particularly behavioural tendencies (Whetzel et al., 2020). Each option represented a different level of competence ranging from 1 (least effective) to 4 (most effective). Participants were invited to select the response option that they believed to be the most probable when confronted with a comparable case or situation while managing a business. Examples of items, scores and rationales can be found in Appendix 1: Table 1-A1.
The following is an example of the question the participants were asked to select the response option: James runs a photo studio and does wedding photography. James did not have much money when he started his business. What should James do?
- Get money from the bank by borrowing it.
- Add his portfolio.
- Make everything by himself and focus on getting new people.
- Work with hotels and wedding planners to find customers.
Participants
The survey was conducted among 512 entrepreneurs residing in major cities across the Indonesian island of Sumatra. The criteria for participation were as follows: (1) owners of MSMEs and (2) those that had been actively running their businesses for at least 3 months. The participants were selected using a non-probability method, particularly convenience sampling. The selected participants consisted of 72.65% females and 23.34% males. They were within the age range of 18–72 years (M = 36.59, standard deviation [s.d.] = 15.45), with an average length of business of 8.75 years (s.d. = 8.67). The majority of the participants had a high school diploma (51.17%), followed by university graduates (36.91%). Before participation, participants were provided informed consent after receiving details about the study’s objectives.
Test administration and scoring
The data collection was conducted between July 2024 and August 2024 using both offline and online methods using Google Forms. Combining online and offline data collection methods is quite effective for expanding the scope and depth of research (Roberts & Voorpostel, 2023). All participants were first directed to review the case questions associated with each number before being inquired about their potential actions when confronted with a comparable scenario in the present business operations. Participants were also prompted to choose one from the four response options (A, B, C, or D). It was essential to acknowledge that no responses were correct or incorrect. Furthermore, participants were directed to select the option that most accurately reflected their individual circumstances. The mean duration attached to the completion of the questionnaire was 20 min.
The response options are presented as behavioural tendencies using a rating scale. Respondents are requested to evaluate the effectiveness in addressing the scenario using a 1–4 Likert scale. A high score indicates the most effective answer choice, while a lower score suggests a less effective answer choice. The subject matter experts, including entrepreneurship scholars and accomplished entrepreneurs, already predetermined the effectiveness level of each option (Suryani et al., 2021).
Data analysis
Prior to the main analysis, initial data screening was conducted to assess the fulfilment of statistical assumptions. Missing data were handled using the listwise deletion method (Dong & Peng, 2013). Normality was evaluated through skewness values, while multivariate outliers were identified using Mahalanobis Distance. The data screening process was carried out using Jamovi version 2.6.44 and SPSS version 23.
Given the multidimensional nature of SJT items and issues identified in previous CFA analyses, this study adopted the ESEM approach. Exploratory structural equation modelling offers greater flexibility compared to traditional CFA by allowing cross-loadings, enabling a more realistic representation of complex factor structures (Marsh et al., 2014). Exploratory structural equation modelling and bifactor ESEM modelling in this study utilised the Robust Maximum Likelihood (RML) estimator, which is considered superior to the conventional Maximum Likelihood (ML) estimator. While ML is commonly used in structural equation modelling, RML provides specific advantages when dealing with data that violate the assumption of multivariate normality. It automatically applies the Satorra-Bentler correction to standard errors and Chi-square test statistics, thereby yielding more accurate parameter estimates, particularly in the presence of skewed distributions, excess kurtosis, or outliers. A target rotation was applied in the ESEM analysis to specify approximate zero loadings on non-hypothesised factors while retaining exploratory flexibility (de Beer & van Zyl, 2019). This rotation method allows for a blend of confirmatory and exploratory approaches, making it particularly suitable for complex factor structures. All data analyses were conducted using Mplus version 7 (van Zyl & Ten Klooster, 2021).
Ethical considerations
Ethical clearance for this study was obtained from the Health Research Ethics Committee at Faculty of Nursing Universitas Airlangga with approval (reference no.: 3106-KEPK). All procedures performed in this study were conducted in accordance with ethical standards. Ethical principles, such as informed consent and confidentiality, were upheld. Prior to participation, all potential participants were provided with a detailed written information sheet outlining the study’s objectives, procedures, potential risks and benefits, the expected duration of their involvement, and their right to withdraw at any point without consequence. We provided a detailed explanation of the study and ensured all participants fully understood the information before providing their consent. Signed consent forms were obtained and securely stored for all participants. Participation in the study was voluntary, and participants could withdraw from the study at any time.
Results
Missing values
The research data consists of three factors (OP, ACH, and INF) with 512 respondents. The OP is measured by eight items, the ACH by seven items, and the INF by seven items. A total of 24 missing data points were identified, distributed across 14 rows of respondent answers, constituting 2.7% of the total number of respondents. The utilisation of complete cases, which involves the exclusion of respondents with missing data, is permissible under certain conditions. Specifically, this approach is justifiable when the proportion of missing data is less than 5%, as outlined in the study by Dong & Peng (2013). The data employed in the ensuing analysis are derived from 498 subjects. Table 1 shows a list of missing data categorised by item.
| TABLE 1: Missing value categorised by item. |
Data normality
Normality checks on complete case data were performed by evaluating the skewness items. According to Hair et al. (2014), skewness values between −1 and +1 are considered very good, but values between −2 and +2 are generally still considered acceptable. Based on Table 2, all items have skewness values between –2 and +2, indicating acceptable normality, except for item INF2 with a skewness of –2.07. This may be because of the options offered by item INF2 being too biased towards one choice that is normatively considered the most appropriate, namely option D. This skewness violation is minor and still acceptable in practice, especially with an adequate sample size.
Outlier checking
To ensure the validity of the data distribution assumptions in the bifactor ESEM analysis, a check was conducted to detect the potential existence of outliers. Outlier detection was performed using two approaches, namely Mahalanobis Distance and the total Z-score of respondents.
Firstly, Mahalanobis Distance was calculated to detect multivariate outliers, i.e. respondents who collectively showed answer patterns that were very different from the majority. The calculation was performed on 22 scale items, and the cutoff value was determined based on the Chi-square distribution with 22 degrees of freedom at a significance level of 0.001, which was 48.27. The results showed that there were 10 respondents with Mahalanobis Distance values exceeding the cutoff. This indicates that there were a small number of individuals who deviated multivariately from the general response pattern.
Secondly, to avoid losing data that could reduce the representativeness of the sample, a univariate examination was also conducted using Z-scores for the total scores of each respondent. The Z-score criterion used was ±3.29, in accordance with the general limits for detecting extreme outliers in normal distributions (Tabachnick & Fidell, 2013). The results of the examination showed that only one respondent had a Z-score outside this range, namely −3.41, indicating a very low total score. Based on the screening results, the researcher decided to retain all 498.
Exploratory structural equation modelling and bifactor exploratory structural equation modelling
The ESEM modelling on these data was able to achieve convergence, whereas the bifactor ESEM modelling did not initially achieve convergence on the data of this study. The failure of convergence in the bifactor ESEM model is likely because of the complexity of the model structure, which requires estimation of multiple parameters (general and specific factors simultaneously), as well as potential model identification issues. After diagnostic evaluation, modifying the model by freeing the parameter θ (residual covariance) between items ACH6 and ACH4 successfully resulted in a converged bifactor ESEM model with adequate goodness-of-fit.
Conceptually, the residual covariance exemption between ACH4 and ACH6 has a strong theoretical justification. The two items do have similar cognitive and affective load characteristics, which both describe real crisis situations that are both financially urgent and concerning business continuity. This situation is fundamentally different from other ACH items that tend to represent internal, administrative, or procedural problems. While the other items present more mundane operational challenges, ACH4 and ACH6 display higher stress as they pose a direct existential threat to business continuity. This qualitative difference explains why the two items share a unique variance beyond the general and specific factor structures established in the model.
From a psychological response perspective, ACH4 and ACH6 have the potential to trigger more intense emotional reactions and require a faster decision-making process than other items. It is these special characteristics that likely cause the two items to require special handling in modelling. The modifications made not only resolve technical convergence issues but also improve the substantive validity of the model by accommodating the psychometric reality of the measured constructs. Path diagrams for ESEM and bifactor ESEM can be seen in Figure 1 and Figure 2.
 |
FIGURE 1: Exploratory structural equation modelling. |
|
 |
FIGURE 2: Bifactor exploratory structural equation modelling. |
|
The ESEM modelling results show the following goodness-of-fit:
Based on Table 3, the ESEM and bifactor ESEM models are generally equally good and fit with slight differences. Both models show good fit based on the root mean square error of approximation (RMSEA) values (0.031 and 0.033 < 0.05) and Tucker-Lewis Index (TLI) (0.918 and 0.906 > 0.90). The CFI index above 0.95 is very good, while both models have CFI = 0.940, which is still classified as good. The Akaike Information Criterion (AIC) value in bifactor ESEM is smaller than the AIC of ESEM, while the BIC value in bifactor ESEM model is larger than ESEM. The slight difference in several fit indices can be caused by the more complex model form in bifactor ESEM (more parameters or factors) than the ESEM model. The p-value in the Chi-square test of both models shows a significant value, indicating that the model does not fit the data. However, Hooper et al. (2008) suggest that the Chi-square statistic is a statistical significance test that is sensitive to sample size, meaning that the Chi-square statistic almost always rejects the model when the sample is large (Bentler & Bonett, 1980; Jöreskog & Sörbom, 1993). Hu and Bentler (1999) recommend CFI, TLI and RMSEA to test model fit rather than the Chi-square criterion.
Table 4 displays factor loadings and cross-loadings in ESEM and bifactor ESEM modelling. The reliability values of Omega ESEM and bifactor ESEM were calculated using factor loadings and residual variance based on Rodriguez et al. (2016) and Raykov & Marcoulides (2016), namely Equation 1:
| TABLE 4: Factor loading and cross-loading in exploratory structural equation modelling and bifactor exploratory structural equation modelling. |
In the ESEM model with three factors (OP, ACH, INF), most items loaded strongly onto the corresponding factors. For example, items OP3 and OP6 showed high loadings on the OP dimension, and items ACH1–ACH7 also had dominant loadings on the ACH dimension (between 0.24 and 0.65). However, there are some indications of issues, such as items OP2 and OP7 loading negatively or very weakly on the OP factor, as well as relatively high cross-loadings (e.g. OP5 also loads strongly on the ACH factor at 0.43). Overall, the internal reliability (ω) for each factor is good, particularly for ACH (ω = 0.808) and INF (ω = 0.759), while OP remains moderately reliable (ω = 0.536). The overall reliability of this model is 0.73, calculated from Equation 2:
The bifactor ESEM model has a general factor alongside three specific factors. In this model, items OP1, OP3, and OP6 still show strong loadings on the specific factor OP, while several other items (such as OP2, OP7, and OP8) show low or even negative loadings. In the ACH factor, it was found that items ACH1, ACH5, and ACH6 had negative loadings on their specific factors but showed stronger loadings on the general factor. The ACH4 had weak loadings on its specific factor and strong loadings on the general factor. ACH2, ACH3, and ACH7 had sufficient loadings on their specific factors, but the general factor loadings were still more dominant. This indicates that the items in ACH may be too general, less discriminative of ACH, and more representative of the general factor than the specific sub-dimensions. The pattern of weak item loadings on specific factors, non-significant loadings, and more dominant loadings on general factors than specific factors is commonly observed in bifactor models, where specific factors may be less distinct from certain general factors (Burns et al., 2020; Eid et al., 2017; Heinrich et al., 2023). The presence of a general factor influences all items and explains the common variance of all items, while specific factors only explain the additional variance unique to a group of items. This model remains a viable choice when it demonstrates better model fit compared to the regular ESEM model. In the INF factor, the items have adequate loadings on the specific factor, although item INF5 has a stronger loading on ACH. Further checking and refinement are needed for item INF5. The overall reliability of this model is 0.83 (very good), calculated from (Equation 3):
The ESEM model produces a cleaner and more consistent factor loadings structure for specific dimensions, supported by good internal reliability values. On the other hand, the bifactor ESEM model allows us to recognise the contribution of general factors to items, as well as revealing that some items (such as OP2 and ACH1) tend to be more general than specific.
Discussion
This study aims to re-examine the factor structure of the IECI using a more sophisticated and appropriate analytical approach, in response to inconsistent findings and methodological criticisms in previous studies. The main results clearly show that the bifactor ESEM model provides a statistically and theoretically superior fit compared to the three-factor ESEM model. These findings not only resolve the conflict between the initial EFA study (Suryani et al., 2021), which identified three dimensions, and the findings from the validation study using CFA (Riyanti et al., 2022) that concluded a one-factor model was more appropriate, but also provides a deeper and more nuanced understanding of the EC construct measured by the IECI.
The most significant finding of this study is the presence of a strong general EC factor (g-factor), which explains most of the common variance among all items. The presence of this dominant g-factor effectively explains why restrictive analytical approaches such as CFA, applied in the initial validation study (Riyanti et al., 2022), tend to produce a single-factor model that is more fit. When most items measure the same general construct, the unidimensional model forced by CFA will appear more parsimonious and valid than a multidimensional model that forces independence among factors.
On the other hand, the presence of three specific factors (albeit weaker) in the bifactor model of this study explains why EFA analysis in the instrument development study (Suryani et al., 2021) was able to identify three distinct dimensions. These specific factors capture the remaining unique variance not explained by the g-factor. Thus, the bifactor model does not reject either of the previous findings but integrates both into a coherent hierarchical framework. This resolves the ‘unidimensional vs. multidimensional’ debate and replaces it with the understanding that EC in this context has a bifactor structure: one common core with several specialisations.
Theoretically, this study provides strong empirical support for conceptualising EC as a hierarchical construct, in line with modern views in the literature (Cárdenas-Gutiérrez et al., 2021; Man et al., 2008). This confirms that EC is not merely a collection of separate abilities, but rather an integrated system in which there are core abilities that are general and underlie various specific manifestations of competence.
Furthermore, this study highlights the importance of alignment between construct complexity and the statistical methods used. The limitations of conventional CFA models in initial validation studies are not an indication that the instruments are poor, but rather that the analytical methods are not flexible enough to capture the complex reality of SJT data. These findings provide important lessons for researchers in Industrial and Organisational Psychology to exercise caution when applying restrictive measurement models to scenario-based instruments and to consider ESEM or bifactor models as more valid alternatives (Sorrel et al., 2016).
The results of this study have clear practical implications for IECI users, including human resource (HR) practitioners, consultants, and entrepreneurship training programme providers [RB8]. Given the dominance of the g-factor and the high reliability of total scores (ω = 0.83), IECI total scores are the most valid and reliable indicator of an individual’s level of general EC. This is particularly useful for selection purposes, potential assessment, or overall programme impact evaluation. Moreover, the reliability of specific factors (especially ACH) is much lower after common variance is removed. This means that sub-scores per dimension should be interpreted with great caution. These sub-scores are not reliable enough to make important decisions about specific strengths or weaknesses of individuals. Instead of being used for definitive diagnosis, these sub-scores are better used as a starting point for further discussion or exploration. Entrepreneurship training programmes should not only focus on developing specific dimensions separately but should also target the development of general core competencies, such as analytical skills, decision-making under uncertainty, and perseverance, which are likely to be at the core of the g-factor.
Limitations and directions for further research
Although this study has addressed several significant weaknesses of previous studies, some limitations need to be acknowledged to guide future research. Firstly, although the bifactor model showed model fit, the parameter analysis indicates the presence of several problematic items (e.g. cross-loading on item INF5 and weak ACH factor specificity). This indicates that further item revision and refinement are needed to improve the psychometric quality of the instrument. Secondly, the sample of this study was limited to entrepreneurs in Sumatera, Indonesia. To increase the generalisability of the findings, cross-validation studies are highly recommended on more diverse samples, both in other regions of Indonesia and in different cultural contexts globally. This will test whether this bifactor structure remains stable across different populations. Thirdly, this study focused on ESEM and bifactor ESEM. Future research could explore more sophisticated psychometric models, such as Multidimensional Item Response Theory (MIRT) or Cognitive Diagnosis Models (CDMs), to gain a deeper understanding of how each individual interacts with each answer choice on SJT items.
Conclusion
In conclusion, this study successfully resolved the ambiguity regarding the factor structure of the IECI by demonstrating that the bifactor ESEM model is the most valid and informative representation. The main finding is that the IECI predominantly measures a strong and reliable general EC construct, with smaller contributions from three specific sub-dimensions. The contribution of this study is to provide a more robust theoretical and methodological foundation for understanding and measuring EC, as well as clearer practical guidelines for the use of the IECI in the assessment and development of entrepreneurs in Indonesia.
Acknowledgements
This article is a part of research originally conducted as part of Siti Zahreni’s doctoral thesis titled, ‘Driving Factors of Entrepreneurial Persistence among Micro and Small Enterprise Actors’, submitted to the Doctoral Psychology Program, Faculty of Psychology, Universitas Airlangga. The thesis was supervised by Seger Handoyo and Fajrianthi. The manuscript has been revised and adapted for journal publication. The original thesis was still in progress and unavailable at the time of publication.
The authors would like to express their gratitude to the Indonesian Education Scholarship (BPI), the Center for Higher Education Funding and Assessment (PPAPT), and the Indonesian Endowment Fund for Education (LPDP) for their support. The authors also extend their appreciation to the anonymous reviewers and editors for their constructive feedback.
Competing interests
The authors reported that they received funding from Beasiswa Pendidikan Indonesia, which may be affected by the research reported in the enclosed publication. The authors have disclosed those interests fully and have implemented an approved plan for managing any potential conflicts arising from their involvement. The terms of these funding arrangements have been reviewed and approved by the affiliated university in accordance with its policy on objectivity in research.
Authors’ contributions
S.Z., S.H. and F.F. contributed to the conceptualisation, data curation, writing (review and editing) and also provided all the necessary resources in this research. S.Z. contributed approximately 60% of the work, while S.H. and F.F. contributed 20% each of the research. S.H. and F.F. played a crucial work in supervising the research. All authors have read and approved the submitted version.
Funding information
The authors are grateful to BPI (Beasiswa Pendidikan Indonesia) for the financial assistance provided and to Universitas Sumatera Utara as well as Universitas Airlangga for supporting this research.
Data availability
The data that support the findings of this study are available from the corresponding author, S.Z. upon reasonable request.
Disclaimer
The views and opinions expressed in this article are those of the authors and are the product of professional research. They do not necessarily reflect the official policy or position of any affiliated institution, funder, agency, or that of the publisher. The authors are responsible for this article’s results, findings, and content.
References
Anderson, N. (2003). Applicant and recruiter reactions to new technology in selection: A critical review and agenda for future research. International Journal of Selection and Assessment, 11(2–3), 121–136. https://doi.org/10.1111/1468-2389.00235
Audretsch, D.B., Carree, M.A., & Thurik, A.R. (2001). Does entrepreneurship reduce unemployment? Tinbergen Institute Discussion Paper 01-074/3, Tinbergen Institute.
Bacigalupo, M., Kampylis, P., Punie, Y., & Van Den Brande, L. (2016). EntreComp: The entrepreneurship competence framework. EUR 27939 EN. Publications Office of the European Union.
Bentler, P.M., & Bonett, D.G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588–606. https://doi.org/10.1037/0033-2909.88.3.588
Bird, B. (2019). Toward a theory of entrepreneurial competency. In Seminal ideas for the next twenty-five years of advances (Vol. 21, pp. 115–131). Emerald Publishing Limited.
Boyatzis, R.E. (1982). The competent manager: A model for effective performance. John Wiley & Sons.
Burns, G.L., Geiser, C., Servera, M., Becker, S.P., & Beauchaine, T.P. (2020). Application of the bifactor S–1 model to multisource ratings of ADHD/ODD symptoms: An appropriate bifactor model for symptom ratings. Journal of Abnormal Child Psychology, 48(7), 881–894. https://doi.org/10.1007/s10802-019-00608-4
Cárdenas-Gutiérrez, A.R., Bernal-Guerrero, A., & Montoro-Fernández, E. (2021). Construction and validation of the basic scale of entrepreneurial competencies for the secondary education level. A study conducted in Spain. PLoS One, 16(4), e0249903. https://doi.org/10.1371/journal.pone.0249903
Christian, M.S., Edwards, B.D., & Bradley, J.C. (2010). Situational judgment tests: Constructs assessed and a meta-analysis of their criterion-related validities. Personnel Psychology, 63(1), 83–117. https://doi.org/10.1111/j.1744-6570.2009.01163.x
Cronk, L. (2017). Culture’s influence on behavior: Steps toward a theory. Evolutionary Behavioral Sciences, 11(1), 36. https://doi.org/10.1037/ebs0000069
De Beer, L.T., & Van Zyl, L.E. (2019). ESEM code generator for Mplus. Software.
Dong, Y., & Peng, C.-Y.J. (2013). Principled missing data methods for researchers. SpringerPlus, 2(1), 222. https://doi.org/10.1186/2193-1801-2-222
Draksler, T.Z., & Sirec, K. (2021). The study of entrepreneurial intentions and entrepreneurial competencies of business vs. non-business students. Journal of Competitiveness, 13(2), 171–188. https://doi.org/10.7441/joc.2021.02.10
Eid, M., Geiser, C., Koch, T., & Heene, M. (2017). Anomalous results in G-factor models: Explanations and alternatives. Psychological Methods, 22(3), 541. https://doi.org/10.1037/met0000083
Fritzsche, B.A., Stagl, K.C., Salas, E., & Burke, C.S. (2006). Enhancing the design, delivery, and evaluation of scenario-based training: Can situational judgment tests contribute. In J.A. Weekley & R.E. Ployhart (Eds.), Situational judgment tests: Theory, measurement, and application (pp. 301–318). Lawrence Erlbaum Associates Publishers.
Guion, R.M. (2011). Assessment, measurement, and prediction for personnel decisions. Routledge.
Hair, J.F., Black, W.C., Babin, B.J., & Anderson, R.E. (2014). Multivariate data analysis: Pearson new international edition PDF eBook. Pearson Higher Ed.
Hausknecht, J.P., Day, D.V., & Thomas, S.C. (2004). Applicant reactions to selection procedures: An updated model and meta-analysis. Personnel Psychology, 57(3), 639–683. https://doi.org/10.1111/j.1744-6570.2004.00003.x
Heinrich, M., Geiser, C., Zagorscak, P., Burns, G.L., Bohn, J., Becker, S.P., Eid, M., Beauchaine, T.P., & Knaevelsrud, C. (2023). On the meaning of the ‘P factor’ in symmetrical bifactor models of psychopathology: Recommendations for future research from the bifactor-(S−1) perspective. Assessment, 30(3), 487–507. https://doi.org/10.1177/10731911211060298
Hooper, D., Coughlan, J., & Mullen, M.R. (2008). Structural equation modelling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6(1), 1.
Hu, L., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
Johnson, B.R. (1990). Toward a multidimensional model of entrepreneurship: The case of achievement motivation and the entrepreneur. Entrepreneurship Theory and Practice, 14(3), 39–54. https://doi.org/10.1177/104225879001400306
Joiner, D.A. (2002). Assessment centers: What’s new? Public Personnel Management, 31(2), 179–185. https://doi.org/10.1177/009102600203100204
Jöreskog, K.G., & Sörbom, D. (1993). LISREL 8: Structural equation modeling with the SIMPLIS command language. Scientific Software International.
Lans, T., Verstegen, J., & Mulder, M. (2011). Analysing, pursuing and networking: Towards a validated three-factor framework for entrepreneurial competence from a small firm perspective. International Small Business Journal, 29(6), 695–713. https://doi.org/10.1177/0266242610369737
Lievens, F., Buyse, T., & Sackett, P.R. (2005). The operational validity of a video-based situational judgment test for medical college admissions: Illustrating the importance of matching predictor and criterion construct domains. Journal of Applied Psychology, 90(3), 442. https://doi.org/10.1037/0021-9010.90.3.442
Man, T.W.Y., Lau, T., & Chan, K.F. (2002). The competitiveness of small and medium enterprises: A conceptualization with focus on entrepreneurial competencies. Journal of Business Venturing, 17(2), 123–142. https://doi.org/10.1016/S0883-9026(00)00058-6
Man, T.W.Y., Lau, T., & Chan, K.F. (2008). Home-grown and abroad-bred entrepreneurs in China: A study of the influences of external context on entrepreneurial competencies. Journal of Enterprising Culture, 16(2), 113–132. https://doi.org/10.1142/S0218495808000065
Marsh, H.W., Morin, A.J.S., Parker, P.D., & Kaur, G. (2014). Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Annual Review of Clinical Psychology, 10, 85–110. https://doi.org/10.1146/ANNUREV-CLINPSY-032813-153700
McDaniel, M.A., Morgeson, F.P., Finnegan, E.B., Campion, M.A., & Braverman, E.P. (2001). Predicting job performance using situational judgment tests: A clarification of the literature. Journal of Applied Psychology, 86(4), 730–740. https://doi.org/10.1037/0021-9010.86.4.730
Mitchelmore, S., & Rowley, J. (2010). Entrepreneurial competencies: A literature review and development agenda. International Journal of Entrepreneurial Behaviour & Research, 16(2), 92–111. https://doi.org/10.1108/13552551011026995
Morin, A.J.S., Myers, N.D., & Lee, S. (2020). Modern factor analytic techniques: Bifactor models, exploratory structural equation modeling (ESEM), and Bifactor-ESEM. In G. Tenenbaum, R.C. Eklund, & N. Boiangin (Eds.), Handbook of sport psychology (pp. 1044–1073). Wiley.
Rahman, S.A., Amran, A., Ahmad, N.H., & Taghizadeh, S.K. (2015). Supporting entrepreneurial business success at the base of pyramid through entrepreneurial competencies. Management Decision, 53(6), 1203–1223. https://doi.org/10.1108/MD-08-2014-0531
Raykov, T., & Marcoulides, G.A. (2016). Scale reliability evaluation under multiple assumption violations. Structural Equation Modeling: A Multidisciplinary Journal, 23(2), 302–313. https://doi.org/10.1080/10705511.2014.938597
Reise, S.P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. https://doi.org/10.1080/00273171.2012.715555
Riyanti, B.P.D., Suryani, A.O., Sandroto, C.W., & Soeharso, S.Y. (2022). The construct and predictive validity testing of Indonesian entrepreneurial competence inventory-situational judgment test model. Journal of Innovation and Entrepreneurship, 11(1), 3. https://doi.org/10.1186/s13731-022-00202-x
Riyanti, B.P.D., & Suwartono, C. (2018). Psychometric evaluation of newly developed self-assessment of entrepreneurial competencies. International Journal of Applied Business and International Management, 3(1), 5–13. https://doi.org/10.32535/ijabim.v3i1.73
Roberts, C., & Voorpostel, M. (2023). Combining data collection modes in longitudinal studies. In D. Spini & E. Widmer (Eds.), Withstanding vulnerability throughout adult life: Dynamics of stressors, resources, and reserves (pp. 359–373). Palgrave Macmillan.
Rodriguez, A., Reise, S.P., & Haviland, M.G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137. https://doi.org/10.1037/met0000045
Schmitt, N., & Chan, D. (2013). Situational judgment tests: Method or construct? In J.A. Weekley & R.E. Ployhart (Eds.), Situational judgment tests (pp. 135–155). Psychology Press.
Shane, S., & Venkataraman, S. (2000). The promise of entrepreneurship as a field of research. The Academy of Management Review, 25(1), 217–226. https://doi.org/10.5465/amr.2000.2791611
Sorrel, M.A., Olea, J., Abad, F.J., De la Torre, J., Aguado, D., & Lievens, F. (2016). Validity and reliability of situational judgement test scores: A new approach based on cognitive diagnosis models. Organizational Research Methods, 19(3), 506–532. https://doi.org/10.1177/1094428116630065
Spencer, L.M., & Spencer, P.S.M. (2008). Competence at work models for superior performance. Wiley India Pvt. Limited. Retrieved from https://books.google.co.id/books?id=2Y8QB-6aIJMC
Suryani, A.O., Riyanti, B.P.D., & Sandroto, C.W. (2021). The construction of entrepreneurial competence test: Situational judgment test model. Academy of Entrepreneurship Journal, 27(5), 1–11. https://doi.org/10.1186/s13731-022-00202-x
Tabachnick, B.G., & Fidell, L.S. (2013). Using multivariate statistics. Pearson Education.
Van Zyl, L.E., & Ten Klooster, P.M. (2021). Exploratory structural equation modeling: Practical guidelines and tutorial with a convenient online tool for Mplus. Frontiers in Psychiatry, 12, 795672. https://doi.org/10.3389/fpsyt.2021.795672
Weekley, J.A., & Ployhart, R.E. (2005). Situational judgment: Antecedents and relationships with performance. Human Performance, 18(1), 81–104. https://doi.org/10.1207/s15327043hup1801_4
Whetzel, D., Sullivan, T., & McCloy, R. (2020). Situational judgment tests: An overview of development practices and psychometric characteristics. Personnel Assessment and Decisions, 6(1), 1. https://doi.org/10.25035/pad.2020.01.001
Wulani, F., Lindawati, T., Putro, A.J.W.T., & Suhartatik, A. (2019). Scale development of entrepreneurial competency of sme owner in Indonesia. Academy of Entrepreneurship Journal, 25(4), 1–12.
Appendix 1
| TABLE 1-A1: Sample items from the Indonesian Entrepreneurial Competence Inventory. |
|
Hide
Show all
Email this article (Login required)
