A SAMPLE SURVEY OF COMPUTER-BASED TRAINING WITH REFERENCE TO SUCCESS CRITERIA AND REMEDIAL PROCEDURES

The aim of the study iJ to conduct an inveniguion into the Uatul-quo of computer-hued trai ning (CBT) :r.t one of the le:r.ding Life Assurancc companies in South AfriCli. The investig:r.tion comprises the development of:r. generic Iheorcliollaxonomy of successful CBT which identifies thc theoretic:r.1 m:r.cro and micro criteria for successful CDT and iu implementation. Core differences between the ideal and :r.ctual CBT in practice are idcrr ti fied. Results indic:r.te th:r.t Ihe CDTal the company satisfies the majori ty of the micro :r.nd macro success criteria. A few inadcqu:r.cies arc identified. Recommendations arc made with regards to various remedial procedures. The incorparollion of the theoretical taxonomy and the remedial procedures would serve to increase the mccess of comp:r.nics CBT and could result in a highly efficient :r.nd effective C UT programme.

In the past, the philosophy on which education and trai ning in South Afrie was built, was of European origin which assumed a homogenous population.The trai nee population , in the"new"South Afric:r., is nowextraordinarily di vene in tenns of educ:r.tional,ethnic and language backgrounds and consequentl y requires an innovative app roach to training (Trollip, 1993).While the traditional lecture form of instruction has its merits, it cannOt be the only method oftTaini ng to cater for a heteroge nous tninec population.A novel form of individualised self-paced instruction is needed to cater for the di ffering b:r.ckgrounds of trainees and the vast numbers of individuals that need to be trained.Technologyb:r.sed training in the form of Computerbased Training (C BT) should be viewed as an essential and urgent trai ning tool to addreS5 these:: needs.In C BTsystems.there is a direct interaction between the trai nee and the computer, which has within its systems the necessary information and instruction:r.lm:r.terials for the programme.The computer's role in such:r.training system typicall y involves administering the trainee programme to the trainees and testing their perform ance after learning.By virtue of its storage and memory capabilities, the computer continuously assesses the trainees'progress :r.nd is able to :r.dapt the method and/or material presented to suit the trainees' particul:r.rneeds (Goldstein, 1993;Wexley & Latham, 1981).Thus, e DT refe ~ to an IIlteractive ~.for wpn1ltJ J'-IJ bt.JJ.wtJIX D. MtElIi,t«t.~ tJj 1nJwJ.tri..t PsrcltollifY.u";vmityofSld~h, Pri..ut &g X ( M",itl.o,..( 1602.11 learning experience between a learner and :r.computer in which the computer provides the majority of the stimulus, the learner must respond, and the computer analyses the respo nse and provides feedback to the learner" (Dickelman, 1994, p. 127).
Companies desire:r.tnini ng system that resul ts in superior job performance :r.mong all their employees.and ultimatel y an increase:: in profit.To accomplish this certain industries arc opting for the CBT method of instruction for it is a widely accepted fact that C DT has the capability of contributing subst:lIltially to the efficiency and effectiveness of trai ning programmes ( Kea~ley, 1983).The viability o f C BT, :r.s an option for training, is clearly illustrated by the benefits that CBT provides: CBTreduces trai ning costs as travel and living costs decrease; C DT reduces the length of training by app roximately 30% w hen compared with classroom training; C DT provides an increased student to instructor ratio; CBT facilitators are able to provide individualised instruction; the computer in CBT del iven standardised instruction every time; CBT is interactive in nature resulting in an increase in motivation on the part of the trainees; C DT provides reinforcement during the learning process through constant f~dback to the trainees and learning results obtained through CDT are seldom dearly better, but almost never worse than those obt:r.inedby mOre tradition:r.lways of le:r.rning (B:r.ird, Schneier & L:r.ird, 1983;Gastkemper, 1984;Goldstein. 1993;H:r.rt, 1987;Ke:r. nley, \983;Russ Eft. 1994;Trollip. 1993).
Certain disadv:r.nt:r.ges of C DT may se::rve as potential barriers to the successful implement:r.tionofCBTand although the adva ntages of CDT certa.inlyoutn umber the dis:r.dvannges, it does not diminish the importance of undennnding the potential drawbacks of CBT.Some disadvantages include cost, development time, availability of software and the lack of the human factor (Gerber, 1990;Goldstein, 1993;Heathman & Kleiner, 1991;Kemley, 1983;Russ Eft, 1994;Schlechter, 1991, Trollip, 1993).
From the above it would se::em that C BT still has the potential to address th e trai ning needs of business in South Africa today.Although many 0T¥anis:r.tionshave recognised these bencfiu and hlVC implemented CBT, the traditional lecturc m ethod of training still occupies a major percentage of training.CBThas, as a result.not been utilised to full capacity in this country (McElligott, 1997;Van Dyk, Ne\, Loccloltf & Haashroek, 1997).Th is phenomenon requires a diagnostic intervention in order to determine those specific factors that arc hindering the success of CBT in this country.In order to obain clarity on Ihis iuue, the initial goal of the survey is to identify and constitute the theoreticli prerequisites/criteria for successful eDT and its implementation.
[1\ identifying the success criteria of CBT, o ne can distinguish between certain micro and macro criteria th at need to be satisfied if successful eDT is to be attained.Micro success criteria refer to certain intri nsic factors pert:aining to th e development and implementation of CBT.These criteria nnge from the creation of effective screen designs to storyboarding and flow_ charting.Macro success criteri:a.on the other hand, relate to certain extnneous issues of C BT having a major beari ng o n the success of the design, development, implementation and the final result of a C BT project.These criteria can be divided imo two categories (I) genenl training principles.ranging from the need analysis to evaluation, thu need to be adhered to regudlcss of the medium of instruction and (2) org;misational factors such as managerial commitment.end-user sup-POrt, trainer dedication, etc. (McElligott. 1997).

Maero Factors: Training Principles
Regardless of the medium of instruction that is to be considered for a training prognmme, there uc general principles that have to be followed before one is able to embark upon the development of a speci fi c training programme.First and foremost it is imperative fo r the training department of an organisation to conduct a thorough training needs :assessment (Cline & Siebert, 1993;Costanzo. 1996).The focus of the needs assessment is placed on the identification of educational gaps and various no rmative-, dem:and-, comparable-and anticipated needs (Caffarella, 1994;Ensmus & Van D yk, 1999).Various models for determining lr3ining needs exist for example the Gnham and Mih:al model and the Mich:alak and Yager model (in Ensmus & V:an Dyk, 1999).Cogniunce must be taken of the fact that one model might not be su it:able for:all situations.Information regarding training needs is usually g:ained by such methods as surveys, job analyses, critic:al incidents, performance appraisals, interviews, focus groups, observation etc. (Erasmus & Van D yk, 1999;Oppenhei mer. 1983;Van Dyk. Nel. Locdolff & Ha:asbroek. 1997).Once the needs have been assessed :and prioritised, t:ask analysis is instigated whereby the knowledge.skills and abilities (KSA's) required for job performance are analysed (Alessi & Trollip, 1991;Holton & Bailey, 1995).
T he task analysis serves as the fram ework for mapping narrower and more specific instructional objectives which are developed to specify what will be accomplished by a training programme.The objectives must provide a specification of the actions, which the trainee should be able to perform, the conditions under which these actions are to be performed and the st:mdards of performa nce w hich must be met.T he sound development of objectives is absolutely essential to the success of the train ing course and is an unconditional prerequisite for the rest of the training design.Focus has however shifted toward learning outcomes.Outcomes place more emphasis on what the student will be able to do. the results of learning.applying learning in new areas and flexibl e allocation of time (Costanzo, 1996;Erasmus & Van Dyk, 1999;Holton & Bailey, 1995~ The next compon ent in th e theoretical taxonom y of successful C BT involves the selection and sequencing of content.Content selection involves the classification of the instructio nal objectives and the placement of specific events in an appropriate sequence for the attainment of the objectives.These specific events are then written as the lesson content (Gagne, Briggs & Wager, 1992).The issue of content selection forms a part of both the macro and micro factors influencing the suc-cess of CBT.With regard to the macro facton, it is dear th~t regardless of the medium of instruction to be utilised in the training, content has to be constructed to specifica.llyaddTffi each of the performance objectives.Added to this, however, is the ability of C BT to offer a considerable amount of softw~re options for varying the structure, features and nature of the lesson content.These options include the tools of branching, Aowchaning and storyboarding that are extremely important in the development phase of C BT programmes.
The success of any traini ng programme relies extellsively on astute decision-mak ing at critical poi nts in the development and implementation.Major decisions need to be made regarding the appropriateness of the media that the organisation plans to use.An investigation into the feasibility of C BTas a traini ng medium is not an option, but a necessity.Various methods are av:ailable in the literature which present guidelines for conducting a feasibi lity study (Adams, 1993;Johnson & Brigando, 1993;Kearsley, 1983;Van D yk, et aI., 1997;Wynn, 1994).Hart (1987) asserts that the many examples offailures associated with C BT are, to a large extent, due to the inappropriate choice and application of the CBT approach to training.Thus. the success of a C BT programme is dearly contingent upon the careful and delibente studyofits feasibility.
Having determined the appropriateness of CDT, an analysis of the trainee population becomes necessary to examine the specific needs of the trainees themselves (Costanzo. 1996;Rushby, \988).The trainee analysis serves to investigate the trainee's current skill level.their current knowledge level, their origins, their language and demographical information (8arotf, 1987).A further consideration.which is critical to the successful implementation of a CBT programme, is the compatibility of the programme with the principles of adult learning.which should also take cognisance of the various ways in which lea.ming takes place (learning theories).These principles should be foremost in the minds of the instructional tea m when C BT i!; embarked upon (Gagne et .1.1..1992;Sheal. 1989;Van Dyk et aI., 1999) The continual rise in training costs is creating an u.rgent need fo r tnining departments to demonstrate improved performance, return on investment (ROI) and financial results to top management.Thus. it is increasingly critical for the training team to investigate whether the skills and knowledge taught in th e training programme are in fa ct transferred and utilised in the working environ ment in order to justify the capital and other investments in training.Goldstein (1993. p. 147) defines evaluation as " .. .the systematic col1ection of descriptive and judgmental information necessal'y to make effective training decisions related to the selection , adoption, value and modi fication of various instructional activities': A differentiation can be made between two types of evaluation, that of formative evaluation and summative evaluation.Formative evaluation is utilised to determine if the tmining programme is operating as originally planned a.nd if improvements are necessary before the programm e is implemented (Goldstein. 1993).Summative evaluation determines the degree to which the training programme has been successful in affecting various criteria related to trainee behaviour and other organisational variables that are affected by the trainee behaviour (Gordon, 1994).Various evaluation models and methods.some more acce pted than others, exist that can be applied (Alliger & Janak, 1994;Bernthal, 1995;Cascio, 1991;Gagne et al.. 1992;Goldstein. 1993;Kirckpatrick, 1987) but ultimately the choice of a particular modelJmethod of evaluation lies with the individual organisation as each enterprise has its own needs regarding the evaluation ofCBT programmes.
Macro Factors: Organisational O rganisational factors frequently represent the underlying reasons for the failure of CBT applications (Adams. 1993).In referring to the Yorganisational factors" influencing the effectiveness of a C BT programme, the focus lies 011 the alignment/integration of C BT with other Human Resources systems within an organisation.C BT should form an align-~~\}\RONMENr T,,_ The theoretical taxo nomy is used in the study to assess the extent to which the dcve10pmem and implementation of eDT at one of the largest Life Assurance companies in South Africa adheres to the prerequisites (or eDT succ~s (McElligott, 1997).The theoretical taxonomy represents the "iclear use of eBT in theory :md provides the foundation for the construction of Ihe questions in the diagnostic questionnaire ; md the focus group which are utilised to asses the usage/application and knowledge base ofCBTat the Life Assurance company.If it becomes apparent that a mismatch exists between the criteria presented in the theoretical uxonomy and the satisfaction of these criteria in practice, then certain remedial procedures wi)) be outlined in order to minimise the discrepancy between the ~actual" and the ~ideal" situation.If, on the other hand, there appears to be complete congruence, then it an be assumed that the CBT in the organisation should be of a successful nature in terms of the planning, development and implemenlation thereof.

METHOD Sample
In this research project.one of the brgest Life Assurance groups in South Africa WlIS ap proached to assist in the survey.
The company initiated their investigation into CBT in the early '80's and is currently one of the brgest CBTusers in the country.A non-probability judgement sampli ng technique was used in the survey which was conducted on a national basis and included every branch of the company that is currently utilising CDT.A tOtal of750 questionnaires w;\s distributed via the company's internal mailing system in order to minimise com and maximise the response rate.Each branch received four qut'Stionnaires to be completed by four experienced C BT users.A fi nal response rate of 43,3% was obtained after two telephonic follow-up attempts were made at t wo week intervals after the expiry of the return date.
Cognisance must be taken of the fact that the non-probabilit y method of sampling curtails the genenlisation of the findings to the broader CBT user popubtion.

Measuring instrum ents
Two measuring instruments are utilised in the study, that of a questionnaire and a focus/discussion group (McElligott.1997).
Essentially, the aim of the instruments is to investigate to what extend the uxonomydepicted in Figure 1 are adhered to in the selected Life Assunnce company.
The diagnostic questionnaire utilised in the su rvey was constructed from the theoretical success criteria of CBT.Before finalisation of the questionnaire, a pilot study was conducted to determine and correct any problem areas in the questionnaire.Six mining/personnel experts in the pilot study verified that eilch item within the questionnaire was unambiguous and that each item would elicit the intended information.The questionnaire comprises four sections.Section A scrved to gather biographic data regarding the end-users• age.length of service, geographical si tuation.sex, academic qualifications, C BT course attendance, general computer usage and job grade.Section 0 aimed to elicit information concerning the endusers' perceptionsj opinions of the content presented in the COTcourse/s.. Section C focused specifically o n the screen displays of the CBT (i.e.micro criteria).The final section of the questionnaire el icited the end-users' general attitudes towards C OT as a method of instruction.A 5-point Likert-type scale was used (McElligott, 1997).
The focus/discussion group, consisting of members of the authoring team and management, was used to obtain further clarification on ceruin issues deriving from the diagnostic question naire.Certain macro issues such as the organisational variables, general training principles, etc., that ultimately have an effect on the successful implementation of a CBT project.were investigated.

Statistical Analysis
The data from the questionnaire were anal ysed using the Statistical Package for the Social Sciences (SPSS).The analysis included calculation of frequencies, me.1lns and standard devi.1ltions.1ISwell as various cross-tabulations and the correlations thereo( The me.1lns and su.ndard deviations of relevant questions are reported in brackets.

RESULTS
In terms of the Life Assurance Company's adherence to the micro success criteria of CBT, the results indicated that these criteria are, on the whole, satisfied in practice at the company.
C OT course content -macro and micro f.1lctors:It is evident that the process of task .1Inalysis is strictly adhered to by the authoring team (x = 1.46;SD = 0.69) and that the success criteria of effective storyboarding and flowcharting are fulfilled in practice (x = 1.62;SO = 0.76).The results indicate that the success criteria pertaining to certain facets of the individualisation component are actualised in practice, however, the success criterium regarding the consideration of the trainees' previous knowledge and experience is not fulfilled due to inadequate pre-testing procedures (i = 263; SO = 1.34).These inadequacies require urgent attention as they have negative implications fo r the effective creation of the branching system and the testing f.1lcilities.The enforcement of stricter controls over the pre-testing process, whereby each trainee is entitled to complete a single pre-test, would greatly assist the au thors in their branching design.The results illustnte that the "remedial" branching facet is successfully integrated into the CBT (i = 208; SO = 1.26).Th is is primarily due to the inadequate testing of the trainees' previous knowledge as well as the inability of the software to provide deuiled branching facilities.A more fl exible authoring s~tem /language may thus constitute a worthwhile investment.
There appears to be insufficient faciliution in the CBT courses as minimal guidance, support and educational facilitation is provided (x = 2.12; SO = 1.16).Extensive tnining is suggested to assist the "CST controllers" in thei r adaptation to the role of that of an educational COT facili u tor.This trai ning should encompass interpersonal skills training, principles of adult learning, subject matter training and hardware and software training.The presence of a well-tnined educational facilitator would remedy the lack of personal intenction in the CDT (x = 2.16; SO = 1.11), thus eliminating one of the major problems associated with the COTat the company, that of the lack of the human factor.
The results highlight certain problems regarding the level of difficulty of the testing within the COT courses (x = 1 12; SO = 1.06).The testing is directed at basic cognitive levels, that of knowledge and comprehension, while the higher-order levels are excluded.The attributing facto rs, once again, involve the limitations of the software and the inadequacy of the pre-test-IIlg process.
Screen design: Regarding the screen design of the COT courses, the text within the visual presentations is adequately presented.although there is 3. slight tendency towards textual screen crowding (x = 14; SO = 1.15).The utilisation of colour is effectively integrated into the instructional design and appears to contribute to the learning process (i = 206; SO = 0.95).Although the graphics in the COT courses are consistent with the textual displays and are effectively integrated into the overall instructional message, it appears that the gnphics do nOt playa major role with regard to the tninees' rate of comprehension and process ofleaming (i = 2.42; SO = 1.05).Thus, the incorporation and purpose of graphics in C OT needs to be questioned.
Macro factors: It appears that the majority of the success criteria, concerning the macro issues of the CST, are actu.1llised in practice at the company.Nonetheless.there are certain inadequacies present in the CBT approach.The resullS from the questionnaire reflect that the end-users find the COT to be effective in improving their learning (i = 2.06: SO = 0.99) and job performance (x = 1.92;SO = 0.%).However, the evaluation attempts fail to con finn these findings.as it appears that the company's approach to summative evaluation focuses primarily on the level one criterium, that of reaction.The company:S attempts to evaluate learning, behaviour/performance and results are insufficient.The measurement oflearning is problematic as it is based o n the inadequate pre-and post-testing process.Although a behaviounl measurement is conducted, this approach needs to be re-defined in order to determine whether the improved job performance can be attributed to the specific COT course treatment.The exclusion of the final level.that of a re-sullS evaluation, is highly problematic and detracts substantially from the entire CDTeffort at the company as the financial impact of the C OT is unknown.The effectiveness of COT is inextricably linked to the results of a cost-benefit analysis.The inclusion of these three levels.namely learning, behilviour and results is essential if the authoring team is intent on demonstnting or proving the worth of COTto the company.
It is evident that an environment of involvement.participation and co-operation exists amongSt the key players in the organisation.However, while tOP management do exhibit support and commitment to the CST effort, they are not directly involved in the progress of the CDT.A possible strategy to elicit the more direct involvement of lOp management in the C BT projects.would be to convince lOp management of the relevance that C BT has to the organisation in terms of its cost-effectiveness and its impact on the tninees' job performance.However, thi s could only be achieved if a formal and systematic summative evaluation of the CBTis implemented within the company.

CONCLUSIONS AND RECOMMENDATIONS
The results indicate that the CSTat the Life Assunnce company is largely adequate in nature and satisfies the majority of the micro and macro success criteria as outlined in the theoretical uxonomy of successful CBT.HO\\-"e"ver, a few inadequacies in the company's approach to CST are identified.the most important of which include problems with the C OT branching facilities, test construction, a lack of faciliUtion on the CDT courses i1 nd an insufficient approach to the summative evaluation courses.The recommendations of the study involved the proposal of various remedial procedures to .1Iddress the inadequacies in the company's C OT development and implementiltion.The incorpontion of these remedial procedures would serve to inaease the success of the compmy's COT and would result in a highly efficient and effective CBTprogramme.
With reg:ud to recommendations for funhe r research, the questionnaire should be re-distributed to the CBTusers in the Life Assur.mce company once the remedial proccduin have been incorpor-ned into the CBT courses in order [Q confirm these results..An attempt should he made to clicit a greater response rate from the C BTtrainee population in order to incre<lse and allow justified genenlisability ofth" results.This could he achieved by involving the facilitators in the study as they would have direct contact with the trainees and would be able to monitor the dissemination and return-datc of the question naires.
The limitations ohhe Life Assu r.lnce company's current COT, :liS presented in this study, should be artfull y considered when the company implements their planned Electronic Perform-:mce Support System s.The most important facton to consider are the implementatio n of cost-benefit analyses, comprehensive evalu~tion studies and feasibility studies.If the areas of concern in their C BT, as highlighted in this study.are taken into account, this will enable the company to proactively approach the implementation of their EPSS's rather than having to reactively respond to proble ms arising after the systems have been implemented.
T he questio nnaire c.m be utilised to construct a "success Criteri a Index ofCBT: This index can be used in any organisation to determine the extent to which the CBT in the o rganisatio n is successful in nature and to identify certain areas that require re mediation.However, it must be remembered that the index canno t be used in isolation to dete rmine the success of a CBT programme.An investigation into the macro facton must accompany the investigation into the micro factors as the twO are inextricably linked.
This study emphasises the fact that CBT has the ability to provide efficient and effective instructio n provided that certain criteria are adhered to in the development an implementation process.With individuals increasingly having to take responsibilit y for their development and learning into their own hands.CBT can provide a vital learning tool to train an emergi ng South Africa.The Government's Skills D evelopment Bill directs us to fi nd new ways of pre paring for the future by developing the capacity of education and training providers.In his introduction to the Green Paper. the then Labour Minister Tito Mboweni emphasised that the re-establishment of the linkages between learning and working is a condition for growth .Technology-enhanced learning, in the form of interactive C BT, h .. s the necessary potential to meet this challenge.Any company that is intent on developing its e mployees into a stronger and more productive workforce, through the CBT medium of instruction, should consider the guidelines provided in this study to ensure t he success of their C BT and the implementation thereof.

FIllldkln8IFigure 1 :
Figure 1: A theoretical taxono m y of successful CDT