Niedermayer.ca
Published on Niedermayer.ca (https://niedermayer.ca)

Home > Identifying Predictors for Success in Information Technology Education > Method

Method

Overview

In this experiment, an incoming first-year class of Computer Information Technology (CIT) students were administered a survey assessing them on a number of independent variables including personality traits, learning styles, time management and logical problem solving ability. The results of this survey were then compared with the dependent variable as measured by the students’ actual academic performance during their first semester of an introductory course in computer programming.

  • Log in [1] to post comments

Participants

49 first year CIT students were enrolled at the start of the fall 2010 semester at Lethbridge College in Lethbridge, Alberta, Canada. Of these students, 43 agreed to participate in the study, an 87% response rate. By the time the first major or midterm exam was conducted in late October, 35 students remained enrolled in the course and wrote the exam. Only 27 students remained in the course to write the second midterm and final exam.

Of the students participating in this study, 2 were female; only 1 of which completed the course. The average age of the participants was 24 years, with the oldest being 50 years old and the youngest being 18.

  • Log in [2] to post comments

Pre-Study Instrument

The pre-test instrument included:

  • The Big Five Inventory[1];
  • The Index of Learning Styles (ILS)[2];
  • 11 questions asking the student to rate their time management ability and practices;
  • 11 questions asking the student to estimate the amount of time spent per week engaged in a number of recreational activities;
  • 12 questions asking the student to solve logical problems that will involve disjunctive logic and analytical reasoning.

The BFI was used because it is a shorter instrument comprising only 44 questions, and is freely available for non-commercial use. Its use by Allen and Robbins has been shown to correlate with measures of student motivation as measured by the SRI.

The ILS was selected because it is developed from a composite of learning models, namely the Myers-Briggs Type Indicator (MBTI), Kolb’s Learning Style Model and the Herrmann Brain Dominance Instrument (Thomas, Ratcliffe, Woodburry, & Jarman, 2002). It is free of charge and is administered as a web application with immediate scoring.

A review of available literature found no suitable questionnaire or other instrument to measure the time management practices or recreational activities of post-secondary students. For this reason, a series of 22 questions were developed to determine if time management and the allocation of recreational time by students can provide any prediction of their academic success. These questions and the range of responses is listed in Appendix A.

Problem solving ability was measured using a series of questions proposed by Toplak and Stanovich (2002) and augmented by some less demanding questions proposed by Frederick (2005). One analytical reasoning question (Newstead, Bradon, Handley, Dennis, & Evans, 2006) was also included in an attempt to give this factor a wide spectrum for assessment. The text of these questions is found in Appendix B.

[1] The BFI is copyright 1991 by Oliver Johns. It is used with permission.

[2] The ILS is copyright 1991 and 1994 by North Carolina State University and authored by Richard M. Felder and Barbara A. Soloman. It is and used with permission.

 

  • Log in [3] to post comments

Procedure

Following an introduction to the study and the solicitation of informed consent from participants, the pre-study instrument was provided to students in early September. Students completed the survey during a regular class period.

The evaluations were graded as follows:

  • The BFI and ILS scores were calculated using the algorithms provided by their developers and authors.
  • A time management scale was calculated from the first 11 time management questions by scoring each Likert item from 1 to 5. Questions numbered 2, 3, 7, 9 and 10 were reverse scored. The total of these values was then divided by the number of questions answered to provide an index against which to compare student responses.
  • Time estimates for each recreational activity were created from questions 12-22 of the time management questions using the median value for each time interval’s range. Since the 6th choice has no maximum value, it was arbitrarily assigned a value of 20 hours for this calculation. For example, if a student chose the 4th option, “More than 5 hours and up to 10 hours”, that activity was given a value of 7.5 hours for that student. The total of all these values for each student was calculated as the total recreation time for the student.
  • Each logical problem question was evaluated on a scale of 0 to 1.
  • For scoring purposes, questions 1 and 8 were counted as a single question and justified by the principle that if a participant is going to choose that 1/3 of a population will live, then he or she must also be prepared to accept that 2/3 will die; or conversely if the student wants to play the odds that there is a one in three chance that everyone will be saved, then he or she must also accept that there is a two in three chance that no one will be saved.
  • Problem 6 was scored so that each card was worth one quarter of the entire question. A full value of 1 point was earned only if the participant turned over every required card and left all unnecessary cards unturned.
  • Problem 12 was similarly scored so that each choice was worth 0.20 of a point. Only by not choosing incorrect choices and choosing the correct choice could a participant earn a full point.
  • All other problem solving questions were scored either 0 (if the participant made an incorrect choice) or 1.

As part of their regular course work, students completed three major exams during the semester; these exams and the final exam grade were then compared to the responses from the pre-study survey. Students who dropped the class prior to completing the first exam were excluded from the study. A cumulative performance score composed of the average of all exams completed by a student was also calculated.

  • Log in [4] to post comments

Source URL:https://niedermayer.ca/node/106

Links
[1] https://niedermayer.ca/user/login?destination=node/106%23comment-form [2] https://niedermayer.ca/user/login?destination=node/107%23comment-form [3] https://niedermayer.ca/user/login?destination=node/108%23comment-form [4] https://niedermayer.ca/user/login?destination=node/109%23comment-form