31.20.40 Medical Student’s Performance and Satisfaction with Online Theory Exams during the Covid-19 Pandemic

Original Article

 

Effect of Online Theory Exams Covid-19

Medical Student’s Performance and Satisfaction with Online Theory Exams during the Covid-19 Pandemic

Muhammad Suleman Sadiq Hashmi, Afifa Tabassum, Sobia Ali and Faiza Ambreen

ABSTRACT

Objective: To compare the performance of medical undergraduate students in online and traditional exams and to gauge student satisfaction regarding online exams.

Study Design: Cross Sectional study

Place and Duration of Study: This study was conducted at the Liaquat National Hospital and Medical College during the period of June-July 2020.

Materials and Methods: The target population was second year MBBS students who had given a paper based traditional exam in their first year of medical college and an online exam in second year. Scores of both the exams were compared. Students ‘opinions about the online exam were collected through an online survey. Data was analyzed using SPSS version-17.

Results: There were 88 students who had attempted both year 1 and 2 examinations. There were statistically significant differences amongst median scores of both exams [p=0.016]. Out of 78 responses received majority of the students were satisfied with the instructions and felt that the amount of proctoring was sufficient for a fair exam. Students were equally divided in term of technical or connectivity issues and background noise. Regarding the entire online examination most students reported similar concerns i.e. not fully prepared for the exam due to Covid-19 patients at home and lack of resources. As compared to traditional exam students found online exam difficult to attempt due to technical issues such as power outages, internet issues and software glitches.

Conclusion: With better performance in an online exam, students still prefer the traditional exam format better than online exams due to several reasons. This highlights the importance of investing in e-learning and e-assessment infrastructure.

Key Words: Students’ Performance, Online assessment, Traditional Exam, Medical Education

Citation of article: Hashmi MSS, Tabassum A, Ali S, Ambreen F. Medical Student’s Performance and Satisfaction with Online Theory Exams during the Covid-19 Pandemic. Med Forum 2020;31(10):172-176.

 

 

INTRODUCTION

Globally educational institutions are facing suspension of all academic activities due to pandemic. Institutes have a responsibility to their students to continue delivering education as these restrictions may be prolonged further. Especially institutions like medical university cannot remain suspended as there is growing demand of medical workforce in the face of COVID-19 pandemic. Hence the current circumstances have increased the utilization of modalities such as video conferencing, mobile apps, surveys, and WhatsApp messages.

 

 

Department of Health Professions Education, Liaquat National Hospital & Medical College, Karachi.

 

 

Correspondence: Dr. Muhammad Suleman Sadiq Hashmi, Lecturer, Department of Health Professions Education, Liaquat National Hospital & Medical College, Karachi.

Contact No: 03105616984

Email: dr_muhammadsuleman@yahoo.com

 

 

Received:    August, 2020

Accepted:    September, 2020

Printed:        October, 2020

 

 

 

Institutions are becoming more interested in engaging their students and delivering quality content1,2

When Covid-19 was declared a pandemic by the WHO soon after medical institutions changed their academic and teaching strategies to meet the students’ needs during lockdown. Some medical institutions focus on online teaching while other emphasized on self-learning abilities of students. Assessment methods were also modified as some medical institute’s implemented pass/fail exams, online exams, and research projects while others postponed their final exams. In Pakistan, medical institutes began online lectures and altered the evaluation benchmarks for the foundation academic years to mainly depend on online examination, assignments and quizzes while the clinical rotations and final exams of the clinical years were postponed3-5.

Worldwide, online examinations are being used for assessment in both traditional and online learning. Due to the current Covid-19 pandemic their use has increased further6. Online examinations come with both advantages and disadvantages. Advantages include time reduction, quick results, cost effectiveness and automated record keeping for item analysis6, 7. It also gives the student the luxury to give the exam in a non-threatening environment leading to decreased test anxiety as compared to a traditional exam format with high test anxiety and stress. This is one important factor that is strongly related to student achievement8. In contrast disadvantages include technical problems, security concerns, external resources required, faculty and student resistance7. There is a small amount of literature evaluating student performance in online examination. Students’ opinions are also important in success and effectiveness of online exams.

Liaquat National Hospital and Medical College initiated the use of online exams in the context of the current pandemic. Online exams were utilized for conducting the Mid-term and Pre-prof exams of MBBS undergraduates. The current study aims to examine the academic accomplishment of students in online exams as compared to traditional exams and to gauge student satisfaction regarding online exams.

MATERIALS AND METHODS

This cross sectional study was conducted at Liaquat National Hospital and Medical College. The target population was second year MBBS students who had given a paper based traditional exam in their first year of medical college and an online exam in second year. The first year exam comprised of four modules i.e. Locomotor, Respiratory-1, Cardiovascular-1 and GIT&Liver-1, which were held during the period of July-September 2019. The second year midterm exam comprised of 3 modules i.e. Neurosciences-1, Special senses and Head & Neck which was held in June 2020. The scores of all students who had given both first year and second year exams were included. Those students who had given either one of the exams or were repeaters were excluded from the study.

Both exams consisted of content from three areas: anatomy, physiology and biochemistry and had questions in the one best choice type format. The traditional paper based exam was conducted in the college premises. Students marked their answers on OMR sheets that were graded later electronically. The online exam was conducted via EDMODO and ZOOM. EDMODO was used to deliver the paper whereas ZOOM was used to monitor the exam. Since this was the first experience of students with online exam, a mock exam was held one day before the exam. Students logged on to both ZOOM and EDMODO from their homes on the day of the exam and they were briefed about the process again. If they encountered any problems they could communicate with administrator or post their query via ZOOM. Upon completion of the exam the papers were graded automatically in EDMODO.

The scores of all students in both the exams were recorded on an excel sheet. The overall maximum score of year 1 exam was 250 and of second year was 150. Student scores were converted to percentages. Data was analyzed using SPSS version-18. Statistical assumptions were evaluated using statistical tests.

Students’ opinions about the online exam were collected through an online survey immediately after the exam. The survey consisted of 15 questions out of which 12 were closed ended and 3 were open ended questions. Frequencies were calculated for quantitative output response. Responses of open ended questions were analyzed and coded.

RESULTS

There were 88 students who had attempted both year 1 and year 2 examinations. As the data was non-normal, Mann-Whitney U Test was used to demonstrate statistical significance of data set. There were statistically significant differences amongst median scores of both exams [p=0.016] (Table I).

Table No. 1: Comparison of student scores in different methods of examination

Ranks

Group

No. of students

Mean Rank

Sum of Ranks

Median

P-Value*

First year traditional exam

88

79.23

6972.00

62.0

0.016

2nd year online exam

88

97.77

8604.00

63.5

Total

176

   

 

 

*Using Mann-Whitney U test with p value <0.05 as statistically significant

 

 

Table No.2:  Students’ perception of online examination

1. How do you feel about online exam process?

Great

Pretty Good

OK

Not so Good

Terrible

12%

15%

33%

27%

13%

2. What was the level of difficulty of exam?

Very Easy

Easy

Neutral

Difficult

Very Difficult

3%

13%

47%

24%

13%

3. How do you feel about your performance in this exam?

Great

Pretty Good

OK

Not so Good

Terrible

4%

21%

40%

25%

10%

4. Is your performance in this exam a good indicator of how well you know/understand the topics/content?

Yes

No

60%

40%

6. How well did Zoom class rooms sessions, Edmodo activities and assignment/quizzes prepare you for this exam?

Not at all                                                 (0)

(1)

(2)

(3)

Really well (4)

14%

24%

26%

22%

14%

7. How much time did you spend studying for this exam?

Weeks

Days

Hours

35%

55%

10%

11. How well did the Mock exam prepare you for this examination?

Not at all                                                 (0)

(1)

(2)

(3)

Really well (4)

20%

23%

17%

17%

23%

12. Instructions and Pre-Requisites for this online exam were clear and communicated prior to exam?

Yes

No

90%

10%

13. Time allocated for the exam was sufficient

Strongly Disagree

Disagree

Neutral

Agree

Strongly Agree

17%

13%

21%

18%

31%

14. The invigilation/proctoring was sufficient for conducting a fair exam?  

Strongly Disagree

Disagree

Neutral

Agree

Strongly Agree

9%

12%

17%

31%

31%

 

15. Did you face any technical or internet issues during the exam?

Yes

No

47%

53%

16. Online exam was better than traditional exam

Strongly Disagree

Dis-agree

Neutral

Agree

Strongly Agree

34%

18%

23%

12%

13%

               

Open ended questions had varied response rates. Responses were analyzed and coded (Table 3).

Table No.3:  Student responses on different aspects of online exam

Question

 

Code

No. of responses

Percentage

Do you think your performance in this exam is a good indicator of how well you know/understand the topic/content?

 

Satisfied with their performance

14

31%

Couldn’t focus on paper due to technical issues or background noises

8

18%

Disturbance due to Covid 19 patients at home

7

16%

Teaching sessions didn’t prepare well for exam

6

13%

Didn’t perform well due to less preparation time or less exam time

8

18%

Lack of resources due to unforeseen events in pandemic

2

4%

Total

45

100

If you could go back in time and study for this exam again, what would you do differently?

Organize study habits/ change learning strategies

27

43%

Give more time to study

24

38%

Focus more on clinical sciences

7

11%

Focus more on basic sciences

5

8%

Total

63

100

Why was Online exam not better than traditional exam?

Technical issues (power outages, internet issues, software glitches)

24

35%

Difficult to attempt as compared to traditional exam

11

17%

Accessibility to unfair means

10

15%

Back ground noise/ disturbance due to students as well as invigilators

10

15%

Exam decorum cannot be maintained at home

5

7%

More financial burden

3

4%

Fine with both versions of exam

3

4%

Less time was given to attempt exam

2

3%

Total

68

100

 

 

 

 

 

DISCUSSION

During the past 5 month of Covid-19 lockdown universities have been closed and students faced uncertainty regarding their academic future1, 9. For the first in both government and private institutions online examinations are being used by universities9.

The objective of the present study was to examine the academic accomplishment of students in online exams as compared to traditional exams and to gauge student satisfaction regarding online exams. The study population was second year MBBS students who have attempted the traditional paper based exam in first year and online mid-term examination in second year with similar subjects i.e. anatomy, physiology and biochemistry. 

Statistical testing showed that students performed significantly well in online examination as compared to tradition (p-value, 0.01) other studies conducted relating to online examinations have reported similar finding in regards to students satisfaction and performance10-13. With regards to the logistics and operations including difficulty level, allocated time, overall feel, and their performance of online examination students’ reported neutral responses. Students were satisfied with the instructions, and Pre-Requisites reported it to be communicated clearly prior to exam and the amount of proctoring was sufficient for conducting a fair exam. In term of technical or connectivity issues students had an equal divide with almost half reporting facing issues with internet and back noise and more than a half percent did not face any such difficulty during exam.  Students considered online examination to reflect their aptitude and a good indicator of how well they understand the content. Similar results have been reported by other studies conducted on related topic14, 15.

Students considered the technical assistance provided by the institute though the Zoom class rooms’ sessions, Edmodo activities and online assignments and quizzes as well as the Mock exam were sufficient to prepare them for the final online examination. On average students spends days to weeks on exam preparation. Although students scored better in online examinations (97.77) as compared to traditional examinations (79.23), based on the responses to question 16 majority students disagree that online exams are better than traditional method16. Previous studies suggested students showed significant satisfaction and better performance with online examination12. One study however reported that students performance and scores were far better with traditional method as compared to online17. In another study result reported that students were satisfied with online examination however their performance did not show any correlation to that finding18.

Students were asked to share their views regarding the entire online examination experience and most students reported similar concern, students share they were not fully prepare for the exam due to Covid-19 patients at home, lack of resources due to unforeseen event in pandemic. Students shared that if given the time they will be more organized, spend more time on studying. As compared to traditional exam students found online exam difficult to attempt due to technical issues such as power outages, internet issues, Edmodo glitches, accessibility to unfair means, background noise/ disturbance due to students as well as invigilators. These difficulties and limitation in online learning and examinations are highlighted by similar studies previously conducted19-22.

These finding highlight the importance of investing in E-learning infrastructure, assist in documenting the effects of changing academic paradigms, according to students though online examination is flexible, and easily assessable still there are some issues that are needed to be resolved.

CONCLUSION

With better performance in an online exam, students still prefer the traditional exam format better than online exams mainly due to technical issues. This highlights the importance of investing in e-learning and e-assessment infrastructure. Understanding the advantages and challenges of online exams will not only help us but other institutes to plan and execute this process really well.

Author’s Contribution:

Concept & Design of Study:

Muhammad Suleman Sadiq Hashmi

Drafting:

Afifa Tabassum, Sobia Ali

Data Analysis:

Sobia Ali, Faiza Ambreen

Revisiting Critically:

Muhammad Suleman Sadiq Hashmi, Afifa Tabassum

Final Approval of version:

Muhammad Suleman Sadiq Hashmi

Conflict of Interest: The study has no conflict of interest to declare by any author.

REFERENCES

1.      Mukhtar K, Javed K, Arooj M, Sethi AJPJoMS. Advantages, Limitations and Recommendations for online learning during COVID-19 pandemic era 2020;36(COVID19-S4).

2.      Hodgson JC, Hagan PJMe. Medical education adaptations during a pandemic: transitioning to virtual student support 2020;54(7):662-3.

3.      Goda O, Mahdy D, Yosef M, Amin HA, Ahmed SSJSP. Students' Awareness as an Underlying Factor for Satisfaction and Compliance, 2020.

4.      Ferrel MN, Ryan JJJC. The impact of COVID-19 on medical education 2020;12(3).

5.      Rose SJJ. Medical student education in the time of COVID-19, 2020.

6.      Ilgaz H, Adanır GAJE, Technologies I. Providing online exams for online learners: Does it really matter for them? 2020;25(2):1255-69.

7.      Barkley APJJoA, Economics A. An analysis of online examinations in college courses. 2002;34(1379-2016-113429):445-58.

8.      Austin EJ, Saklofske DH, Mastoras SMJAJoP. Emotional intelligence, coping and exam‐related stress in Canadian undergraduate students. 2010; 62(1):42-50.

9.      Ahmed Abdullah N, Sultana Mirza MJIRoRiO, Learning D. Evaluating Pre-Service Teaching Practice for Online and Distance Education Students in Pakistan: Evaluation of Teaching Practice 2020;21(2):81-97.

10.  Ambikairajah A, Tisdell CCJJoETS. E-Examinations and the Student Experience Regarding Appropriateness of Assessment and Course Quality in Science and Med Sci  2019; 47(4):460-78.

11.  Azam F, Shaheen A, Irshad K, Javed N, Ata MJJoST-e-MU. Trends of undergoing formative assessment in undergraduate medical students 2018;1(1):21-6.

12.  Al-Majeed S, Al-Zubaidy SN, Al Salmi S. Online Exams For Better Students' Performance 2015.

13.  Al Salmi S, Al-Majeed S, Karam J. Online Exams For Better Students' Performance 2019.

14.  Shraim KJTOJoDE. Online Examination Practices in Higher Education Institutions: Learners' Perspectives 2019;20(4):185-96.

15.  Turner J, Briggs GJA, Education EiH. To see or not to see? Comparing the effectiveness of examinations and end of module assessments in online distance learning 2018;43(7):1048-60.

16.  Sarrayrih MA, Ilyas MJIJoCSI. Challenges of online exam, performances and problems for online university exam 2013;10(1):439.

17.  Nahar KM, Shquier MMA, Alsmadi IJIJoCEE, Learning LL. Students performance between classical and online education: a comparative study 2016;26(4):359-71.

18.  Bahati B, Fors U, Hansen P, Nouri J, Mukama EJIJoETiL. Measuring Learner Satisfaction with Formative e-Assessment Strategies 2019;14(07): 61-79.

19.  Nekoueizadeh M, Bahrani TJIJoLL, Applied Linguistics World Considering challenges in educational system for implementation of
e-assessment 2013:89-99.

20.  Ramu T, Arivoli TJIJSER. A framework of secure biometric based online exam authentication: an alternative to traditional exam 2013;4(11):52-60.

21.  Brothen T, Peterson GJIJoIT, Learning D. Online exam cheating: A natural experiment 2012;9(2):
15-20.

22.  Kolcu G, Demir S, Gülle K, Atay T, Kolcu MİB, Koşar A. Evaluation of Transition to Distance Education in COVİD-19 Pandemic 2020.