The Revival of Essay-Type Questions in Medical Education: Harnessing Artificial Intelligence and Machine Learning

Muhammad Shahid Shamim, Syed Jaffar Abbas Zaidi, Abdur Rehman

Research output: Contribution to journalArticlepeer-review

Abstract

OBJECTIVE: To analyse and compare the assessment and grading of human-written and machine-written formative essays. STUDY DESIGN: Quasi-experimental, qualitative cross-sectional study. Place and Duration of the Study: Department of Science of Dental Materials, Hamdard College of Medicine & Dentistry, Hamdard University, Karachi, from February to April 2023. METHODOLOGY: Ten short formative essays of final-year dental students were manually assessed and graded. These essays were then graded using ChatGPT version 3.5. The chatbot responses and prompts were recorded and matched with manually graded essays. Qualitative analysis of the chatbot responses was then performed. RESULTS: Four different prompts were given to the artificial intelligence (AI) driven platform of ChatGPT to grade the summative essays. These were the chatbot's initial responses without grading, the chatbot's response to grading against criteria, the chatbot's response to criteria-wise grading, and the chatbot's response to questions for the difference in grading. Based on the results, four innovative ways of using AI and machine learning (ML) have been proposed for medical educators: Automated grading, content analysis, plagiarism detection, and formative assessment. ChatGPT provided a comprehensive report with feedback on writing skills, as opposed to manual grading of essays. CONCLUSION: The chatbot's responses were fascinating and thought-provoking. AI and ML technologies can potentially supplement human grading in the assessment of essays. Medical educators need to embrace AI and ML technology to enhance the standards and quality of medical education, particularly when assessing long and short essay-type questions. Further empirical research and evaluation are needed to confirm their effectiveness. KEY WORDS: Machine learning, Artificial intelligence, Essays, ChatGPT, Formative assessment.

Original languageEnglish
Pages (from-to)595-599
Number of pages5
JournalJournal of the College of Physicians and Surgeons--Pakistan : JCPSP
Volume34
Issue number5
DOIs
Publication statusPublished - 1 May 2024

Fingerprint

Dive into the research topics of 'The Revival of Essay-Type Questions in Medical Education: Harnessing Artificial Intelligence and Machine Learning'. Together they form a unique fingerprint.

Cite this