Analyzing The Accuracy of The Sentence Structure of Grammar Checker Across AI Tools and Students’ Perception

Abstract

This study aims to analyze the accuracy of AI-based grammar checking tools, they are QuillBot, Grammarly, and ChatGPT, in evaluating sentence structures within students’ hortatory exposition texts, as well as to explore students’ perceptions of using these tools in the writing process. Using a descriptive mixed-methods design, the research involved 36 eleventh-grade students at MA Almaarif Singosari, who were asked to write hortatory exposition texts. These texts were then evaluated by the three grammar checking tools. The resulting grammar scores were statistically analyzed using One Way ANOVA and Post Hoc Tukey tests to determine whether there were significant differences in grammar assessment accuracy among the tools. In addition, qualitative data were collected through semi-structured interviews with three students who obtained the highest scores from each tool group to gain insights into their experiences and perceptions. The findings revealed significant differences in the grammar scores produced by the three tools. Grammarly generated the lowest average score, indicating a high sensitivity in detecting grammatical errors, while ChatGPT gave the highest scores, suggesting a more lenient error detection process. QuillBot was found to be moderately accurate, detecting grammar issues and offering paraphrasing assistance, but lacking deep contextual understanding. From the interview results, students generally expressed positive perceptions, particularly appreciating the ease of use and flexibility of ChatGPT, the accuracy of Grammarly, and the paraphrasing features of QuillBot. However, some students also noted limitations, such as premium feature restrictions in Grammarly and occasional lack of contextual relevance in QuillBot and ChatGPT feedback. This study finds that although AI grammar checkers are a useful tool for enhancing writing, there are differences in their accuracy and usability. Teachers and students should use these tools critically and complementarily to enhance writing instruction, especially in argumentative genres like hortatory exposition. The study contributes to the increasing quantity of knowledge on AI-assisted language learning, offering both theoretical and practical implications for writing pedagogy in EFL contexts.