For Faculty: ChatGPT & AI: AI Detectors

As the technology stands today, it's safest to avoid automated AI detection tools completely. "As of now, AI writing is undetectable and likely to remain so," frequent AI analyst and Wharton professor Ethan Mollick told Ars in July. "AI detectors have high false positive rates, and they should not be used as a result."

The article presents the discussion on detecting AI cheating on writing assignments. Topics include sports' most famous coaching curmudgeons have stories of lowly assistants invited into film rooms to talk shop; and copying the AI text word for word, occasionally deleting and changing words to make the rhythm look like authentic writing.

ChatGPT's performance is comparable, if not superior, to that of students in a multitude of courses. Moreover, current AI-text classifiers cannot reliably detect ChatGPT's use in school work, due to both their propensity to classify human-written answers as AI-generated, as well as the relative ease with which AI-generated text can be edited to evade detection. Finally, there seems to be an emerging consensus among students to use the tool, and among educators to treat its use as plagiarism. Our findings offer insights that could guide policy discussions addressing the integration of artificial intelligence into educational frameworks.

A recent crop of AI systems claiming to detect AI-generated text perform poorly—and it doesn’t take much to get past them. July 7, 2023