AI Detectors
AI detectors are not very reliable. Temple University tested Turnitin and found it to be "incredibly inaccurate" (Assoc. Press, 8/10/23). According to an associate vice provost at Temple, it worked "best at confirming human work... but was spotty in identifying chatbot-generated text and least reliable with hybrid.
Students may get falsely flagged for cheating with AI. For example, a Texas A&M professor wrongly accused an entire class of using ChatGPT on final assignments. He pasted their responses into ChatGTP and asked if it generated the answers. ChatGPT affirmed that it had written the responses when it really hadn't.
According to the conclusions of a 2023 research article by Deora Weber-Wullf et al., current detection tools are neither accurate or reliable. All detection tools, including Turnitin, that were tested scored below 80% accuracy. Only five of the twelve tools tested above 70% accuracy. Educators "should focus on the process of development of student skills rather than the final product."
The Best Detector is Yourself
There are several ways to spot AI content:
- Unnaturalness. A lack of human touch is the most obvious.
- Cohesion. AI generated content may lack development and organization. On the flipside, the writing may be overly organized.
- Indirect speech. The writing may use indirect speech which reports on what someone else said or wrote without using that person's exact words.
- Repetition. It may contain repetitive phrases or sentences about the same content.
- Incorrect Sources of Information. AI recognized patterns of a college paper which typically include cited sources: volume, issue number, journal title, page numbers, etc. However, AI will make up the content, and when someone looks up the original source it does not exist.
- Inaccurate information. Facts and information may not add up to what you know or what is common knowledge.
- Inconsistent Writing Style. AI tends to maintain a consistent tone. Human writer, particularly students, tend to be inconsistent in tone throughout their writing. On the flipside, keep an eye out for sections that seem inconsistent. Parts of a paper may use AI generated content which can lead to an inconsistent writing style.
- Formulaic writing. AI is great with organizing, listing, and formatting, and this is a key characteristic of writing by AI.
- Mistakes, Typos, slang, and different styles of speech. AI will rarely make grammatical mistakes. However, humans will. A typo is actually a really good indicator that it was human written.
- Lack of Empathy, personalization, or a human touch. Content generated by AI tends to be factual and does not include personal opinions.
Common AI Detectors
