The increase in availability of generative AI has brought forth a number of AI detection tools that claim to distinguish between AI and human-generated text. Turnitin, GPTZero, and Copyleaks are a few examples. Relying on predictable word patterns and sentence structure, these tools provide a percentage indicating how much of the text was allegedly produced by a human versus AI.

Understandably, these tools have generated a great deal of interest among instructors who are anxious to know how much their students are using AI to complete coursework. But how reliable are these tools? Despite lofty claims, the accuracy of these tools is questionable. OpenAI published a statement that they do not work well. Numerous faculty have experienced mixed results when putting ChatGPT to the test. Some colleges that use Turnitin have decided to disable Turnitin’s AI detection software due to reports of false positives and ongoing fears of falsely accusing students of academic misconduct. A recent study at Stanford concluded that AI detectors are biased against non-native English speakers. Additionally, Open AI recently shut down its own detection software, AI Text Classifier, due to its low rate of accuracy.  

As of this writing, Johns Hopkins University (JHU) does not have an official policy or statement about using AI detection software. Faculty should communicate to students what are and are not appropriate uses of generative AI in the course. You should work with your divisional academic integrity officer or the Office of Student Conduct if students violate your course policies. If misconduct is suspected, it is highly recommended that instructors speak to students before making any accusations.  If you choose to use one of these tools, we suggest using it in conjunction with one or more of the following alternatives.

Alternative approaches to using AI detection tools: 

  • Include a syllabus statement with clear expectations of how AI is to be used in the course (if at all). Ask students to self-report how they used generative AI for the assignment.
  • Experiment with generative AI to see how it behaves; consider uploading your own assignments to look for patterns or common errors that emerge 
  • Familiarize yourself with students’ writing styles 
  • Consider redesigning assignments to make them more resistant to AI: 
    • Reference specific in-class discussions, activities, or readings in the assignment 
    • Ask students to include personal examples in their work 
    • Scaffold larger assignments into phased projects in which students must show iterative development of their work
    • Allow students to work on assignments during class 
    • Include a variety of assignment types (create a video, podcast, illustration, etc.) 

Facilitating Discussion Example: Naveeda Khan, Anthropology – Invitation to Anthropology Course

In the Invitation to Anthropology Course, the TAs and instructor would invite students whose papers had been flagged with a high probability of relying on generative AI to a discussion about the paper. The goal was not to accuse the students of misconduct, but evaluate if they understood the concepts included in the paper. This could be shared with students when the assignment is published: “Students whose papers are flagged by an AI detector will be invited to an oral discussion about the paper with the instructor or TA to evaluate their knowledge of the concepts and arguments made.”