by Michael A. Smith, Professor of Political Science, Emporia State University

The first day of #MPSA2024 is in the books, and it’s gotten off to a strong start.  All panels I attended today were well-attended and thought provoking.  I wrapped up my workday by attending an excellent panel on teaching introductory courses, where the topic of teaching in an age of Artificial Intelligence (AI) moved front and center.

The use of AI to write student papers is a tricky situation.  All panel attendees and audience members had caught students doing this.  AI poses some unique problems because it generally cannot be detected with 100% accuracy.  When students plagiarize, the professor can document their transgression by producing the document which was plagiarized (often a Wikipedia entry or other, easily discovered source). If one’s university subscribes to them, apps like TurnItIn are available as plugins to learning management systems, so the work is done preemptively and a score measuring similarity to online sources is generated. The only way to plagiarize successfully is to use sources that evade these detectors and the professor’s other tools.  This is possible, but it generally involves more time and effort than simply writing one’s own, original paper in the first place.

Catching AI is a bit tricker.  AI is dynamic, that is, it is always adapting. Entering the same prompt into an AI generator twice will generate two different results.  AI detectors like GPTZero are incredibly useful, but they can generate only a probability that a paper is written by AI.  Furthermore, some critics argue that the use of AI for a variety of different tasks is inevitable now and in the future, so professors should adapt instead of fighting it.  What to do?

Our panelists generated a variety of different ideas, and audience members contributed a few additional thoughts.  Here they are, in no particular order.

  1. Return to in-class assignments. For face-to-face classes, there is no substitute for returning to classwork.  Bring back the blue book—take-home papers can be replaced with essay exams written under the watchful eye of the professor/proctor.  Several panelists have shifted their course points totals, de-emphasizing take-home assignments and moving the points to in-class exams.   These are often given with rules requiring that electronic devices (especially phones and laptops) be turned off and put away during the exam.
  2. Use AI yourself and get to know how it works. Know your enemy!  Panelists and audience members have also been getting to know AI.  I have done quite a bit of this myself, both using AI to help me get unstuck when writing computer code, and simply entertaining myself by writing ridiculous prompts (example:  write a script that explains Hegel’s Phenomenology of Spirit with a puppet show). As one becomes familiar with AI, a certain structure becomes clear.  AI-generated essays tend to have perfect spelling and grammar.  Most students do not, so that is the first tell.  In addition, the essays tend to feature boldface subheadings and have broad, vague, thematic answers.  With experience, it becomes relatively easy to spot these.  The text can then be entered into AI checkers like GPTZero to calculate a probability of AI generation.  Now what?
  3. Give them a second chance. This was my own contribution.  As noted above, AI checkers cannot usually determine a 100% probability.  Therefore, what I do when I get a paper with a high probability of AI generation is simply refuse to grade it, tell the student to rewrite it in their own words, and set a new deadline.  Some panelists were stricter, but I find that my approach tends to reduce student defensiveness and bypass academic appeals.  Students would rather rewrite a paper than face a time-consuming appeals process that they are likely to lose.
  4. Use AI as a teaching tool. One of my colleagues at Emporia State teachers her Journalism students by having them use AI to generate a paper and then having them write a paper themselves on the same topic. She then has them compare the two papers side by side and evaluate the strengths and weaknesses of each.  The AI-generated paper typically emerges as the weaker one, and the students can see how AI output compares with their own, authentically-written student papers.
  5. Be specific. As noted above, AI-generated papers tend to be vague and broad.  Thus, many of us are putting a greater emphasis on specificity.  For book reviews, my grading criteria now puts a stronger emphasis on specific quotes, page numbers, and concrete examples from the book.  Reviews that are just general overviews do not receive good grades.  Specificity is not AI’s strong suit.
  6. Send students to the writing center. Panelists stressed the value of offering students extra credit and other incentives to improve their own writing skills.  Some students may turn to AI not because they are lazy or fundamentally dishonest, but because they lack confidence in their own writing skills.  Helping them develop these skills reduces the incentive to turn to AI.
  7. Look for obvious mistakes. One panelist received a student paper that included the words, “my database hasn’t been updated since 2018.”  That was a particularly obvious giveaway!  As noted above, not all students turn to AI because of laziness, but those that do will often overlook these obvious tells when they cut and paste the AI output into a word processing document.

AI is here to stay.  Because of its dynamic learning, it will only get more sophisticated.  It is up to us professors to stay one step ahead.  The challenge is ongoing, but the strategies above demonstrate that there is no need to give up in exasperation.

Finally, as an example, I have entered the prompt “how should professors respond to students using AI to write their papers” into the popular AI generator, ChatGPT.  The unedited answer is below.  Compare and contrast this essay with my human-written one above. See if you can spot the tells of AI-written essays.

ChatGPT:

Professors should respond to students using AI to write their papers with a combination of vigilance, education, and appropriate consequences. Here are some steps they can take:

  1. Awareness and Monitoring: Professors should stay informed about advancements in AI writing tools and be vigilant for signs of their use in student work. They can use plagiarism detection software and other tools to identify suspicious content.
  2. Education: Professors can educate students about the ethical implications of using AI to write their papers. They should emphasize the importance of academic integrity, critical thinking, and the development of writing skills.
  3. Clear Expectations: Professors should clearly communicate their expectations regarding originality and the use of AI tools in assignments. They can include statements in their syllabi or assignment instructions explicitly prohibiting the use of AI-generated content.
  4. Detection and Consequences: If professors suspect that a student has used AI to write their paper, they should investigate the matter thoroughly. If plagiarism is confirmed, they should follow the established academic integrity policies of their institution, which may include penalties such as failing the assignment or the course.
  5. Offer Support: Professors should also consider offering support to students who may feel overwhelmed by academic demands or tempted to use AI to cope with their workload. They can provide resources for improving writing skills, time management techniques, and academic support services.
  6. Adaptation: As technology continues to evolve, professors may need to adapt their teaching methods and assessment strategies to address the challenges posed by AI-generated content. This might involve placing greater emphasis on tasks that require original thought and critical analysis rather than rote memorization or regurgitation of information.

Overall, the goal should be to foster a culture of academic integrity and responsible use of technology while providing support and guidance to students as they navigate the complexities of higher education.

 

About the Author

Michael A. Smith is a Professor of Political Science at Emporia State University.  He has authored or co-authored five books, the most recent of which is Reform and Reaction: The Arc of Modern Kansas Politics.  Co-edited by H. Edward Flentje, the book will be published later in 2024.  He has other academic publications as well, and also writes newspaper columns carried throughout Kansas as part of the Insight Kansas group and blogs for the MPSA. Michael appears occasionally on television and radio in Kansas and western Missouri to discuss state and national politics.  He was an expert witness for the plantiff in the Bednasek v Kobach case, decided together with Fish v Kobach by the federal district court for Kansas in 2018.  Michael teaches courses in American politics, state and local government, and political philosophy. He received his Ph.D. from the University of Missouri in 2000. Follow Michael on X (formerly known as Twitter).