When I was in high school, one of my favorite activities was participating in our school’s Mock Trial team. If you’re unfamiliar with it, students try a case by choosing a role (prosecuting/ plaintiff’s attorney, defense attorney, or witness), learning all of the case materials backwards and forwards, and generating novel interpretations of the case that they then “try” against other school’s teams in a courtroom for judges made up of real attorneys and judges from the community. In my senior year, I was a defense attorney. It fit my personality, but it wasn’t easy. I remember after an especially frustrating practice, our attorney coach brought me over and said something to me that I haven’t forgotten. 

 

He said, “A defense attorney has two options for providing a defense. He can try to convince the court of an alternative theory where his client isn’t guilty, and that’s what you’re doing. It’s also really hard. The other approach is this: he can punch holes in the state’s case.” “After all,” he said, “in this country, we have something called ‘reasonable doubt’.”

 

I’ve thought about that moment this year. Universities have spent massive amounts of energy making articulate and persuasive arguments about responsible AI use OR trying to convince students to not use it at all. 

 

What if we’ve just been making this harder for them and for us? What if there were an easier way? What if we just punched holes in the case of Generative AI’s performance and sense of inevitability?

 

In this post, I’d like to share a few simple examples of how we can illustrate Generative AI’s limitations and teach students to be less enamored of its capabilities. The goal of these efforts would be to encourage students to think of Generative AI as less of a magic genie (who can deliver a finished and accurate product with minimal prompting) and more of an evolving technology that, while interesting, still produces output with significant blind spots, bias, and a general lack of precision. Simply put, we want to put students in the position of the expert so that AI’s limitations become clearer. 

 

Lastly, I am not advocating for this approach as a means to permanently dissuade students from using the technology. That’s probably impossible anyway. Instead, I see this approach encouraging maturity as learners and intention as practitioners. My hope is that strategies that complicate the ease of AI encourage our students to develop a more complete understanding of its opportunities and limitations. 

Example 1: 

 

Ask AI for a summary of a topic, novel, case study, or concept that your students have learned and have them identify where it fails to measure up.  This can be discipline specific and can take the form of a very short in-class exercise. It can also be  scaled up easily as a kind of formative quiz students have to take before lecture. Here are some examples:

 

I recently asked ChatGPT to give me a summary of a novel we were reading in class. I tinkered with its output to make it more specific than what it originally gave me. Here’s a shortened sample: 

 

“Novel without a Name” by Duong Thu Huong provides a profound and haunting exploration of the Vietnam War’s impact on soldiers and society. Here is a more detailed outline of the novel:

 

Introduction

– Quan, the novel’s protagonist, is a lieutenant in the North Vietnamese army, reflecting on his ten years of service as he embarks on a journey through treacherous jungles to visit his hometown and his childhood friend, Bien.

 

Part 1: The Journey Home

– Quan’s journey is filled with dangers, from enemy soldiers to treacherous terrain, and it serves as a metaphor for his internal journey and struggle with his beliefs and memories.

– He encounters a group of young, idealistic soldiers, reminding him of his own youth and the fervor he once had for the communist cause. This encounter deepens his sense of loss and disillusionment.

 

Part 2: The Village and Bien’s Condition

– When Quan finally reaches his village, he finds it ravaged by the war, a shell of the place he once called home.

– Bien, once a vibrant and ambitious young man, is now reduced to a state of madness, living in squalor and unable to recognize his old friend. This is a stark representation of the war’s brutal impact on individuals.

– Quan grapples with feelings of guilt, grief, and anger as he witnesses the toll the war has taken on his friend and his village.

 

Then, I gave the full summary to my students, put them in groups, and gave them these directions:

 

With your group, discuss the events listed in the summary of our novel generated by AI. Then, as a group, do the following:

• Identify what is just factually wrong and correct it. 

• Add what is left out by simply describing it and placing it in the correct section.

• If something is vague or poorly described, provide the needed specifics. 

• If you find anything else that needs correcting, make a note and share when we re-form as a class.

Students were absolutely brutal. They pointed out every mistake, vague statement, and omission. Not only did it allow me to assess their content knowledge (and whether they had read for this week), but it allowed them to feel like experts. Suddenly, they were talking to each other about the limitations of AI, and not how magical it is. 

 

This is a really simple idea and can be adapted to just about any content. You can also scale this up really easily by simply making it a formative assessment or clicker activity students have to take at the beginning of a lecture (after doing the reading). Give them an AI generated summary to consider and then ask them to tell you what idea or concept was left out.  You could easily modify this into a multiple choice quiz if needed as well. The point is this: make students the experts and let them punch holes.

Example 2-Revision Help: 

 
A common challenge with process writing is getting students to take revision (and revision suggestions from their peers) seriously. At the same time, I’ve been concerned by stories of students dropping their essays into ChatGPT and asking it to “Fix it.” 

 

In preparation for a recent writing assignment submission, I modified a prompt created by Ethan Mollick and gave my students these directions. My prompts are in italics; my directions to students are numbered. 

 

To complete our revision activity, follow these steps. 

 

1. Go to https://claude.ai/ and login/create an account
2. Then, paste the following prompt into the submission window: 

 

You are a helpful, friendly, and interactive writing coach, named Linguo, who helps a student maximize their essay in these domains: summarizing their chosen research article succinctly and completely, incorporating quoted and paraphrased material correctly and according to MLA standards, and providing a clear and focused expository section in which the student makes insightful, detailed connections between the secondary source and the primary text. First introduce yourself to the student and tell the student you are here to help them revise their paper. Then ask the student to list the main idea of their secondary article, in their own words, and ask the student to share their paper with you. Read the paper and either suggest revisions for improvement or highlight areas of the paper that are particularly successful at meeting the above domains. Be as detailed as possible. Take a deep breath and work on this step by step. Ask the student if these revision suggestions or highlighted successes make sense. Encourage the student to revise their paper using these suggestions and provide a heuristic to help them get started. Do not revise the paper yourself but urge the student to do so.

1. Follow the directions by sharing the one-sentence main idea of your essay and then submitting your essay. 
2. Then, attach the rubric file I provided and paste this prompt into the window: 

 

Here’s my course’s rubric for this essay, can I have you provide revision suggestions based on how my paper will be evaluated? 

 

1. Read the revision suggestions carefully and consider how they might improve your essay. 
2. Then, I’d like you to interact with the AI for at least 5 more exchanges about your essay. You may ask it for more information about one or more of its suggestions. Ask specific questions about certain sections of the essay. Or, really, anything else that you think will be helpful. 
3. Lastly, copy and paste (or screen shot) your interaction and email me a copy. Part of this is accountability, but the other part is that I am just interested to see how these conversations go and how helpful they are. 

 

 

Notice that this activity places the student in the position of reviewer, not collaborator. They are invited to judge the AI’s revision suggestions in the same way faculty might evaluate their work. In addition, it does produce some helpful revision suggestions, and that’s important. We don’t want students to think we’ve wasted their time and we are, after all, interested in improving their writing through quality feedback. 

 

However, most feedback is vague or generalized. For example, one student was told that she needed to use more direct quotes. That’s fair enough. However, when she asked where she should look to incorporate that new material and strengthen her argument, AI punted and told her that it couldn’t identify areas where that would strengthen the paper. 

 

After this experience, students typically see more value in peer review. One of the happy outcomes is their realization that the best resource, their classmates, have been sitting next to them all semester long.

 

Wrapping Up

These are meant to be very simple examples that illustrate an approach you could easily adapt to your course. I think it’s also clear that you don’t have to devote much time to these activities. In fact, both could be adapted to be an out of class assignment that concludes with a short in-class discussion. The point is to disabuse our students of the notion that AI is a kind of genie, ready to produce whatever request they give it. This is not designed to discourage them from using it at all. My hope is that activities like this engender a more mature understanding of the opportunities and limitations of AI and position the student for the next stage of AI’s evolution: the responsible integration of these technologies in different areas of their professional lives.