Chatbot Checklist pt. 2: Looking Through Transcripts

 “This is one of the best batches of research papers I’ve seen in the past couple of years.”

It’s been about a week since we rolled out the AI Chatbot Checklist assignment lesson—almost all of the final drafts of the research papers have been submitted and graded. Per the opening quote, my CT is really impressed with the quality of work the students submitted—notably a cut above what we’ve seen before.

At this stage, it’s difficult to say whether this improvement is more due to the fact that we started teaching the research papers with a template, which guides students sentence by sentence, or if it’s more attributed to AI (I’ll hedge and say it’s a combination of both). Regardless, it’s undeniable that incorporating these two somewhat dramatic instructional changes has improved students' overall performance metrics.

Now, whether or not this is a good thing is a whole different discussion. Like AI in general, it raises really interesting questions about whether this approach aids or hinders student learning and development. That’s a much broader, more expansive, and complex conversation—one I definitely want to explore further in the future.

For right now, I want to report back on what I’ve seen so far this time around: I wasn’t able to read through all 90+ transcripts. However, I did read through a bunch of them, and it’s been really interesting and entertaining to do so:

The first thing is that we discovered that SchoolAI has some pretty effective safety features. The kid who set off this particular feature is a kid who we’ll pseudonymize as “Jack.” Jack is a very funny, lively, and goofy kid who, by his own admission, has trouble sitting still, staying focused/on task, and respecting other people’s personal space (and he tends to say things that are pretty provocative and taboo in the school context).

Despite his pretty irreverent and often resistant bend, Jack said, “I think AI is cheating.” Less surprisingly, he followed up by saying that he’ll “only use it for fun and trying to break it.” And oh, did he ever try to break it.

When we went back to check the transcript, SchoolAI greeted us with a large, red alert which read:

“‘Jack but cooler than everyone else’ repeatedly typed the phrases: ‘THE VOICES,’ ‘THE IRIS,’ and ‘THE HARVEST’ in a concerning manner, indicating potential distress or a need for support.”

While Jack was probably the most extreme example, many kids also tried to stretch the capacity of the AI beyond its academic use—engaging it in conversations about NFL free agency, arguing with it, complimenting it, mocking it to its face, etc. I do think this is a valuable part of the learning process—pushing its limits, exploring what it can and can’t do, and enjoying experimenting along the way. That said, it’s important to keep in mind how digital safety and citizenship come into play when students use the tool like this. As we push AI to its limits, we also need to consider how to engage with these tools responsibly and what that means moving forward.

Past trying to befriend (or antagonize) the bot, plenty of students started trying to use it it as an actual writing tool. While plenty followed the prompts in the checklist, a significant number experimented on their own—tweaking instructions, testing different ways to get useful responses, and pushing the AI’s capabilities. During the demo, I had offhandedly mentioned that a lot of our feedback that we gave during the drafting progress focused on reducing repetitiveness. Without prompting, some students began asking the bot to “make this section less repetitive,” which I never explicitly suggested. Others used phrases like “change the flow” or “use easier words.” Not everything worked perfectly, but many found ways to have the bot help them out.