Transparency in the Process
A guest article about involving students
I’m closing out the school year by slowing down to actually look back and make sense of what happened. It’s a metacognitive act that I, for one, could certainly do more of.
In this article, I’m inviting a few guests from the podcast episode “How Did It Actually Go?” (link below) to guest-write an article. Each of them agreed to go a little deeper in writing, reflecting on how process-based learning with AI actually played out in their schools.
In this article, Leon Lam reflects on building an AI chatbot for process-based learning and what he’d do differently next time. Reflecting back, what he found is at the center of this piece, so I won’t spoil it. But here’s a takeaway I want to reinforce: he assumed students would follow the process without ever explaining why. A process that remains invisible asks students to comply rather than learn. Adolescents need both to understand why we do something and how it helps them grow.
Let’s check in with Leon to hear his thoughts.
Intro
Leon Lam here, guest posting on Alex’s Substack. This will be an extension of what I discussed on the podcast episode “How Did It Actually Go?” I want to dive deeper into what I learned building an AI chatbot for process-based learning, and what I would do differently next time.
A lot of teachers use custom chatbots in their classrooms. I went a step further and built an entire platform for creating custom chatbots, mostly for analytics and data. I wanted to know whether or not the AI chatbots were making an impact. My hope is that what I learned can help you decide whether to invite an AI tutor into your classroom or to keep it outside.
What I Built
As mentioned in the podcast episode, my Socratic essay-writing bot coached students through Cambridge AS Economics 12-mark essays. It was structured around stages: question analysis, planning, and paragraph coaching.
In my bot, Alex’s Think, Generate, Edit process was built into each stage. Students had to think through the questions the AI gave, craft their own responses, which the AI gave feedback on according to preset criteria, and then they had to edit their work until it matched the criteria. The stages were important because they allowed students to plan first instead of jumping into the writing immediately.
What I Observed
I categorized student behavior into three patterns. The first group of students spent hours with the bot. They exchanged hundreds of messages. They followed the bot’s strictly enforced reply format. They complied with it for as long as it took to complete the essay on the platform. It looked like they were fully engaged, but upon deeper digging, I discovered that the chatbot I made was needlessly ruthless, and that the learning could’ve happened much more quickly if I had built in human touchpoints or relaxed the restrictions. Students had reported how cumbersome it was to get the exact answer the AI wanted.
The second group of students tried to game it for answers. The bot was designed to ask them questions, so they worked at getting around that. These students weren’t really learning anything. They tried prompt injection, off-topic detours, anything to extract an answer. They mostly failed, but time was wasted on trying to manipulate an AI instead of learning.
The third group of students did not engage with it at all. That was just not how they wanted to learn. In the end, the bot was just a chat interface. They wanted a teacher, and I saw their eyes light up when I took back the reins in the classroom.
Performance on summative assessments did not change since using the bot, either. Students who did well before continued to do well. Students who struggled before kept struggling. What did change was my ability to see the process. I could zoom in on student artifacts and query the data with an LLM, and that visibility was genuinely useful. So in that way, you might say the bot served as an assessment, which provided data that in many ways reinforced what I already knew about the students.
I suspect the issues came from two main reasons:
I did not name the thinking processes out loud with the students. I built it into the bot and assumed they would just follow along. Thinking back, I should have made them aware of the process so they know why I chose to set up the assignment this way. My thinking is that it would help the hacker group and the disengaged group to want to use the bot meaningfully.
I removed too much of myself from the process. In theory, the assignment should have worked. In practice, my students handed the entire feedback process over to an AI, and the efficiency I was chasing became the flaw. Aimée had named it before I did. She’d been a guest on the same podcast; when I heard her portion of the episode, her idea of “gates” mapped exactly onto what I’d been considering ever since I put the bot in front of students.
What I would do differently
I do believe that process-based learning is the way to go in the age of AI, but my next iteration of this assignment will definitely be different. Here’s how I will pivot moving forward:
I will reinsert myself in the feedback loop. AI should not give feedback on its own, no matter how well it is trained. Giving feedback to students is what builds trust and rapport; that’s the teacher’s job. There’s still a real role for AI, though. An AI can be trained to spot specific writing weaknesses and tag them to feedback I’ve already written, or to extend a comment of mine by pointing to a resource. The condition is that everything passes under my eyes before it reaches the student. The main point is that AI-use should reinforce things that we are learning and want to support in class.
I’m going to teach the process out loud. I will explicitly teach my students Think, Generate, Edit, and other processes, or co-create a process with them in class that suits the task. This time, they got a tool that already knew the answer to that question, and they were left to comply with it. However, if I had printed our process on paper and written underneath each step where and how AI was used, and why, my students would have understood the design from the inside and hopefully have been engaged with every step.
I’m changing how I grade. Because some students will focus too much on the final output, even if I ask for process artifacts, I won’t accept a finished essay unless the artifacts back it up. The artifacts will be graded, too, but the bulk of the grade for the final product will be awarded only if the final output was the natural product of the process.
I’m magnifying authentic, in-person assessment without neglecting AI literacy. I want students to share opinions that are actually theirs, in front of other humans, with their screens closed. That is often uncomfortable for them, and that is the point. My ideal classroom is one that maximizes original thought, critical thinking, and other capacities needed to interact effectively with AI, which I am making a focus outside of the classroom. Students will be instructed to interact with AI without my supervision. This means I will need to teach AI literacy, so my students can remain thoughtful and responsible.
I’m still building, but smarter. I built a platform that made students learn through Socratic chatbots, a workflow that’s still new and unproven in their minds, piled on top of everything else they already had to do. So the next iteration starts from what students already do instead of inventing something foreign. I rewrote the entire textbook with AI for accessibility, same flow through the topics, but with simpler, more direct wording. I put MCQs inline for formative checks, and digitized over 3,500 past-paper MCQs organized by topic so students can set up their own mock tests, complete with explanations for the wrong answers. Because I own the platform, I’ll know exactly which topics a class struggled with. So I will keep building my own platform and iterate on the processes students already go through, improving their learning and my teaching at the same time.
Conclusion
Thinking back to my own time in schooling, I don’t remember which teachers had the coolest PowerPoints or used the latest gadgets. I remember the chance to question alongside other students, the jokes, the moments of wonder and care. I miss watching students struggle through the thinking, take pride in what they made, and own their learning. With the changes I’ve shared, that’s the classroom I want to build in this AI-enabled world.
Monday Ready Resources
One of the most important realizations from this experience is that I need to talk to students along the way about the process or even co-design it with them, as well as teach AI literacy. Here are three ways you can get started with your students to make a process or build AI literacy:
1. Go to Alex’s AI Enhanced Process generator and take a crack at making your own process on your own or with your class.
2. Take Anthropic’s free course on AI Fluency: Framework & Foundations. This can also be done on your own or with your students.
3. Use my Process-Based AI Use Scale which you can post around the room, or use with students to determine how much AI should be used in each specific step of the process.
AI Disclosure
I wrote this whole article, and then Claude was consulted on for wording, structure and flow. Some of Claude’s suggestions made it to the final version. Some suggestions inspired other original changes. Ultimately, the words are entirely my own and represent my opinion. Aside from the screenshots of my application, I gave ChatGPT the final version of this article, asked for image suggestions, and asked it to craft the prompt that generated the images you see in the article.
Images generated by ChatGPT and Gemini.