EP 40 - Does Allowing AI Reduce AI? A Surprising Finding with Dr. Megan Haselschwerdt

In EP 40, John and Jason talk with Megan Haselschwerdt about her transformative semester moving from a futile "cat-and-mouse" game of AI detection to a trust-based partnership with students, demonstrating how transparent dialogue, a more open policy, and addressing "insurmountable" assignment loads are far more effective than policing.

See complete notes and transcripts at www.onlinelearningpodcast.com

Join Our LinkedIn Group - *Online Learning Podcast (Also feel free to connect with John and Jason at LinkedIn too)*

Guest Bio:

Megan Haselschwerdt, Ph.D., serves as an Associate Professor and HDFS Graduate Program Director in Human Development and Family Science at the University of Tennessee, Knoxville. She earned her Ph.D. and M.S. in Human and Community Development from the University of Illinois-Urbana Champaign and a B.S. in Psychology from Indiana University-Bloomington. As an interpersonal violence and family science scholar, Dr. Haselschwerdt’s research focuses on intimate partner violence (IPV) from the perspectives of victimized adults, young adults with childhood exposure, and support professionals. Specializing in qualitative methodologies like grounded theory and reflexive thematic analysis, she also collaborates on mixed-methods studies to examine help-seeking behaviors and develop trauma-informed interventions. She currently directs the Family Violence Across the Lifespan research team, leading initiatives such as the REVEAL Project and the Young Adults Live and Learn Project to promote safety, healing, and justice.

Resources:

• Dr. Haselschwerdt’s Scholarship: https://scholar.google.com/citations?hl=en&user=hTSsBcQAAAAJ&inst=9897619243961157265
• Megan’s AI Use Policies: https://docs.google.com/document/d/1I2THuGIaKYstGyZylQS4FgvbNV0B1j3Q3hEpGI4K_l4/edit?usp=sharing
• Jason’s AI Policy (and free / open source icons for communication) https://docs.google.com/document/d/1MG9h68__uqPSz6HXNeVymJhal1VNapjyK-2PFa5QFxI/edit?usp=sharing

Theme Music: Pumped by RoccoW is licensed under an Attribution-NonCommercial License.

Transcript

We use a combination of computer-generated transcriptions and human editing. Please check with the recorded file before quoting anything. Please check with us if you have any questions or can help with any corrections!

 

[00:00:00] Jason: I'm going to butcher your last name probably

[00:00:02] Megan: Oh yeah, that's okay. We do too.

[00:00:03] Jason: us with that

[00:00:05] Megan: Yeah. So, we say it all very differently in our family, so it's totally fine. I, I say Haselschwerdt as though there's a z Yeah. Other people, other family members say hassle, but yeah, I say it as though there's a z Hazel Schwart.

[00:00:16] John Nash: Okay, cool. I was,

[00:00:18] Jason: Hazel Schwart

[00:00:20] Megan: Mm-hmm.

[00:00:21] John Nash: I took German for years and so I love the sound of German, so I Yeah. I said,

[00:00:27] Megan: Yes. Yeah, we, we.

[00:00:29] John Nash: be totally German of

[00:00:30] Megan: We offend Germans when I say, like, what, how we say our last name? Yeah. It's unrecognizable. Yep.

[00:00:36] John Nash: Nice shirt. Yeah. Okay, cool.

[00:00:40] Jason: Haz-el-Schwart

[00:00:41] John Nash: Haselschwerdt,

[00:00:42] Jason: You

[00:00:42] Megan: Yep. That's totally fine.

[00:00:44] John Nash: Okay. Haselschwerdt.

[00:00:45] Megan: I think Jo, uh, John has it the authentic way and Jason's American butchered from Ellis Island Way works just well too.

[00:00:53] John Nash: I can't, yeah, I can't help myself though. So

[00:00:55] Jason: I can't help myself either

[00:00:57] Megan: ha ha...

Intro

[00:00:58] John Nash: I'm John Nash here with Jason Johnston.

[00:01:01] Jason: Hey John Hey everyone. And this is Online Learning the second half the Online Learning Podcast

[00:01:06] John Nash: Yeah, we're doing this podcast to let you in on a conversation we've been having for the last three years about online education. Look, online learning has had its chance to be great, and some of it is, but a lot of it still has a ways to go. How are we going to get to the next stage? Jason?

[00:01:22] Jason: That's a great question How about we do a podcast and talk about it

[00:01:26] John Nash: love that idea. What do you want to talk about today? I.

[00:01:29] Jason: today I don't know how many times I can do this joke but how about we talk a little bit about AI and education

[00:01:34] John Nash: Wait, is that a thing now?

[00:01:36] Jason: Yeah

[00:01:37] John Nash: I'm willing to try.

[00:01:38] Jason: Yeah, I don't know much about it seems like everything I learn about it then I unlearn about it as well. But I'm really interested today We've got a colleague here one of my colleagues from University of Tennessee Knoxville with us Megan and we are just talking about me butchering her last name And so I'm just going to let her introduce herself Megan

[00:01:58] Megan: Hello, I'm Megan Haselschwerdt

[00:02:00] Jason: Welcome to the podcast It's so great to have you here and tell us a little bit about what do you teach what do you do at University of Tennessee Knoxville

[00:02:08] Megan: So, I'm an associate professor in the Department of Counseling, Human Development and Family Science. I'm in the Human Development and Family Science side of things, and I'm the Director of Graduate Studies. Outside of teaching, I'm an intimate partner violence researcher, family violence researcher. But in the context of teaching at the undergraduate level, I teach HDFS 385, which is Child and Family Diversity. I teach this course at the graduate level, and I also teach some qualitative research methods, family theory, that sort of thing. But for undergrads I've been teaching Child and Family Diversity, fully in person, hybrid, kind of split between in person and asynchronous online.

this semester is our first time with the fully launched distance ed online course.

[00:02:50] Jason: That's great we really got talking because of really my role at University of Tennessee which is in more of a centralized office And I was included in a conversation where we started talking a little bit about AI the classroom And first could you give a little bit of a context about how long the course that we're talking about and then how long you had been teaching that course.

[00:03:16] Megan: Absolutely. So, I've taught this class since fall of 2020. And so, it's had the, I can't even keep track of the number of modalities I've taught it in since 2020, which I think some people don't realize if they haven't taught between multiple modalities, that really, you're ending up creating a new class multiple time

[00:03:35] Jason: Yeah

[00:03:36] Megan: Things don't translate. And I, that's part of the learning this semester is what translates well from even hybrid into fully online asynchronous. So, this course is a semester long class. It's a three-credit hour course. And this is my first time teaching it this fall 2025 in this current form.

it's mostly traditional brick and mortar students. It's our new distance ed program. So only five of about our 85 are fully online. The other 75 or so are, traditional students who either opted into the online version or were registered for an in-person version that we ended up canceling and merging into my course.

[00:04:16] Jason: Interesting. I wonder about how many of our classes and programs will shift into more that digital doesn't necessarily mean distance

[00:04:25] Megan: yeah.

[00:04:26] Jason: know that as students are taking these online that as we flex modalities you talked about the modality shift of your classes. Thinking about the number of modalities that they have taken to go through an undergrad Then potentially a graduate program

[00:04:41] Megan: And it's been interesting, our students, we collected data, coming out of, the early parts of the pandemic at least. And what we found is that students really wanted more in-person classes. they said they felt frustrated that they'd had mostly online courses.

And so, we tried to make sure we were always offering a good balance. But when we offer an online and an in-person in the same semester, the in-person course is under-enrolled. We think that maybe this is specific more to juniors and seniors who have established their social networks, they have work, they have volunteering, they have just a lot of complicated demands, and so maybe for them online is more appealing.

But we've also heard from students how difficult it is to, if you have an online class and an in-person, like racing out of the in-person to find a place where they can do the online, especially if it's a synchronous online class.

So, the data that we get does not align with students' registration decisions, and that's made things a little complicated.

[00:05:36] Jason: Yeah, so getting into this class, I think that's the wonderful context and I love the fact that your program is thinking about this and being thoughtful about the student experience. Thinking about this class you've been teaching it for five years or so tell us I really want to get into the story of this and

[00:05:52] Megan: Yeah.

[00:05:53] Jason: what has happened this semester Tell us a little bit about why you reached out to our department initially.

[00:06:00] Megan: So, when I was building this class, so I've historically always taken you know, University of Tennessee provides us with an AI kind of general policy syllabus statement of, you know, open access, moderate use AI or like no AI. And I've always historically done the moderate use.

However, I have not taught undergrads in the AI era until this semester. I'd only taught graduate students. And so, this semester went into it with my syllabus, decided because it's fully an online, the smartest thing I could do just my own thinking, not from literature was I should say no AI use because I really want this class to be reflective. I want it to be drawing upon their personal lived experiences. I want it to be. And so I went back on a policy, like kind of how I'd approach things in the past. And so there was a blanket no AI use. We made clear as best as we could through our syllabus, through our videos, through all the onboarding that we're not grading for grammar. We're not grading for your opinion. We're not grading for you to agree with the content. It doesn't need to be polished. And I think we naively thought that through these engagements and how we wrote about it and how we talked about it, that students would feel really comfortable trusting that we were really comfortable with the unpolished version of their written selves.

And so that's where we started. I have a 20 hour a week graduate teaching assistant who was invaluable to this class. And in the beginning, we both were grading and taking note of things. because I had taught this class hybrid in fall of 2023, before AI really became, like in our class, we, I think it, we hadn't seen as much in our undergrad class of around how AI was going to be utilized. I have all these examples of how students responded to the same kind of prompt.

And so, in this class, we do a premodule reflection just to kind of get them, it's supposed to be like a 10-minute brain dump. We make; we try to make it as low stakes. We suggest you set a timer for 10 minutes and you just kind of brain dump on the topic. So, if our topic is going to be around foundations like, kind of like white supremacy, the foundations of like, on which the country was founded. We have a prompt that's like ", tell us the story that you remember learning of the founding of the United States focusing on pilgrims, Indians. Like, kind of like the, the prompt kind of reckons back to like, what were you taught about this in elementary school?

And so, we're used to getting very specific, like I remember this time. Mr. Johnson's second grade class, you could tell just like real examples from their childhood.

But what we noticed early on, this was like week two or week three of the class that we were getting like almost the exact same thing. And told in the same style and told with the same phrasing and using words that we hadn't put in the prompt, but like we're very generic and just, and you know, if you saw one or two, that's one thing, but all of a sudden we're like at 60, 65 of the 80 or you know, and so then we went down this rabbit hole using AI ourselves to see like to what extent does this seem like we, is it, how is, how likely is it that everybody just has the same sort of thing to say? Then I went back to my, you know, the 2023 responses and realized no one of that year said anything like these. So, we watched this for a little bit, and then the final straw I think was at the end of that week.

It was like, if there's a quote from one of the documentaries, you know, like "America is sold as this melting pot, but like, is it really?" And so, I'm like, so if America's not a melting pot, like. What is it? And it was like one third --the TA started calculating, right? So now we are spending ample amount of time trying to figure out how much of a problem this is.

It was like one third of them described it as "a salad plate" or "a salad bowl" in quotes. And I never had that response. And it was all of, and it was like in quotes, and it was just it, you know? Then I went back to 2021. Nobody else talked about a salad plate or a salad bowl, so that's when we started to feel like, honestly, like in our, like we felt like hostile towards the students.

Like we felt just because also with building an online class. I don't think I understood, like truly, and you know, we've done survival online teaching during the pandemic, but having gone through Jumpstart, I don't think I, you know, it is so important and so kind time consuming, especially that first time.

So, I'm like, I am spending all this time trying to create this exciting and engaging and thoughtful course and my TA's working so hard, and so we were just like, but it looks like they're just slopping everything into ChatGPT, and they're not even engaging it. And so, we felt like, I think hostile would be a fair way to describe it.

And again, since we're not interacting with them face to face, like we don't see them as humans and it feels like they don't see us, they don't, they're not respecting our time. and so that's when I was like, okay, we need to do something different. And that's when I reached out to digital learning and TLI.

What do we do here? Like, this isn't working. during, fall 2021 we had an experience where I made a small tweak to the assignment. I took out a final exam and made it a reflection paper. And that unraveled some students, that change.

Just like any change, just some students were just like, the syllabus says this, how could you, you know? And so, I've been afraid of making substantial changes in an undergraduate course during the semester thinking, you know, people need stability, they need the same, making a big change could cause, you know, even more disruption.

But we had the idea of doing a survey. So, we did a survey. It was instead of one of their normal assignments, the assignment was to complete this survey, and we asked them about their AI use. We asked them just general questions about how they felt the class was going, their trust in us.

Usually what I would do at a mid-semester point, this was a quarter semester at this point, check in, how they feel about the content. is the content the impediment? and so what came out through the written and then the more open-ended responses, was that it was anonymous.

That's right. 'Cause it was, are you using it? How, and if so, how are you using it? You know, the blanket statement says no use, but how, okay. Clearly, it's being used. So, if so, how are you using it and why? And that's when it got very interesting. So, most of them said they were using it to refine what they wrote.

Some of them said they were using it for brainstorming, very few anonymously, acknowledged that they were fully, just plugging in the prompt and, getting something out. But most said that they were using it, I think almost nearly all said that they were using it in some way and the reasons why is what really surprised us.

So, most of them said I acknowledge that you said we didn't have to pay attention to grammar, that kind of thing. They were deeply afraid of being judged for their writing style, and they were very uncomfortable and not fully trusting in how we would receive their more unpolished writing because it really was in these pre and post module reflections that are supposed to be kind of ugly and not pretty, not written perfectly, where we saw the most pristine writing.

So, though we thought we did enough to build up trust and mutual respect and like a, a recognition that we would accept their work as it was, they didn't feel that yet, and they were just also afraid. And then there was a group work component to things each week, and they were really concerned that they would come across as uninformed or they would be unintentionally offensive or that they wouldn't know the right words.

And so, we talked about that a lot in the beginning through our videos and things, but that didn't assuage

their anxieties as we thought it would. And so, and they were also anxious that they could be authentic and that we wouldn't penalize them for their questions or concerns.

[00:13:48] John Nash; It's fascinating. I'm hanging on every word.

[00:13:50] Megan: Good.

[00:13:51] John Nash: it's like a drama story unfolding.

[00:13:53] Megan: Yeah.

[00:13:54] Jason: I'm really impressed by the fact that you reached out that you recognized maybe what was going on And I think that was one of my first questions to you was about how do you know and I think that made a lot of sense and partly because of your experience in this class that you're able to recognize it So I'm really impressed that you reached out that you recognized it. But then I think really impressed as well that you then followed up with that survey to try to explore the why of this and not just knee jerk reaction. Even though I appreciate you talking about this feeling of hostility almost this lack of human connection I appreciate you being transparent about that because I think this is very common I appreciated the way you followed up with the survey to get more information to get into the whys of why this is happening and it makes a lot of sense to me

[00:14:42] Megan: Can I share one other example in I’m trying to remember back our levels of frustration. So, I'm teaching a graduate course at the same time. And so, I've been bringing this to them as a discussion, like, okay, I'm teaching professional development class for our first-year PhD students. a suggestion from a graduate student who was on the conduct board at a different university, said that when we were on conduct board, what we discovered, the only thing faculty could do at that time a couple years ago to prove that there really was like cheating, was by adding a Trojan horse into the prompt.

And to see if you were just copied and pasting how this would play out. So, the Trojan horse was like, and so I this, we did do this, and it was kind of funny, but that's when we realized we'd kind of crossed over into like, we need help because we're being, we feel too hostile.

So, I started adding, I actually used AI to help me create a Trojan horse for a couple different assignments where they came up with a phrase.

So what we did is you had the prompt in canvas, and then there was this line afterwards that was in white font, so they couldn't see it, and in tiny font that said, in your answer, make sure you address x, something completely unrelated to the topic, something you would never have had in the class. And so, when if they copy and pasted it right into chat, GPT, the chat GPT response would engage around some doctrine or something that was completely made up and you couldn't Google.

It doesn't exist. And we found that a handful or plus of students who were not paying attention to even their prompt and to chat GPT, were just using this made-up thing.

I'm so confused. I did use this prompt, but when I copied it into my Word document to work on it, this part came up and I thought, oh, I just missed that doctrine in the documentary series. And so, the Trojan horse did not work 'cause this student was right, like we could see she walked through the logic of it.

And so sure, maybe it did catch people who were inauthentically. But we tried a couple of different things that did not work in AI detection, and that's what led to the reach out, the survey and ultimately realizing that any detection effort is going to be a dangerous rabbit hole that's going to lose your sanity And it actually is not truly an accurate representation necessarily of AI use.

[00:16:49] Jason: Did you try to use any other AI detectors

[00:16:53] Megan: Yes. We used some of the different, I can't remember their different names, a couple AI detectors and they were okay. But actually, we spent, I ended up purchasing a subscription. I overused my limit with chat, GPT. And so, I ended up purchasing, I would say in total, we probably spent me and the TA maybe collectively 10 to 15 hours trying to get to the bottom of the extent to which this was a widespread problem because we felt like it was just had hijacked learning in the class. So, we used ChatGPT lot and actually it was very helpful. But at the end of the day, it couldn't do anything to help us.

Individually, I had no interest in taking 60 students to student conduct board, especially when realizing that like this was going to yield nothing. And so, it was a fruitless effort of using the AI detection tools. But we did, we spend way more time than we should have tried to deal with that.

[00:17:49] John Nash: Megan, it sounds as though there was a point then where you and your TA sort of collectively threw up your arms and so you, you consulted Jason shop and you went in a direction.

And before you tell us that story, I want to talk a little bit, if it's okay with you, about what you think you learned right to that point. Was it because we've talked to different people and I've sort of consulted my own thinking around how assignments should be made in the effort to try to sort of catch students and have them, and I'm going to use words carefully here, but I understand how you might feel resentment towards the students for not taking the work seriously. And, and in some ways, I've even felt it, you say, I need them to bend to my will because this course is important, and I've spent a lot of time structuring this. And so, I need them to, to you know,

[00:18:42] Megan: adhere

[00:18:43] John Nash: to the standards that I'm sitting here. And then at some point. you realized maybe I'm not structuring it the right way, and maybe I

[00:18:51] Megan: Yep.

[00:18:51] John Nash: rethink this, these strict notions I have of how they must bend to my will.

That the learning outcomes could still be achieved. But what I'm doing now is not going to, because now I'm going down this rabbit hole.

[00:19:03] Megan: Yep.

[00:19:04] John Nash: is that fair?

[00:19:05] Megan: I think that through this and the data that we obtained, I realized also Jason, I can't remember if it was you, somebody sent me that tool for calculating how much time was being spent in the class based on the number of pages, assigned videos, that kind of thing.

[00:19:17] Jason: one of our instructional designers

[00:19:19] Megan: Oh, that's right. Yes. Yep, that's right. Who was my digital learning coach this summer? What I realized with that, 'cause one of the things that came up in the survey was that the one other reason they were using AI was because of the volume of content that they were receiving.

I, when I re, when I did the calculate, my assessment of how much time was spent on the class was probably half of the reality. You know? So, if I was thinking this would take them four to six hours when I plugged in some early weeks, we were getting in the six to eight plus hours, which technically according to university standards, that falls within whatever.

But it was, it was a lot. It was, it was more. And I think that part of that is the issue of teaching, like. The diversity class. Not that we don't have people interweaving diverse ways of thinking and doing in families and children across other courses, but in this class, I do feel like a very strong obligation to like to ensure that, if nothing else, these students leave our program without doing harm to others. And I feel like this is like the one shot I might have, you know, to really help that. And I, that's not really a fair depiction of like our courses or curriculum, but I think because of that I do tend to have too much in there.

And so, I think that that did help us reflect on how much we were assigning, how many assignments we were assigning. Also, that we did module zero, like the introduction and module one in the same week. And next time we will devote a whole week to the module zero and onboarding to the class. And I think that that actually will make a big difference in terms of setting the tone to help with some of the trust pieces.

There was a reflection on it wasn't just them, it was us, and that we had to make some pivots in this class, and then some that we couldn't control, like how we onboarded the class in the future.

[00:21:08] John Nash: I feel like it was new to you that they, that you naively thought that they would trust you out of the box. I wonder in 2023 and 21, did you feel like they trusted you or did something shift?

[00:21:21] Megan: I think that, so interestingly prior to 2021 when we were in person masks off. I am able to. Engage with students in a way where they see my humanity and we can get to know each other even if we don't know each other quite well. 2021 was when we were in person, but in masks.

and also online, I can build rapport, I can build that trust. But 2021, I was really surprised when we were wearing a mask how. It was very hard to build rapport with them and vice versa. They couldn't see when we're talking about hard things, they don't see me smiling.

They don't see. And so, I actually felt 2021 of all the semesters I've taught was the most challenging for them seeing me as a human, as an instructor. The reason I could tell this is because I was pregnant for the second time in that semester. The first time I was pregnant in 2019.

I had to stop the students like kindly from like their questions and curiosities and excitement surrounding my pregnancy. We teach mostly women, young women who work with children. So, there was a lot of, in curiosity in 2021. When I had to explain that I was pregnant 'cause there was a day we missed 'cause I was unwell and it was, I was becoming clearly more pregnant.

There was no human connection with me at all around the pregnancy. And that was fine. It just was a noticeable difference. Like they did not see the masks did hinder them from seeing me as human. So, I was mindful of that. So, 2023, it was fine. Again, normal back in person. Great. So, I am still kind of always mindful of that 2021, where how the class is perceived when they don't see the instructor who's teaching con content that challenges them personally as a human.

And so, I think sometimes when you've gathered a lot of respect from students and trust, I forget that I forgot this semester was a whole new group. They don't know me. They have no connection to me. There's no reason why they should trust me.

This is a supercharged political, historical moment. I think that I overestimated students' ability to trust me based on what I said in a video or on the syllabus.

[00:23:22] John Nash: Yeah. Nice. Thank you.

[00:23:24] Jason: Yeah, pick up the story So you've

[00:23:26] Megan: Yeah.

[00:23:27] Jason: For those listening we're in the fall of 2025 for those listening in the future. Actually everybody will be listening in the future but we're in the fall of 2025 and so you've come to some of these realizations some of these design changes that perhaps you want to make and you've decided that you're going to step ahead with your TA to try to make some of these changes At what point in the semester do you start to make this pivot

[00:23:53] Megan: I think that this was during week four is when we did the quarter semester check-in. And I substituted out an assignment. So, this was a full a full assignment shift.

and we started making the changes immediately. So, you know, I think module five, like week five had already been loaded. And so, they were already underway, we explained the changes, and I think it started in module six. So, you know, it was pretty quick, but it wasn't immediate, but it was as quick as we could in terms of uploading the next content.

The first thing we did was explain the new AI policy shift and, you know, Jason kindly sent me what he uses in his syllabus And so I modified that for my class of, you know, human first and human last. You know, we want your brain doing the work, but we're here to figure out how we can use AI to kind of help you.

Further your thoughts, further your work, and so on and so forth. And so that's when we implemented that. We implemented it like the icons above each different assignment. A icon of like, no, like there are certain things like quizzes where you, like, they, like, we're not supposed to, it would be pretty hard to just the timeline and things, but like other assignments where you can, you could, and that they were supposed to utilize a statement of how they utilized it.

I would say some utilized that statement throughout, others didn't, and we didn't really have a policy in place where you were penalized for not doing so. But that's something to kind of think about in the future. But pretty early on in this semester we made this big shift in our policy and its implications for the rest of the semester.

That seems really quick to me to be able to pivot so good on you and your TA to move that quickly You know, at the end of the day, the TA is the one ultimately at this point, who's grading everything. And I know that she agreed that we needed to make the pivot. I think that it was a continued tension for her throughout this semester. And we had talked about it some, because I think.

Even though we, saw some substantial improvements. It did still feel hard to feel hindered in like what really could be done when it was clear that a student or two or a couple students were really misusing now this flexibility. So even though it's kind of like the one bad evaluation, everything's great, you're kind of focused on the one.

I think that that still was hard. So, I will say it, for me, it's easier for me to feel like, yes, great progress. 'Cause I didn't see, the extent to which the TA was subjected to kind of this shift still in what we're expecting student writing to look like.

[00:26:23] John Nash: but you saw some kind of shift that went in a better direction than you expected or a positive

[00:26:28] Megan: Yeah,

[00:26:29] John Nash: that?

[00:26:29] Megan: We actually saw, it almost seems like students started using AI less. On these reflections, we started to see specific examples. We had a couple students meet with the TA to talk about, like, we realized that many of them did not understand what we meant by personal examples.

And so, a student who would give feedback like this doesn't show authentic student voice came and met with a TA and as the student's talking our TA's, like, that's exactly what we're looking for. Like that story, that little snippet of your third grade or whatever and they were like, oh.

So, we realized in the future we need to be much more clear by what we mean by specific examples. It also seemed that this quarter semester check-in really helped build trust. I summarized, you know, and I shared, I said, here's how I used AI to summarize your feedback.

You know, and so there was a lot of transparency involved, and I think that that also started to begin the process of them actually trusting, at least what they were submitting to us. I think also as a semester goes on and students’ kind of test the water with what they push back on or whatever, and they earn full points 'cause they've engaged at the assignment, they started to trust further that they could authentically engage.

But we did see a shift in a reduction of what appeared to be AI generated content as we move throughout the rest of the semester.

[00:27:46] John Nash: That shift, you noticed there was sort of a, what there's an old methods textbook by Krathwohl. He talks about knowing judgments sort of is one way that knowledge gets produced and that experts may, You and your TA made knowing judgements that these, this looks qualitatively different than

[00:28:01] Megan: Yep.

[00:28:02] John Nash: Right.

[00:28:02] Megan: Yep.

[00:28:03] John Nash: totally legitimate. And not to paint too broadly with a brush stroke here, but you saw two things. One is we opened up the allowances of use of AI and AI use went down and by asking students what they thought and felt and incorporating that feedback rapidly, improvement was seen. Are those two big points that I can say are fair to say.

[00:28:30] Megan: Another point, and I will say, you know. On the record if you want to use it, or honestly this could be a whole different discussion, which I actually would love to have, is we started using AI very and I, and I typically do in my grad classes, like we play with it, we utilize it, we mess with it. But I had some of our more big or controversial topics. I created assignments with ChatGPT, and I had them plug it into AI and then they analyzed AI's response. And I think our use of it maybe also helped, but I think that that's probably not the substantial part of it. But I do think as like another topic like how to use AI to like, navigate the sociopolitical climate in a way where I wasn't teaching certain things. Like, comparing administration's policy approaches. AI gave the breakdown and then they reflected and unpacked it in assignments. And so, I think our AI use may have helped, but I do think the two big takeaways that you mentioned are the key point.

[00:29:35] John Nash: And now I hear it's almost as though prior to this shift, you were on two sides of this situation where you had content and the students were supposed to learn the content, whereas now, after the shift, you came to the same side and particularly around AI saying, here's how I can use AI

[00:29:53] Megan: Yep.

[00:29:54] John Nash: And here's how you might use it with me. Is that kind of like how it is now?

[00:29:59] Megan: I think that's a good way to think about it. And that's also helpful for me to think about because I won't be teaching this again until the fall. But thinking about in that week, that first week, having assignment or something that's kind of around like a video of me talking about the ways that I use ai, the ethical lines that I've grappled with, with ai what we know about the science and employment for people who rely too heavily on ai, like, to get them thinking about that too, the way I do in my grad classes. So, I think giving a little bit more trust and respect to them upfront and owning my use

[00:30:27] John Nash: Mm-hmm.

[00:30:28] Jason: I was struck too about how you said the survey itself was a trust building exercise essentially, right?

[00:30:36] Megan: Yes.

[00:30:37] Jason: Which makes complete I really thought about it more of a this is an opportunity for us to gather information so we could move ahead .But really there was something that changed perhaps in the students In the fact that you were asking them and that you were reaching out to them in that way and they had an opportunity to be a little bit more transparent about it And not just trust building but probably communicative in that way too right There's a

[00:31:01] Megan: Yes.

[00:31:02] Jason: survey communicated something a shift the issue perhaps some changes that were coming down the way

so.

[00:31:11] Megan: Yeah.

[00:31:12] John Nash: Yeah So, Megan, what happened at the end of the semester?

[00:31:16] Megan: So, we had been feeling really good. I mean every once in a while, the, like my TA and I would send me like a funny, like one that like clearly, you know, one of 80, you know, that just kind of made us laugh 'cause it's just not how. A 21, 20-year-old would engage in a con, you know, but for the most part we were feeling, you know, pretty good with it.

Or we were noticing where we could do things a little different and that would've probably changed the kind of answer. But now fast forward to the end of the semester and we actually see a huge uptick in clear AI use again. In the survey they talked about that there was a large amount of work in the class and that was one reason why they or others. So, I think in the survey we said why might someone use AI? And so that helped it to not be, you know, we know how students usually are responding about themselves, but it helps take that off of them a little bit.

I think the end of the semester, even though I had actually been really reducing some of the work as the semester wrapped up for their sanity and ours I think the overwhelm of the end of the semester, even if it's unrelated to our class, did probably play a role in what we saw was a bit more AI use.

So, there was two kind of final reflective assignments and in one was individual and one was in their group and in the reflective individual assignments. We had a question around looking back at the course description or the goals, you know, like how did this align? And then there was one that was like, what would you have liked more of?

Like what would you have liked to have understood More? Like what? Or like, you know, something and we had like two. There was like something more about resilience and then something more about disability, but in a very specific way that was clear that most of these had had a very substantial AI role versus just editing.

So that was one thing we noticed. And then in the, the final group reflection in their team. So, they'd been with the whole semester, they were just supposed to think back. I think a couple questions of like, how did doing this work in a team like enhance your learning or something like that. And my TA sent this is making her cuckoo bananas like all over again because it was so many of them what they plugged into ChatGPT, ChatGPT interpreted this as a group project,

You know, we're all contributing to a presentation where this is just a canvas discussion board that they were having. There was no, and so these responses are completely disconnected from actually what they were doing. And we saw this among students, even those that had not, had been really strong and not using AI throughout.

And so there was, at the end of this semester where we were most excited by these more authentic reflections. We hope to receive a bit of a return to the beginning of the semester, so we haven't fully unpacked that at this point. But there is something there were maybe the stress of the end of the semester; they just need to get this done.

Maybe they have another big exam, and this is small scale stuff where we saw a return back to overuse. From our perspective,

[00:34:09] John Nash: Fascinating.

I can concur from personal anecdote, similar situations, not necessarily to use of AI, but it, when students, undergraduates have taken my class, they've seen it as a secondary effort to their courses in their major. Whether it's in chemistry or in business, and they'll complain that they have so much to do over there that their effort towards the end of the term when our big project was happening was lackluster.

And I think, I wonder if it makes us think about the ways in which we should load up assignments. Towards the end. We wanted to be a culminating, like, look at the bow we've put on this thinking and look where you've come. And now I wonder from your story and my experience, how we maybe rethink that a little bit.

[00:34:51] Megan: Yeah.

[00:34:52] John Nash: Hmm.

[00:34:53] Megan: That's where we're kind of left. So very positive shifts. We learned tremendous amount as an instructor, as a ta, and as the TA will independently teach in the future too. But I think that, I will say, if I were to ask my TA kind of how they would've described this grading experience, I think that they would probably say it overall was kind of a soul sucking experience.

And I think part of it was. Having to shift how we think about doing things, and I think also she would say that there was tremendous growth in the students. There was a lot of positives. But I think as learners whenever we have signs where it feels like people aren't using their brain, I think from my TA's perspective, that felt a little scary about the future of learning.

[00:35:34] John Nash: That's almost like back to, that's the knowing judgment, isn't it? I, I'm getting feeling in my gut that people aren't using their brain and that's, yeah, that's real. Jason, you want to ask this cool, final question.

[00:35:46] Jason: Oh, sure I was going to throw in just as an idea of, we've been reading through this book Opposite of Cheating which I highly recommend

[00:35:54] Megan: Okay.

[00:35:55] Jason: and in chapter three talk about one of the things they talked about just made me think of why students are more likely to cheat when they feel the following One of them is that an assessment is insurmountable And I thought that was interesting like as you described this great midpoint where it seems like that students are in the flow and they're being more authentic and they're using AI less and so on And then this description of coming up to the end of the semester And I think about not just a single assessment but you think about the end of a semester sometimes feeling insurmountable and how those feelings may compound across your courses When this is our finals week a library and told me yesterday that the library was packed right They were pulling out foldable tables for people to sit down and to do their work and You think about that And I wonder about too in terms of overall design whether or not there are ways in which can rethink perhaps these as wonderful as they are as you said John putting a bow on the semester Let's see really dig in Let's see really what you have learned but maybe there's some ways to rethink the end of our semesters in terms of these big assignments What do you think?

[00:37:13] Megan: No, absolutely. I agree. And I think actually when you said this about insurmountable, it made me think, so the questions, I think, so there was like five short, but open-ended questions like kind of reflection paper kind of style.

And the first two questions we asked might have honestly been too intellectually challenging for this level. They are the same questions I've used in the graduate version of this class. So, I expect a different kind of response from the undergraduates, so the goal of this class is to understand how did we get here and why does it matter?

And I ask them to reflect on that. That might be too big. For, the group. And then the other one was how well did the class, tell the tale of the ongoing struggle of oppression, resilience? And those might have been a bit too intellectual like that might've.

Triggered some anxiety or imposter syndrome. Like I know what those responses typically look like, but that those questions might have been too big. And then I get into these smaller ones of like, what did you like? What did you not like? What would you tell future students? And those questions are lower stakes,

And so, I think thinking about what kinds of questions we're asking at the end of the semester is probably important.

[00:38:27] Jason: That's good thanks for reflecting out loud with us on this You're just barely ending the semester here so we really appreciate you spending the time with us I kind of had a final question which was as we have a lot of instructors and instructional designers who listen to this podcast what advice do you think that you would give instructors then given your experience this semester when they're thinking about setting up their class for the next semester

[00:38:57] Megan: Yeah, I think that I was so focused on the diversity, like addressing those pieces and then jumpstart and that was me, you know, and also just the demands. I think that there should be a substantial amount, more time spent discussing. AI when working with faculty members who are going into distance ed courses or online teaching.

I think that that wasn't something I did enough of probably. So, I take responsibility, like, you know, I think that there should be a lot more thought and reflection and thinking. Before teaching your first, or as you're teaching online distance Ed. So, people working with faculty I think we need to have a lot more engagement around it because I have some faculty members who are much less engaged with it and have no idea how much AI use was even going on in their course.

And so, I think that there's a large range within faculty of those who are like, no, stay away from it. Avoid it. Don't engage. Hope it's not happening. And so, they don't see kind of patterns to those of us who are like, who went too far down this rabbit hole of trying to like to solve the problem. And so, I think that that range exists probably in all departments.

And so, I think more discussion and support as built into like those structured programs like Jumpstart, like I know the university offers all kinds of workshops and trainings and so forth, but I think when working with faculty, I think, that would be ideal to be part of the conversation more so.

[00:40:11] Jason: Yeah, that's good

[00:40:13] John Nash: Wonderful. Thank you so much, Megan, for joining us. This was a tale for everyone to hear. I really think so. I think, yeah. Yeah.

[00:40:21] Jason: you be willing to share the final policy that you used in your class

[00:40:26] Megan: Mm-hmm.

[00:40:26] Jason: With us Okay That'd be great we'll put that as well as I can share the policy that you started with The one that I share out to faculty I actually have some icons as well that I use for communication that I can share out And we can put those in the Podcast notes and for those listening online learning podcast.com where you can find this episode And a lot more episodes too we've building a little bit of a library John but where you can access these notes and look at these policies for yourself Think about how you're applying your thoughts about AI into your next Course or semester So thank you so much for spending this time with us It was amazing to connect with you a little bit more and hear your story

[00:41:12] Megan: Thank you so much. This was really great. Thank you for having me.

[00:41:14] John Nash: We did it.                   

[00:41:15] Megan: Woo!

[00:41:17] Jason: We did it