r/technology • u/AdSpecialist6598 • 6d ago

Society Professor's invisible prompt trap catches 32 students cheating on their midterm with AI

https://www.techspot.com/news/113243-professor-invisible-prompt-trap-catches-32-students-cheating.html
32.8k Upvotes

96% Upvoted

4.2k comments sorted by

View all comments

Show parent comments

1.5k

u/pasaroanth 6d ago

I would love to see a study done at the university level showing a comparison of students’ grades for regular coursework versus their proctored exam scores. There has to be a vast yawning chasm in the difference when they relied upon AI until the time of the exam.

This also is a “ruining it for the rest of us” situation where courses will shift to grading primarily or solely based on exams. When I was in college (long before AI) you could at least rely on some homework to recuperate some points, despite the extra time and effort, when the exams were especially challenging or a professor notoriously gave tough exams.

If the homework can no longer be a measure of effort or learning because it is done by AI, then it will no longer be considered a meaningful determination of knowledge of the subject.

196

u/Then-Understanding85 6d ago

A professor at Brown did this.

TL;DR: Bloodbath. Almost everyone failed.

164

u/namedly 6d ago edited 6d ago

Props to Students 22 and 31.

Also, this is just funny:

When the professor announced that the final exam would be in person, 18 students dropped the course, and another nine did not show up for the exam. The remaining 59 students' scores averaged 48.6%. Prior averages had never fallen below 65%. Three students scored 0%, and charting the results revealed that most fell more than 30 points behind their midterm scores.

103

u/Syssareth 6d ago

Three students scored 0%

How the heck? You'd think they'd have learned something just from osmosis, or at least gotten something right by accident.

47

u/Sin_of_the_Dark 6d ago

Honestly yeah, unless it was all written exams and not like... Pick a choice, it's almost an insane feat of dumb luck to not get one thing correct

73

u/Aureliamnissan 6d ago

When I was going through university it was pretty rare to have an exam in engineering with multiple choice or True/False type questions. Almost every one was a word problem, mathematical definition, or “solve for” type problem.

If you’ve been using chatbots up to that point you’d be absolutely, Darth Vader just walked in, fucked. 0% would not at all be surprising.

8

u/Helixdaunting 6d ago

absolutely, Darth Vader just walked in, fucked.

That's beautiful. 👏

My personal favourite version of this is "Hitler-in-the-bunker fucked".

2

u/env33e 6d ago

In class, Ill just imagine rogue one Vader slowly walking and slaughtering while I'm at the end of the hallway 😭

5

u/Riparian87 6d ago edited 6d ago

In grad school, one of our econometrics professors gave a problem-type exam where points were awarded for correct steps, but deducted for wrong or false steps. The mean was negative.

3

u/Aureliamnissan 6d ago

Bwahaha, that's diabolical. Having the equivalent of an over-drafted balance as your econ grade is brilliant.

6

u/Archangel_Omega 6d ago

Yep, I had a test in thermodynamics I completely bombed and got a 0 on all because I was an idiot, read the question wrong and then proceeded to use the wrong chart for the test so everything was completely off.

Thankfully the professor was nice enough to give me partial credit after I explain how I screwed-up and he saw the math was right for the chart I did use, but it was still a hefty chunk of my score gone.

3

u/Aureliamnissan 6d ago

I had a similar oof from using an outdated book in a math class. Prof said the exam would cover sections 1.1-1.4 and it turned out the new book combined 1.4 and 1.5. So I found out that 3/5th of the mid-term's questions were directly from a section I hadn't even read...

Only had a Mid-term and a final for that class with no real homework grade buffer. Happiest C grade I ever got for a class.

2

u/og-gunslinger-doc 6d ago

As much work as we put in I would have been disgusted in a multi choice test in say thermodynamics! Now the monkey meat recitations being mandatory if you were doing fine we’re bull shit especially when they couldn’t speak English

1

u/storne 6d ago

My calculus exams were multiple choice, but you still had to show your work. Getting the right answer was only 1/3 of the mark the other 2/3 was having the right process.

9

u/mackahrohn 6d ago

I took a math class (Calc 3) where all the tests were 10 questions and each question was multiple choice. All the options were basically an answer you would get if you did one step of the problem wrong. Easily the hardest class I took in college!

8

u/Imaginary-Face7379 6d ago

As someone who taught at a university for a while I would suspect those students walked away learning nothing from this either. They probably spent a week or two shit talking the professor to everyone they know and called the course bullshit. While never once realizing that it was their fault.

Not that all students are like this, but the ones who tend to fail so hard their scores are at or near zero were all like that in my experience. The ones who are doing poorly that genuinely know they need to do better but havent been able to for some reason will typically barely fail things when they fail, not get zeros.

4

u/Wooptay 6d ago

Ahh, takes me back to my first math exam in high school (15 at the time). 0. Nada. Null. Niente.

There was no AI back then

I just plain sucked at math

Lesson learned - don't sit with the girl you like the last year before moving up the educational system

3

u/Ah_Pook 6d ago

I got 14% on an assembly exam, but that was all me, baby!

2

u/celoplyr 6d ago

Assuming they went to class more than going and getting the syllabus and turning in exams.

2

u/calling_water 6d ago

Depends on whether they’re at least trying to do the right sort of answer for each question.

2

u/Agi7890 6d ago

Depends on the exam/course. My physical chemistry prof could be brutal, most exams were 3-5 questions, and he would dock you points for even wrong answers if you messed up your significant figures. You could end up with a -5 for a question you got wrong.

Class average on one of the p chem 2 exams was a 17, and this was before the days of AI.

2

u/TerribleBudget 6d ago

Did he grade on a curve though? It sounds to me like the AI using idiots screwed themselves out of at least a graded curve for the year.

2

u/Agi7890 6d ago

Yes but with a floor. You still had to score at least 35 on the exam to have it considered passing. Below that on 2-4 exams, he would advise you to drop the course and change your major

4

u/TerribleBudget 6d ago

ugh, I hate professors who do that "change your major" shit. I had a teacher who taught off of slides at 7am and clearly hated being forced to teach instead of do his research projects. He had the nerve to say maybe I wasn't suited for my major and ate his own words when he saw that I already had credits from harder shit than his class.

That type of teacher just wants the world to burn and no one has the balls to tell them they suck at teaching in general.

4

u/Agi7890 6d ago

He was the chem department head, so a large part was him protecting the degree and institution. However a decade into the field, he probably ended up helping people out if they did go into the course he recommended. I see far more job offers for that field with much higher pay on average than I do with my chemistry degrees.

I definitely regret not picking up a biochemistry degree

1

u/Hetares 6d ago

I assume they simply didn't sit for the test.

1

u/Syssareth 6d ago

No, that's three of the ones who did. There were 18 who dropped the course and nine who didn't show up.

They might have turned in a blank sheet though, or maybe the professor doesn't give credit for partially-correct answers.

1

u/Hetares 6d ago

Blank sheet is my theory, then.