HN

Comments (407)

• Menu

  recursivedoubts
  In the AI era, tests are going to have to be in person and hand written.I have written an article on how I have adjusted my classes to the situation:https://htmx.org/essays/universities-and-ai/Ironically, I think the AI era may make university degrees a better signal of the intellectual abilities of students due to the presence of pre-computer infrastructure like large lecture halls, industrial-scale copiers, etc.
• Menu

  bkallus
  I have seen it firsthand in the CS department here at Dartmouth. It is bad.We're currently designing a new intro systems curriculum, and we're thinking of it as an adversarial problem. That is, we're designing the course to ensure that a student optimizing for the best grade per unit work still meets our learning objectives. That means, as everyone else is saying, paper exams, but also 1-on-1 interviews to check that students understand each assignment they turn in. These interviews feature both factual questions ("You're using this macro from that library. What does it do?", "Please describe what this function does and how it works.") and conceptual questions ("Why is this code structured this way instead of $whatever?", "How else did you try solving this?", etc.) This doesn't stop students from generating code, but at least they have to understand that code in detail.This is not as good as writing the code yourself, but how much worse is it? For math classes, this gap is gigantic. Obviously, understanding someone else's proof is much easier than writing your own. For programming classes, I think (without evidence) that the gap is somewhat smaller.My experience from the past is that when this kind of evaluation is made clear up front, the students know what to expect and either do fine or drop the class in the first week. If you start with take-home exams and then spring paper exams on them halfway through the course, then half the class is cheating and won't be able to recover, as we read in the article.In general, our students are somewhat motivated by an abstract desire to learn, but are much more motivated by grades. If there exists a straightforward path through your course that leads to a good grade without doing much work, most students will take it. (Our undergrads' course review website is literally called "Layup List." They are actually this shameless.) It's our job as instructors to ensure that all paths leading to a good grade either require learning the material or are more difficult to pull off than just learning the material.It's best not to blame the students. They are good at optimizing metrics; that's how they ended up here in the first place. We just need to better align the evaluation metrics with the outcomes that we're looking for.
• Menu

  cherryteastain
  His research is in Game Theory. He should have realized that, in a situation where all competitors are (possibly) using LLMs, the game theoretic optimal choice is to use LLMs.
• Menu

  yiyingzhang
  As a university professor, I honestly don't understand the point of grading. Who will look at and care about grades? Likely company HR. But then why should we (professors) do the screening for companies for free? Also, grades have long been inflated to a point we might as well just give everyone an A and let companies figure out how to select people.
• Menu

  userbinator
  take-home, closed-book typeWhat an oxymoron. I agree with the others here that AI isn't the problem.
• Menu

  zabzonk
  While I am in no way a supporter of AI cheating, or whatever we want to call it, I can tell you from experience that there is nothing more tedious or soul destroying than invigilating a written multi-hour exam. It put me off teaching in higher education.IMHO to solve many problems we should go with Ivan Illich's ideas: https://en.wikipedia.org/wiki/Deschooling_Society and make education about education, not testing and certification.
• Menu

  pants2
  When you're a student in a competitive program at a top university, graded on a curve, and you know your fellow classmates are cheating with AI, you have little choice but to do the same. Especially when jobs for new grads are harder to come by and there's more pressure to also go above and beyond with internships and side projects during your time in school. There's no way to compete without cheating.
• Menu

  BinRoo
  > In the AI era...Back in my day, you could also just Google the problems and find the solutions. What mitigated cheating at UVA was the honor code and each professor's faith and trust in our integrity. That culture was enough to not cheat.Imo, the fix should be to work on culture. Cheating should always be a tempting choice, so that the student may challenge their integrity, which is a muscle that can atrophy.
• Menu

  nitwit005
  These articles consistently fail to acknowledge students were cheating in large numbers prior to these AI tools being available.It was certainly not difficult to cheat at a "closed book" take home exam before.
• Menu

  danny_codes
  Damn that's crazy. Guess the take home test is dead now.I never understood this behavior from undergrads though, you're paying so much for an education and then you just skip the education part? Why bother?
• tmsh
  let people use as much ai as possible. encourage it. and as an educator, you have to learn to leverage it oneself or not (depending on the subject). and be better at using it than students if it does make one 'better'.if ai doesn't help, then it won't help. if it does help, then you should use it. the metric is your output of whatever is being tested. writing an essay well and clearly understanding the material. solving a pset. whatever.if you give access all the time for that, and then you test on a hard problem that could be done with or without ai, then it's fair. e.g. "clearly explain these four sentences of Y." obviously ai researching loosely and blathering isn't useful. won't be high signal / dense and correct and worthy of an 'a'. but someone who can harness ai and someone who knows the material well in the end will be rewarded the same by society. what you are testing is correctness and information density in a response. so you have to start now in accepting the reality that those who use ai to get there should be rewarded just the same as those who don't.the burden is on educators to be as good as they can with ai if it is relevant or not if it is not relevant (and schools to fund them and ai companies to fund them if they have excess capital and are humanitarian).and note the hard part even for us engineers at tech companies is in the correctness. it is very hard. but the sooner we start teaching how to do things correctly with ai, the more prepared the next generations will be.
• haunter
  I'm from Hungary and the majority of the exams here are oral one-on-one interviews (depends on the course of course but still). I've never ever had take home exams and or even quiz like tests were very uncommon.
• aneesh
  This is not surprising. While cheating has always been around, it seems to be more prevalent now with high pressure and easy access.I’ve talked to a bunch of teachers and school leaders, and see three main ways schools are handling AI use in assessments:1. Punish it: Detect AI use on homework and take home exams; treat it as cheating.2. Prevent it: Move to live assessments – oral or offline – that are hard to cheat on.3. Embrace it: Assess the process, not the output.The second one seems to be the only real answer for foundational subjects. And the third one can also work for more creative or project-based work.
• adithyassekhar
  I’m no longer in college still, The day I longer have to worry about putting a meal on my table, I’ll treat college as this fantasy “thirst for knowledge” thing as some of the commenters here suggest. Till then, it’s just a meal ticket.
• Balgair
  It seems that the Oxbridge model is really going to be the only workable one here in the future. Small groups of about 8 total, lead by a proctor, meeting regularly. The social pressures to not cheat in such small groups keep it honest.The obvious problem with that is it is terribly expensive. You need Masters or Doctoral level people for long periods alone with students and you need to trust that these proctors won't be some form of -ist towards the students and also that their grades will be fair.It is by design not something that scales.But it seems that is where the path lies at this point.Essentially, the aristocracy gets education again and the plebs get to fight/cheat it out amongst each other and paying to do nothing in the end.Damn it
• fourseventy
  "Take home closed book" exam is asking for mass cheating. It was probably happening way before AI too. AI is just an excuse now.
• Menu

  billsmithaustin
  A similar thing happened in a first-year CS class at Purdue this year: https://thecheatsheet.substack.com/p/432-cheating-at-purdue-....
• whatever1
  We don’t allow calculators in elementary school when we teach and test students for multiplication and division. We don’t allow for dictionary when we test for vocabulary.
• Menu

  linzhangrun
  AI has drastically reduced the cost and concealment for cheating, even in offline exams.A single person can easily do that using glasses with a micro-camera & rice-sized earbuds, and almost impossible to be caught.
• Menu

  Meekro
  Nobody would hire a chess coach and then use Stockfish to cheat on the problems. Whatever the students are paying for here, it certainly isn't an education.
• Menu

  tty456
  "take home, closed book"This is a trap. I understand they've done this in the past, but profs are paranoid now.I don't believe he's 100% correct on each incident of fraud and he's going to ruin students [academic] lives because of it.
• zdc1
  One of the hardest and best exams I had to do in university was my behavioural finance midterm exam: the format was that I had to sit down with the professor and have a five minute discussion on a topic of my choosing. It was surprisingly tough, and the process of verbalising what you know about a topic doesn't give you much room to hide.Accurate and high-quality exams are a solved problem. The issue is that universities aren't necessarily judged on teaching quality and opt for examination methods that scale well.
• rayiner
  Instead of denouncing them, the professors should expel every single one of these students. These people cannot be allowed into the economy. (And don’t “what about” me about dishonestly already existing in the economy. I know it does. But it’s like littering—you’re not justified in adding another piece even if the ground is already dirty.)
• Menu

  gchallen
  > He has conclusive evidence that at least 50 students cheated on the March midterm exam, making it the biggest known scandal at Brown and in the entire Ivy LeagueI'd like a citation for this being the "biggest known scandal" in the "entire Ivy League". Frequently such situations are kept somewhat quiet, for a variety of reasons. But fifty students is not a large number in courses that can enroll hundreds or up to a thousand students.
• WalterBright
  When I visited Yale recently, a professor who taught comp sci complained to me that most of his students were using AI to do the work. I asked him if he knew which students wanted to learn. He said yes. I suggested he teach to them, and to heck with the cheaters.
• WalterBright
  Juxtapose the mass fraud on exams with the greater difficulty of finding a job after graduation.
• francisofascii
  Assessment should probably a mix of all of the various forms: handwritten blue book, verbal, multiple choice, and AI assisted essay. It really depends on what is being asssesed.
• Menu

  beloch
  "This year, the economist decided that both the midterm and the final exams for his course would be of the take-home, closed-book type (there is a certain tradition of this at Ivy League schools). “It’s a very nice kind of exam, because as you’re giving students practically unlimited time to complete it, it lets you make it harder than normal, to see how far they can go.”..."But it also hurts him that the one time in 34 years that he decided to offer a take-home exam, for highly justified reasons, the response was wide-scale fraud."-----------------Not to in any way defend or condone academic misconduct, the fact that this was his teaching-career-first take-home exam is probably relevant. Take home exams can be fiendish. I remember having one in grad school where we were given a very insufficient 36 hours to complete it, and many people just didn't sleep. That was from a prof who knew what he was doing. This guy may have accidentally made his exam absolutely sadistic.Couple this with the fact that students often have other exams they need to be studying for in the same time window. The pressure can be immense. The temptation to use AI to help is going to be hard for many to resist unless the penalties are severe and strictly enforced.AI cheating is probably going to be a problem going forward in all situations, but open-book, take-home tests are going to bring it out more strongly than other test formats.
• neilv
  I'm saddened and concerned by these allegations of a deficit of integrity.I was very fortunate to attend Brown University for grad school, and consider it a great place.Why would many people who were also fortunate to attend there not honor that opportunity?
• Menu

  throwawaypath
  Administration needs to eschew "technology" and demand analog solutions: hand written exams in proctored rooms, no devices out in the classroom, no take home work, etc.
• trashface
  Seems like these youths are doing exactly what their future employers will expect them to.
• Menu

  jdshaffer
  I've been trying to figure out how to avoid AI fraud in my classes, like many others. While not perfect, I've written down my thoughts and attempts in the hope it might help someone else:https://web.hedc.shizuoka.ac.jp/msg-from-center/creating-mea...Edit: Didn't realize the original URL was paywalled. Sorry. This newer URL is open. Apologies.
• ungreased0675
  > “…if we want to preserve the future of higher education”He understands the stakes here. If a university degree becomes useless, then what?
• Menu

  michaelfm1211
  The problem isn't AI, it's that you gave a take-home exam expected no one to cheat.
• JumpCrisscross
  Yawn. When a university starts expelling students for cheating, I’ll pay attention. I suspect donors, politicians and employers will, too.
• jongjong
  He is fighting a losing battle. That's why nobody in the administration cared. Academic integrity already doesn't exist anymore. As Nassim Taleb would say; there are plenty of highly educated idiots with PhDs.Even the idea that MIT is somehow better than some other universities is itself a fiction. People are conflating financial success with academic ability. The former is mostly the result of social connections which are formed within the academic institutions and have very little to do with actual capabilities.Universities should just sell degrees for a high price without requiring the students to attend. If they're rich enough, their skills aren't going to matter anyway; they'll succeed in their careers regardless so the university will still look good. 'Academic integrity' will be intact. Especially true for business, economic degrees or other humanities.
• adamnemecek
  > He thinks the time has come for an in-depth debate so the technology does not signal the end of higher educationI hope it does.
• xp84
  This may be a hot take, but:The problem is that kids, even (or especially?) in the elite schools, are treating college as a box to tick. At the Ivies, for many students, it's nothing more than the requisite means to get their ticket punched so they can become/remain part of the elite class. This has been the conventional wisdom for half a century.Now, this paragraph doesn't apply to an economics class like his, which is actually a useful degree, but many students will never use what they learn in college in a professional capacity, so it barely even matters if some (maybe even most?) students cheat because outside of the useful degrees, it's mainly a sorting hat to determine who works at Starbucks to pay back student loans, while rolling their eyes because of their 'valuable degree,' and who works there unironically, to pay their bills.What needs to happen though is that students at all levels need to either believe that they need to learn what they're choosing to take courses in, or even better, actually innately want to learn it to satisfy their own interest. Either will do. If you have neither intention toward the material, of course you'll cheat your way through it. No student who actually wants to learn would waste their time and money taking a class only to not learn, and cheat their way through it.
• nephihaha
  Time for hand written essays again. That way, at least if they do use AI, they will have had to process some of the content a bit more.
• Barrin92
  All these over-complicated "solutions" are incredibly funny. At my uni in Germany education worked in a very straight forward way. There is no graded homework (you're in a university not an elementary school), at the end of the semester there's a three hour exam, on pen and paper (our CS profs deducted a point per syntax error btw so mind your parentheses) and if you don't do any homework or don't show up that's on you because you're an adult, but good luck making it through the test.Of course 70% or so usually crashed out in particular in Calculus and I suspect given that US education is paid for daycare that's exactly the thing that can't happen which is why they're never fixing it
• Menu

  JimsonYang
  As of now chatgpt subsidies its consumer subscription-I wonder if cheating on exams will be still promiment once students are forced to pay $30 a monthSince students are notorious for being cheap
• djoldman
  > This year, the economist decided that both the midterm and the final exams for his course would be of the take-home, closed-book type (there is a certain tradition of this at Ivy League schools).These news articles are just tiresome at this point. Obviously folks cheated previously, obviously it's easier now, obviously the answer has been to not have take homes all along.
• Menu

  fhn
  the professor has all the power in the classroom. If you don't want cheating, define better conditions for the exam. You allowed a take-home exam which means students are able to use any and all resources.
• Menu

  hackermailman
  They're going to have change everything so use of an AI assistant doesn't matter because once they graduate they're just going to continue using it anyway.If it's a math for finance course then some kind of model building for the midterm and being marked on the quality of the model or something. If AI becomes so good that it always chooses the best fitting model and requires no numerical optimization then they will have to change the courses to be more like UChicago where it's primarily undergrad directed research but AI assisted.
• Menu

  jackphilson
  It's the old game that he's trying to preserve. It's time to move on to the new game. When the landscape shifts beneath you, its very low probability that the existing structures on the landscape are a good fit for the new landscape, and the structures on the new landscape must be rethought from first principles.