Is using AI to write a paper academic misconduct?

We’ve come a long way from ELIZA. Or the ridiculous duelling chatbots.

Natural language artificial intelligence has recently gotten far better than I think many people realize, and today’s article in Scientific American points that out.

A researcher asked an open source artificial intelligence program, GPT-3, to write an academic paper. It did such a good job that the preprint is out and the paper is now under review at a technical journal.

Publicity stunt? It smells a little like that, but then again, this is an area that needs some publicity.

As natural language program like  GPT-3 get more widely available and more widely known, of course university students are going to do what these researchers did. They are going to get the programs to write their papers.

How is that going to shape up in our thinking about teaching?

It using an artificial intelligence to write a term paper cheating? I suspect a lot of my colleagues would say, “Yes,” for the same reason that asking an actual person to write a paper for you is cheating.

But how would you detect that?

The new article suggests that these papers are not going to be obviously defective. If anything, the clue for a professor might be that the paper is too good.

Every one would be a unique output of the artificial intelligence, so that it might skirt plagiarism detectors. I don’t know enough about how GPT-3 generates text to know if it has a “tell”: predictable quirks in expression that might indicate it was an artificial intelligence rather than a person.

I don’t think many university professors are thinking at all about what this means for student assessment. We professors have traditionally wanted to build towards using writing as the preferred assessment. We ask grad students to write theses, after all. But writing as a form of assessment keeps getting compromised and harder to validate.

For another look at what GPT-3 can do, check out this video of someone who used it to recreate a childhood imaginary friend - which turned evil and tried to kill him.

Disclaimer: The story in the video is so wild that I can’t help but wonder if some of it is staged. 

Update, 2 December 2022: Veteran blogger Chad Orzel has a Substack post on this issue: “Why do we assign writing?”. The take-away is that if your writing assignment can be done by an artificial intelligence chatbot, you’re probably doing it wrong.

External links

We asked GPT-3 to write an academic paper about itself. Then we tried to get it published

Prediction: American creationists will try again to get evolution out of schools

On Friday, the United States Supreme Court repealed Roe v. Wade, which was precedent for the nation-wide legal right to an abortion.

That sucks. 

I won’t pretend for an instant that I have anything particularly insightful to say about that particular case. But I do want to post something here about something this signals that are relevant to my own particular interests, namely science education.

It is clear that the US Supreme Court repealed Roe v. Wade because they could. Far right conservatives have been wanting this and threatening this for years. It was always clear that as soon as there was conservative majority on the court, Roe v. Wade would be under threat. And now it’s done.

There was not a reasoned legal decision. This was a partisan power play to give far right conservatives what they wanted.

There is every reason to think that far right conservatives are going send a host of cases going to the Supreme Court. Andy Kim reported hearing, “Let’s keep this going now” on the floor of the House of Representatives. 

Judge Clarence Thomas (who will not bury a hatchet) practically invited it when he laid out three cases that he thinks the court should revisit. They are all high on the conservative wish list to repeal and all concern people’s sex lives. (Which is, by the way, super creepy.)

But Edwards v. Aguillard has to on the far right conservative hit list. 

That’s the ruling that said it was unconstitutional to teach creationism (or “creation science” as it was called for a while) in science classes in US K-12 public schools because it violated the Establishment Clause of the First Amendment. That said, “states can’t get promote one particular religion over the others.”

No, Edwards v. Aguillard hasn’t been mentioned by name by Thomas or others. I haven’t seen anyone else bring this up yet. But I would bet that creationists are already looking to line up court cases in hope of getting the issue on the docket of the Supreme Court. 

Given that Kitzmiller v. Dover never got to the national stage, it wouldn’t surprise me if the Discovery Institute or some other entity decided to make another run at making “intelligent design” legal in schoolrooms.

Creationism is important to the same far right, religious fundamentalist conservatives who have been spearheading the assault on Roe v. Wade. Mike the Mad Biologist has often written about “Nothing in movement conservatism makes sense except in the light of creationism.” (2012, 2019, to give a couple of examples.) My take is similar. The tactics used by creationists for decades were eventually adopted by the wider US conservative political machine as their default mode of operation.

The teaching of creationism will be come up, and soon, if the court continues the way it’s been acting.

This week, the national meeting of the Society for the Study of Evolution has been happening and I’ve asked a couple of times if anyone is even talking about Edwards v. Aguillard. No answer yet. This makes me worry that my colleagues are going to be shocked and appalled if this goes to court and they lose, because they won’t have taken any action or prepared for such an eventuality.

This is small potatoes compare to the big ticket fantasies that conservatives have (i.e., terrorize people who are not white conservative Christian fundamentalists). An enormous amount of damage could be done well before the courts get to repealing a science education case.

But it’s not nothing, either. 

Attacks on education undercut the future you can even imagine, never mind the future you can build.

Edit: On reflection, it might not be creationism that goes up for a court challenge. It might be Engel v. Vitals, which ruled against prayer in schools. Or something else about Christianity (disguised as generic “religion”) in public schools. Because it’s one of their big bugbears. They know the power of public education in shaping culture and they want to influence it.

Update, 27 June 2022: Well, this morning’s ruling on Kennedy v. Bremerton School District seems relevant to this post. In particular, the Supreme Court rejects the so-called “Lemon test” that I have seen cited consistently in cases regarding the teaching of creationism.

This is getting out of my knowledge base very fast, but here is one early reaction:

(T)he Supreme Court effectively grants special, heightened First Amendment rights to religious speech, allowing public school teachers to pray on the job while denying most other public employees basic free speech rights.

And from David Shiffman:

I went to public school in a very blue part of a purple state.

As a Jew, I requested that our music class “holiday concert” include at least one song that’s not a Christmas song.

A Christian classmate threatened to murder me.

Today the Supreme Court sided with him, not me.

This seems to be a very clear move towards more religion in public schools. And that would include science classes.

Update 2, 27 June 2022: Something that puzzled me in the decision was verbiage that the Supreme Court had long ago abandoned the Lemon test or words to that effect. Steve Vladek disputes this.

As Leah notes, the conservative majority in Kennedy overrules SCOTUS’s major prior Establishment Clause precedent in Lemon, but tries to pretend that the Court had already overruled it in prior cases (spoiler alert: it hadn’t [Emphasis added.]). This is sketchy even if you think it's correct.

And other people seem to agree. This isn’t ye olde coute rulyngs, but the first time the Supreme Court has explicitly rejected the Lemon test. Tweets here and here, for instance.

End abstract sponsorship for the Neuroscience meeting

Tomorrow is the deadline to submit abstracts for the Neuroscience meeting (the biggest academic meeting in the world).

This meeting does something that I have never seen at any other meeting. Every presentation and poster needs a society member to “sponsor” the abstract. And a member can only sponsor one scientific and one “metascience” presentation.

“So just become a member.” Not that easy, because membership also requires you to be sponsored by two active existing members. So if you are in a smaller campus, there may be no existing member who can sponsor you.

If you are in a lab with three society members but want to present four posters, you’re stuck.

The problem is so obvious that the Society’s Twitter account has taken to trying to help people rustle up a member to sponsor abstracts by retweeting requests. Like this.

Desperate last minute request from a @UniLeiden postdoc! Would anybody mind sharing their @SfNtweets membership ID for me to be able to use as a sponsor? Much appreciated 🙃

Or this or this. Not to mention this and this.

This is a failure on a couple of levels. First, it’s a stressful waste of time for people who want to present at the meeting. Second, it’s clear that people are willing to sponsor presentations they had nothing to do with.

I suspect that this policy could also cause problems with around representation, which occur pretty much any time you create an obstacle that has anything to do with money.

And it shouldn’t be up to a social media account to try to fix a conference admission problem.

I can see three possible reasons for this policy – two okay and one bad.

1. Keeps out cranks, kooks, and quacks who want to present crackpot ideas. I think there are better ways of achieving this.
   
2. Limits the size of the meeting. Sorry, but that ship has sailed.
3. Drives membership. This is the bad reason. Look, either make membership worth having or increase the registration fee for non-members to compensate for lost revenue.
   

The meeting has done this for a long time. I ran into the problem the very first time I submitted an abstract. And that was, as they say, a while ago now. It feels like the kind of policy that sticks around because “We’ve always done it this way” instead of serving any valuable purpose.

It is time for the Society to publicly say why they limit submissions this way, or get rid of the policy altogether.

External links

Neuroscience call for abstracts

New podcast epiode for ABT Time

My newest podcast interview is the ABT Time podcast, episode 39, hosted by Randy Olson.

Randy has featured on the blog a few times before, so long time readers may recognize that “ABT” in ABT Time is an abbreviation for “And, but, therefore” – the key words for making a concise narrative.

The ABT structure features prominently in the Better Posters book because it is an powerful tool for encapsulating a project in a sentence. 

The podcast mostly talks about narrative and posters, but because I’ve crossed paths with Randy a few times, our chat is more conversational than formal interview.

The ABT Time podcast should be available wherever you get your podcasts (Apple, Spotify, YouTube, etc.).

External Links

ABT Time #39 on ABT Agenda