Generative AI: Two of of three is bad

There’s an old joke: “Fast, cheap, good: Pick two.”

Generative AI is fast and cheap. It will not be good.

I would like “good” to be a higher priority.

Edit, 1 December 2024: I think academics, under the intense pressure to be productive, prioritize those three characteristics in that order. The first two might switch (grad students are more likely to prioritize “cheap”), but I think “good” is regularly coming last.

Stay out of my academic searches, gen AI

Something I had long dreaded came to pass yesterday.

Google Scholar introduced generative AI.

“New! AI outlines in Scholar PDF Reader: skim the bullets, deep read what you need.”

Andrew Thaler had the perfect riposte to this new feature.

If only scholarly publications came with a short synopsis of the paper right up front, written by the authors to highlight the important and salient points of the study.

We could give it a nifty name, like “abstract.”

Exactly! Not only do researchers already outline papers, many journals require two such outlines: an abstract and some sort of plain English summary.

I don’t need this. I don’t want this. No more generative AI infiltrating into every nook and cranny of the Web, please.

What is misinformation for?

A new article on how many people in the US are increasingly hostile to reality has much to contemplate, but I wanted to briefly muse on this:

So much of the conversation around misinformation suggests that its primary job is to persuade. But as Michael Caulfield, an information researcher at the University of Washington, has argued, “The primary use of ‘misinformation’ is not to change the beliefs of other people at all. Instead, the vast majority of misinformation is offered as a service for people to maintain their beliefs in face of overwhelming evidence to the contrary.” This distinction is important, in part because it assigns agency to those who consume and share obviously fake information.

I see the point, and agree with it to some extent, but I think this underestimates the persuasive power of misinformation.

It neglects the “rabbit hole” effect that misinformation has had on fostering conspiracy theories and radicalization. It neglects the slow corrosion that has been happening in political discourse. It’s not just that political parties (particularly in the US) are polarized, but that some have gone ever more extreme.

I can see a connection between Caulfield’s “misinformation helps maintain beliefs” and persuasion. People’s beliefs are informed by different points of view. Without countervailing points of view, those existing beliefs can become more certain and more readily drift to ever more extreme versions of that belief.

Misinformation is often better described as straight-up propaganda, though. But we seem to have lost that word through fear of calling lies, lies.

External links

I’m running out of ways to explain how bad this is

Reliable shortcuts

There’s an old saying that if you build a better mousetrap, the world will beat a path to your door.

I think that’s true of shortcuts, not mousetraps.

Everyone wants reliable shortcuts. They don’t want to have to assess every available option every single time.

Best seller lists, Consumer Reports, “Two thumbs up!” from Siskel and Ebert, “Certified fresh” on Rotten Tomatoes, “People also shopped for” on Amazon, “Five stars” on Yelp!, and awards shows are all efforts to create shortcuts.

An amazing number of arguments in academia are about shortcuts. Almost every debate about tenure and promotion and assessments of academics I have seen or read is about shortcuts. Arguments about the GRE are about shortcuts.

I got thinking about shortcuts because of this article on what journals go into PubMed.

For some members of the scientific community, the presence of predatory journals, publications that tend to churn out low-quality content and engage in unethical publishing practices—has been a pressing concern.

Rebecca Burdine is among the concerned, because she advised people to use PubMed as a shortcut.

I could tell parents “researching” their rare disease of interest that if it wasn’t on PubMed, then it shouldn’t be given lots of weight as a source.

Stephen Floor thinks the problem is even wider:

This has also contributed to the undermining of “peer reviewed” as a measure of validity.

But again, “peer reviewed” is a shortcut. Anyone who’s been in scientific publishing for a while knows that assessing scientific evidence is messy and complicated. Every working scientist has their own “That should never have gotten past peer review!” story.

We will never, ever get rid of shortcuts. People crave certainty and simple decision making rules. But we should talk about using shortcuts in science in realistic ways.

It is not reasonable to expect any shortcut to be perfectly reliable all the time. Don’t ask, “Which shortcut is better?” but “How can I use a few different shortcuts?”

Unfortunately, scientists who understand the nuances of a situation often do a shoddy job of conveying that nuance. Or maybe they just get tired of being pressed for shortcuts. So we have kind of brought this on ourselves.

External links

Academics Raise Concerns About Predatory Journals on PubMed

All scholarship is hard

Nicholas Evans wrote:

The solution levied by synthetic biologists is to get more biologists doing ethics. That this is always the suggestion tells me a) you think it’s easier to think about ethics than synbio; b) you want to keep the analysis in house. Neither are good.

Seconded, confirmed, and oh my God yes. I’ve been through several iterations of this in biology curriculum meetings, where I or others have suggested incorporating some non-biology class into a degree program, or even just an elective students funded by a training grant have to take. And the reaction is just what Nicholas describes:

“Why don’t we just do it ourselves?”

The single exception seemed to be chemistry. Maybe there was less suspicion because of the blurry line between molecular biology and biochemistry. Or maybe it was because their department was right above ours and we knew the people better. But when it was ethics or writing or statistics: nope, we’ll develop out own class taught by our own faculty in our own department.

I get a lot of variations of “Is is easier to do this or that in academia?” questions on Quora, too.

In an institution, this attitude of “We know best” is made worse by administrative measurements. Departments are evaluated by how many credit hours they generate. So when I suggest students might take a course taught by the Philosophy or Math or Communications or Psychology department, the response is, “We’re just giving credit hours away.” Since credit hours are one thing that are looked at to determine resources, it’s an understandable reaction. It’s Goodheart’s law in action. The measure becomes a target and changes what the measure does.

Nicholas notes:

The vast majority of people talking synbio ethics have almost no training in ethics. You wouldn’t accept that in the technical side of synbio, so don’t accept it in ethics.

Exactly. We often complain about how people don’t respect expertise on many controversial subjects, like evolution, climate change, or vaccination. But we see the same disrespect within universities for scholarship in different fields. Scholarship in every field is hard, and “My field is better than your field” is a shitty game.

Hat tip to Janet Stemwedel.

The myth of fingerprints

Could you have made a mistake?

If you are a fingerprint examiner in court giving testimony, the answer was once, “No,” according to Mnookin (2001).

(T)he primary professional organization for fingerprint examiners, the International Association for Identification, passed a resolution in 1979 making it professional misconduct for any fingerprint examiner to provide courtroom testimony that labeled a match “possible, probable or likely” rather than “certain.”

(I’ve been unable to find is this is still true.)

A new paper by Ulery and colleagues is a follow-up to a paper they published last year on fingerprint analysis. The previous paper found 85% of fingerprint examiners made mistakes where two fingerprints were judged to be from different people, when in fact they were from the same person (false negative). There was much more analysis, but you get the idea.

The researchers wanted to see how consistent the decisions were after time had passed. For this paper, they used some of the same fingerprint examiners that had been tested before (72 of 169 from he previous paper). It had been seven months since the fingerprint examiners had seen these prints. They were all prints that they’d seen for the previous research, but Ulery and colleagues didn’t tell them that.

Because the experimenters wanted to see if examiners who had made a mistake before would make the same mistakes again, the choice of what pairs of fingerprints to make was somewhat complicated. But all examiners saw nine pairs fingerprints that were not matched (from different people) and sixteen pairs that were matched (same people). And it’s also important to note that the fingerprints chosen were chosen in part because they were difficult.

In the original test, the fingerprint examiners only rarely said two fingerprints were from the same person when they weren’t (false positives). On the retest, there were no cases of false positives, either repeated mistakes from the previous test or entirely new mistakes.

The reverse mistake, the false negatives, were more common. Of the false negative errors made in the previous paper, about 30% were made again in the new study. And the examiners made new mistakes that hadn’t been made before.

There is some good news here, however. One piece of good news in this paper is that in some cases the examiners’ ratings of the difficulty were correlated with probability they would make the same decisions as before. But he examiner’s ratings of difficulty, however, only weakly predicted the errors that they made.

Another important finding is evidence that the best way to reduce errors is to have fingerprints examined by multiple people, rather than multiple examinations by the same person. The authors write:

Much of the observed lack of reproducibility is associated with prints on which individual examiners were not consistent, rather than persistent differences among examiners.

Nevertheless, even with two examiners checking fingerprints, Ulery and colleagues estimate that 19% of false negatives would not be picked out by having another examiner check the prints.

These papers all concern decisions made by experts, which is obviously the logical place to start from a policy and pragmatic point of view. As an exercise in seeing how expertise develops, tt would be interesting to see if beginners showed the same types of patterns in decision making.

References

Mookin JL. 2001. Fingerprint evidence in an age of DNA profiling. Brooklyn Law Review 67: 13.

Saks M. (2005). The coming paradigm shift in forensic identification science Science, 309 (5736), 892-895 DOI: 10.1126/science.1111565

Ulery B, Hicklin R, Buscaglia J, & Roberts M (2012). Repeatability and Reproducibility of Decisions by Latent Fingerprint Examiners PLoS ONE, 7 (3) DOI: 10.1371/journal.pone.0032800

Photo by Vince Alongi on Flickr; used under a Creative Commons license.

Credential evaluation

“Strong support for the argument that video game violence is indeed harmful.”

What would that strong support be? According to this story, it’s whether authors have published a scientific paper about media violence.

Read that carefully. It’s not about scientific papers on media violence, it’s about the authors of scientific papers on media violence.

Here’s the deal. There’s a court case. California wants to be able to ban the sale of video games to people under 18 based on the violence content. People get to file amicus briefs to offer their opinions for the court to consider. The authors of this study decide to look for evidence that video games cause violence by examining the scientific credentials of who wrote the court briefs.

They compared whether the amicus brief authors had published papers about media violence. Again, they’re not saying anything about the papers, just whether a brief author has written any in that area. In theory, someone who published a study on media violence that showed no effect, but who argued in favour of the proposed laws, you would be counted on that side, even if your research didn’t support a link. (Admittedly, that seems unlikely.)

In both sides of the case, authors with published scientific articles on media violence are in the minority. Those arguing the side claiming that violent video games are not problematic have a smaller percentage.

Still, most of the people writing briefs don’t have scientific expertise on the subject, which in and of itself is worrying.

But is comparing the percentages valid at all? Who decides who gets to submit amicus briefs? If briefs are submitted voluntarily, there could be any number of biases in the generation and selection of briefs.

Another piece of evidence that is considered “strong support” is by analyzing the impact factor of the journals the brief authors have published in. Impact factors have many problems, but their use here is weird. Again, the authors are not examining the impact factors of journals that published articles on media violence (as far as I can see), but whether the amicus author had published in high impact journals, ever.

Sorry, but that is not “Strong support for the argument that video game violence is indeed harmful.” It’s barely support at all. If they had said, “Supporters for laws limiting violent video games have more expertise than those opposing such laws,” there would be no problem.

It would have been better to look for peer-reviewed articles cited in the briefs. Than start rating those articles for the quality of their evidence, using basic criteria like:

• Is the paper actually about video games? (I.e., is it relevant?)
• Was it a randomized, double blind experiment?
• How big was the sample size?
• How big was the effect size?
• Has the finding been replicated?
• How often has the paper been cited?

Credentials are important, but they shouldn’t be a substitute for evidence.

Caveat! The paper that this research will be described in will not be published in May. It is possible that the actual research is better than the story in Science Daily (which I’ve been baffled by before).

Hat tip to Julie Dirksen.

A-B-C-D

Boscoh at the Trapped in the USA blog laments a perceived lack of curiosity.

Many researchers become masters of their own research field but have minimal interest in areas outside their own. For such scientists, it is the qualification of knowing that counts, not the knowing itself. We do research so that we can publish prestigious research so that we can be recognized as prestigious scientists. Anything other than this is a sign of amateurism. Perhaps it’s a way for socially awkward people who played too much dungeons-and-dragons in high-school to claw back some kind of respect in a hostile world.

Speaking as socially awkward high schooler (and undergrad, and grad student, and...well, you get the idea)...

It’s one thing to be interested. It’s another to get the job done.

David Mamet has a memorable chewing out scene in Glengarry Glen Ross, where the character of Blake yells at his team (edited to be work safe):

Because only one thing counts in this life! Get them to sign on the line which is dotted! You hear me...?

(Blake flips over a blackboard which has two sets of letters on it: ABC, and AIDA.)

A-B-C. A-always, B-be, C-closing. Always be closing! Always be closing!!

For science, it’s A-B-C-D. – Always Be Collecting Data. The point of the exercise is to generate new knowledge. To do that, you need data. Getting data takes focus. It takes discipline. As New Scientist helpfully pointed out earlier this week, science is boring. A nearly obsessive desire to complete a task is sometimes what it takes to do the job. For an extreme case, listen to this interview about Grigori Perelman, the recluse who solved the Poincaré conjecture.

For instance, one of the hardest things for me to do is run replicates, because I know what the answer is going to be. Perhaps it indicates I am more interested in qualifying of knowing rather knowing. But it is necessary.

I say this as someone who agrees with Boscoh. I have taken pride in the number of different species have been featured in my research. I joke that have scientific ADD. I’m a dilettante.

But I shouldn’t pretend for a second that it hasn’t cost me.

If someone were to come up to me, like Mamet’s Blake, and say, “I have a continuously funded R01 at a major research university. How much did you bring in? You see, pal, that’s who I am. And you’re nothing,” I might call him rude, and say that’s not what matters to me (which are both true). But I have to admit that such people are often very good at what they do. They get the data. A. B. C. D.

A lot of students are also dilettantes. And it’s no surprise that many students have problems making the transition to professional scientist, because being a professional requires a certain clarity of purpose to getting the job done. Some are able to keep that bit of breadth, but professional science, like selling real estate, is about closing. The great ones finish.

Focus and discipline and getting the data out the door are not the only skills a scientist should have. There is a huge advantage to reading widely, and to having different points of view. But focus and discipline should be respected.

Geek space

From Finite Attention Span.

Science is my sword

Previously, I argued that science was never meant to be the reserve of the super intelligent. How did it become viewed as something that was elitist instead of democratic?

Science is like a sword: effective in the hands of the most rank and unskilled amateur, devastating when wielded by a skilled master.

It’s pointless to ask if successful researchers owe more to scientific methods or their intelligence, since those with both will outdo those who have only one.

Science has become practiced by a few by virtue of its own success. As the questions to be tested have become ever more subtle, it has taken more and more time just to establish the working knowledge necessary not to duplicate previous work. Also, the types of equipment needed to answer many of these very subtle questions are often beyond the realm of what most people are able to get their hands on.

Even MythBusters, one of the best examples of DIY science there is right now, often has resources beyond what the average person can cough up. Sure, building a giant ball of Lego is easy in principle... until you realize how many pieces are needed. Just because you know what experiment to do doesn’t mean you can pull it off.

Truth for the hard of thinking

Science is hard, according to common expression. After all, when something is easy, what is it compared to? “It ain’t rocket science,” which ends the race in a dead heat with, “It ain’t brain surgery.”

And this feeds into people’s feelings of inferiority where science is concerned. There is this idea that science can only be practiced by the very bright.

This is strange, given that arguably, science was conceived as a way that anyone could find truth about matters. Let’s compare science to the other methods of finding truth that preceded it.

Philosophy was a way of getting at truth that was always seen as an intensely rarefied and intellectual endeavor. In other words, it really was only for smart people. The first Western philosopher, Thales (pictured), was said to have fallen into a well while contemplating the stars. The maiden who rescued him asked how he could know what was in the heavens when he did not know the ground at his feet.

Religion was another way of uncovering truth. But much religion revolves around revelation. Those not blessed with revelation had to hope to be blessed with faith. But those not blessed with faith were kind of out of luck.

Science was a way for both smart and stupid people to get at the truth.

Not very bright or blessed by the divine? Here’s what you do. Look for patterns. Stick to numbers and things that are verifiable. Make predictions. Do tests. Compare the outcomes with predictions. Do that, and you will get answers to your questions. In other words, science spelled out methods that anyone could use. In that sense, science is emphatically not the elitist exercise it’s come to be seen as.

How did science become seen as something that only geniuses could do? That’s some idle speculation for another post.

What are you good at, and what do you suck at?

Hot on the heels of my Intellectual Styles post comes, "What are you good at?" seen at Sciencewomen and Uncertain Principles.

You are in a room with a bunch of other female faculty/post-docs/grad students from your university. You know a few of them, but most of them are unfamiliar to you. The convener of the meeting asks each of you to introduce yourself by answering the following question: "What is one aspect of your professional life that you are good at?"

While I think it's important to recognize what you're good at, I think the flipside is also important, so I'll throw this out for comments:

What are you good at, and what do you suck at?

Professionally, I think I'm pretty good at presentations, writing, and graphics. I suck at routine organization and cleaning.

Intellectual styles

I'm fascinated by how scientists have different intellectual styles. Some are methodical, others disorganized; some are great in the lab, others excel at interpreting data.

This post at The Quantum Pontiff reminded me of this question, presenting an idiosyncratic listing of different types of intellectual styles.

I became aware of different intellectual styles when one of my professors told a story about how several grad students were sitting around a table, chit chatting. One said he loved coming up with the questions to ask. Another said she loved the actual experiment, when the possibilities seemed infinitely variable. A third said, “No, it’s the numbers,” and enjoyed the process of running through the data, looking for the patterns. And the last enjoyed the writing, the pulling all the threads together into a final form.

Then they looked at each other and agreed that they should all write a paper together.

Another neurobiologist I met said the thing that he loved the most was listening to action potentials on an speaker. “I never get tired of that.”

The thing that keeps me going are the beginning and ends of the process: I love the ideas, and I love communicating them. The actual gathering of data, I have to admit, I sometimes find a bit of a grind. I particularly get frustrated running replicates. I know they're important, and I know I have to do them, but typically, when I running a replicate experiment, I usually have a very good idea of what the answer's going to be.

For my fellow scientists: What’s your favourite part of the research process?

Fail, 8 December 2008 edition

One of my students is trying to share some work with me through Google Docs. I try to login, and can't. I contact the specified email address. I am told, "You have to share the documents with their @gmail.com email addresses."

He apparently hasn't looked at the login screen.



Fail.

Even the might of Google is no match for institutional cluelessness.

How I'd love to be shown up

Sometimes I get tired of being the alpha geek.

I keep hearing about how my people the age of my students are all so wired and comfortable with technology. Then I ask my students to do a couple of simple "push button" tasks on the web like creating an RSS feed, and it completely flummoxes them.

I feel validated by this article, which says:

(A)lthough young people demonstrate an ease and familiarity with computers, they rely on the most basic search tools and do not possess the critical and analytical skills to assess the information that they find on the web(.)

So instead of expertise, students have only a thin veneer of technical skills. I would so much like to learn things from my students -- at least occasionally -- about technical things.

Long live the scientific method

Chris Anderson provokes with an article titled, "The End of Theory: The Data Deluge Makes the Scientific Method Obsolete."

There's some interesting ideas, but the argument is based on a false premise.

This is a world where massive amounts of data and applied mathematics replace every other tool that might be brought to bear.

It's perhaps understandable that an outsider, a non-scientist would mistakenly believe this premise to be true: that there are massive amounts of data available for all scientific problems.

There are not.

There are only a few fields of science that generate large amounts of high-quality data. I'm thinking maybe some branches of physics (like nuclear physics, maybe astronomy), social sciences (demographic and census data, automatic tracking of web useage), and maybe genetic data for a select few animals (humans, mice, fruit flies, Arabidopsis).

These are the exceptions.

In most cases, scientists have to eke out by hand one experiment at a time. It's not automated, it's not massive, and it doesn't generate huge numbers. To take an example from my field, invertebrate neurobiology, there isn't really good agreement on how to describe neurons in such as way that they can be put into a searchable database (although the NeuronBank project is making an effort to at least think about that problem).

Anderson goes on to say:

There is now a better way. Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.

Scientific theories have three traditional virtues. Predict, control, explain. Massive datasets may indeed give us pretty good predictive power -- correlations often do. It may not give us control. And it certainly doesn't explain. We really need causal mechanisms to explain.

For instance, let's take climate change. If it were the case that massive data is all you need, there would seem to be no need for the ongoing debates about climate change. We have massive datasets there. And indeed, the scientific questions are supported by a large consensus. But people don't care that there's a correlation between carbon output and temperature change, they want to know if one is caused by the other. The policy decisions are very different depending on what your thinking of causal mechanisms are. Cause is king.

Make no mistake, automation changes things. But it doesn't change everything.

Expertise and belief

Two interesting articles that I want to point out.

The first is a Psychology Today article on magical thinking. This, to me, is another way of looking at the more fundamental question about the basic nature of belief, which fascinates me. Why do people believe something or not? Are there limits to what we can believe? And as a teacher, how can I affect what people believe -- or should I?

Second is a Time article looking at research into expertise. As a teacher, I want to pass on expertise. As a researcher, I want to hone my expertise. How do I do that?

Ericsson's primary finding is that rather than mere experience or even raw talent, it is dedicated, slogging, generally solitary exertion — repeatedly practicing the most difficult physical tasks for an athlete, repeatedly performing new and highly intricate computations for a mathematician — that leads to first-rate performance. And it should never get easier; if it does, you are coasting, not improving.

Expertise

As an academic, you're a professional expert. But sometimes you think, "But why does this take so long?" Bachelor's degree, graduate work, post docs more often than not. One answer may be because that it takes about ten years of hard study to become truly expert in anything, according to this article in Scientific American online.

That's the bad news. The good news seems to be that innate talent doesn't seem to be the deciding factor in developing expertise: continued study is.

On a somewhat related note, expertise -- not to mention passion and intelligence and lots of other good things -- are on glorious display at the TED website. The talks are wonderful, sometimes astonishing, must visit links. The blog has plenty of excellent tidbits as well.

Science is a doozy

I was channel surfing a couple of nights ago, and came across a show about Jay Leno’s discovery of a rare, vintage Duesenberg car (described by Leno himself here). Apparently, Duesenbergs were so opulent that the name was the origin of the term “doozy,” meaning extravagant. During the show, Leno was explaining the difference between early cars and those made now, and said, roughly, that it used to be that technology was expensive and labour was cheap. Now, technology is cheap and labour is expansive.

It occurred to me this morning that science is one of the few areas in the modern industrialized society where that isn’t true. The production of scientific “product” (data) is generated by expensive technology and cheap labour. Even standard pieces of equipment will often cost tens of thousands of dollars. Most actual research is physically carried out by graduate students, whose pay sucks. Because of that, I’ve read articles calling successful scientists modern “plantation barons.”

The most successful areas of biology right now are arguably cell and molecular biology, and I don’t think it’s any accident that those are areas where the most automation has occurred. I’ve talked before about robots that can run experiments, for instance. I wonder if other areas of biology can catch up. In my own area, neurophysiology, the task of placing electrodes and getting recordings is sufficiently delicate that automating data collection seems a long way off. Automating data analysis, however, is more feasible. And I worry about whether those areas of science that have the luck of being more automated are going to out-compete those kinds of science that aren’t able to do so. Er. Perhaps I should say more than they're already out competing those non-automated sciences.

Of course, making that transition in the manufacturing industry (particularly the automotive industry) was not easy. I remember a lot of grief kid over massive layoffs and job becoming obsolete. Perhaps one good thing is that because the cheap labour in science is driven by short-term labour (students), rather than people who were counting on a particular industry to provide a livelihood for decades.

Photo by Roman Boad on Flickr; used under a Creative Commons license.