29 August 2021

Kill the chalk talk?

A “chalk talk” is an unscripted presentation that allows a job candidate to outline something “on the spot,” live. In many departments, this is supposed to be a chance for a candidate to talk about their research and how they plan to create an ongoing, funded research program.

Since chalkboards have mostly been abandoned for years, the term probably only still exists because it rhymes. In fact, in engineering and computing business and industry job interviews, these are called whiteboard interviews. But there, some have pointed out that whiteboarding is a problematic interview practice.

It’s got nothing to do with the skills needed to succeed at the job.

Wayne Brady
Asking a scientist to try to create a grant proposal on the spot is like asking a film actor to do an improv comedy scene as part of an audition. If that were the norm. Wayne Brady would probably be the biggest start in Hollywood. Sure, some actors are great at improv, but that’s generally not how film works. Films have scripts and preparation time. 

Chalk talks are likely to be one of those common practices that contribute to academia’s low diversity. It seems likely that there could be many people who are thoughtful but slow to process questions who might fare poorly in a chalk talk. On the other hand, there may be people who are exceedingly good at oral explanations but who are undisciplined writers. 

And if this is all allegedly about grants, grants live or die on the page, not the stage. Besides, job candidates always send a research statement as part of the application package. So marking someone explain it all again feels more like hazing than a useful way of assessing a candidate.

Departments don’t need these. I witnessed a fair number of academic job interviews in at least three institutions in as many countries. None of them asked a job candidate to do a so-called “chalk talk.” 

Update, 30 August 2021: Unsurprisingly, some people disagree with this. Jason Rasgon offered this defense (I’ve stitched together a few tweets):

Chalk talks have some value for a very narrow slice of institutions (generally soft money Medical or professional schools). I agree, mostly useless for other places. But when I was soft money I saw many faculty applicants who had a written document who couldn't articulate a plan of attack at the chalk talk. Several of them were still hired, they didn't last. When you’re soft money (the institution pays little to no salary, you have to pay your own salary by your own grants - ZF), you don’t have luxury of taking your time to organically develop a research program. You have to hit ground sprinting from day 1. Chalk talk should be easy, as you should have all this developed (and maybe submitted already). If you don’t, you’ll have problems. The people I saw have issues knew in general what they wanted to do but hadn't sat down and worked out the specifics.

Michael Eisen writes (again, stitching tweets together):

I’ve attended over 100 faculty candidate chalk talks, and would say they've played a significant role in hiring decisions maybe a dozen times, all in the favor of the candidate, because they gave us insights into strengths that did not manifest in their application. In contrast to the dominant opinion, which seems to be that chalk talks are about gotchas or testing peoples' ability to think on their feet, in my estimate their major role in the process is to give people an opportunity to address weaknesses in the rest of their application. When the format and purpose is clearly articulated, and when individual faculty are not allowed to derail the process (which can happen), they are an immensely useful part of the process. ... (T)he call for eliminating chalk talks is a copout that doesn't actually address the problem, and is part of a bad trend to provide candidates with fewer ways to demonstrate their qualities

I’m thinking about these points. I appreciate what they are getting at, but I think the qualifiers in these are hugely important. “When the purpose is clearly articulated” is my big one, but “when not derailed” is also up there.

We want job candidate to interact with people and answer questions about their research, but it seems unclear what people are trying to assess and whether this is the best format to do it.

Eisen is right that if you remove part of the interview process, you have less information about the candidate. So re-evaluating the chalk talk concept should also include “Can we come up with something that more directly evaluates the relevant skills?”

If the goal is to evaluate someone’s research, maybe those statements of research interest that is required of every job application should play a bigger role? 

A written statement better reflects grant writing. It’s faster and more time efficient to read a few pages than listen to hour long talk. And it’s less stressful for candidate.

If the research statement that was part of the initial application is too short, maybe creating a beefed-up version can be part of the shortlist interview process.

More additional: Later, Eisen wrote:

i actually wish we had prospective candidates teach a class, but that's obviously a bit too much to ask of them
For many institutions, asking a candidate to prepare a research seminar and a guest lecture in a class is probably more helpful than asking a job candidate to effectively give two research talks (seminar and chalk talk).

Andy Fraser wrote:

I have seen candidates with stellar CVs that gave extremely polished research talks that apparently could not cogently discuss a single part of their research proposal or who reacted with hostility to questions. I don't know how else I would have found that out.

In light of the excellent commentary, I have added a question mark to the title of this post. 😉

External links

It’s time to end whiteboard interviews for software engineers

14 August 2021

Are we trying to solve the wrong problem with anonymity in peer review?

One of academia’s evergreen topics, anonymity in peer review, has cropped up again on Twitter. 

This is something I’ve changed my mind about a lot since I’ve been blogging. But today, the thought struck me that we put too much emphasis over whether review should be anonymous or not.

We do not put enough emphasis on people not being assholes about getting reviews.

The whole thing driving arguments around anonymity is, as far as I can see, driven by fear of retribution. “If I write a bad review of someone else, they will be petty and have the ability to sink something of mine later.”

As much as I appreciate the argument that people are horrible (they are) and you have to account for things that people regularly do (people are often petty and vindictive), it seems to me there is a lot of possibility to move the dial on author behaviour. To say, “We won’t put up with your little revenge plots.”

Do I know how to do this? No, not yet. I’m just blogging here.

It’s also likely that a lot of bad behaviour here is driven by the sense of research being a zero sum game. Which, in the current funding climate, it is dangerously close to being. If there was no so much competition and windows of opportunity were not so small, people would worry less about whether some jerk is going to try to give you a bad review on a grant because you were critical of their manuscript.

10 August 2021

Thanks to you, ComSciCon!

Zen Faulkes on Zoom screen with Godzilla in background.Just briefly wanted to follow up on my appearance last week at ComSciCon. The storytelling panel I was part of was excellent (despite my presence) and I was also able to work with small group on short journalism pitches. 
There was not a coherent thread or panel hashtag, so here are tweets from the storytelling session. (Darn it, I miss Storify.) These are in roughly chronological order, and I would normally cut and paste the tweet text, but I have some other tasks to do and it would take too long to make it pretty right now.

Thanks to the organizers for posting a non-derpy Zoom screenshot of me!
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Related posts