Women Get Cited Less. What Can We Do About It?

A few weeks ago, a paper came out about the fate of research papers in ecology and evolution (my field!) pre- and post-publication, comparing outcomes between male and female authors.

I want to focus on just one aspect of their nuanced analysis (you can read the whole paper here for free). In this section, the authors gathered citation data on over 100,000 papers published in 142 journals in our field.

They found a slight, but significant, difference in how often papers by men and by women got cited. On average, controlling for the impact factor (a proxy for quality[1]), papers by women accrued 2% less citations.

A hundred thousand papers! That’s a lot, and it’s why the authors could detect this small difference. The sample size allowed them to do a quite powerful analysis.

It also revealed some interesting interactions. For example, papers with female last authors (which often indicates seniority or leadership of the research group) were cited less than those with male last authors at high impact journals, but the effect was reversed at low-impact journals.

Because more people cite papers in high-impact journals, this meant that overall, women were cited less often. [2]

I guess this shouldn’t have been a surprise, but it’s not something I had seen data about before, and consequently something I hadn’t really thought about.

But it is something that matters. Like it or not, citations are a metric of success that is easy to measure, and therefore whether others cite your work is a good piece of evidence that you’re a valuable scientist when you’re up for a job position, tenure, or an award. It’s not great for women if there is bias preventing them from doing well on this metric. [3]

Sounds About Right

Like a lot of research and headlines about challenges facing women in science, this got me mad. I’m a feminist, and this stuff pisses me off.

I see the experiences of my female friends and colleagues, and see when they are treated differently than their male peers. Not always, of course, but enough to make a pattern of our own anecdotal experience.

Then there’s the fact that in my study topics, almost every giant in the field is a man. The ones that defined the discipline and published the equations in a top journal? Men. There are women there, doing great work, but they are less famous.

When data on various aspects of academic life backs up our experience it’s like, “yup, sounds about right.”

Academia is harsh on most people; it’s a place with a lot of rejection, high standards, low pay, job insecurity, and so many power dynamics. The fact that academic science is even harder on women is just not fair.

And that’s not getting into the more complex and devastating nature of the structural problems in academia, which are even worse for other minorities, especially minority women. This paper found that on average women are cited 2% less than men; how much less often are minority women cited?

Anyway, I read this paper, and I was mad.

What I Did Next

Being mad doesn’t accomplish all that much. I tried to think about what we, in our daily lives as scientists, could do to work against this problem.

As an individual early-career scientist, I can only do a very little bit about peer review outcomes or paper acceptances.

But citations? That’s different. I am always writing papers, and I am always citing others’ work. I realized that this was a small step I could take to try to contribute to supporting female scientists. It sounds trivial, but I can cite their work. Could we all do this?

I brought this idea to our lab group, and was curious what they would think.

Some colleagues immediately recognized that this was a problem, and something we don’t think about enough. There are a handful of big names in our subfield, and we mostly cite them over and over. But do we need to do that? Are their papers that we cite every time actually the most relevant? Maybe not. There is probably a lot of work being done around the world we could cite that instead, or at least in addition to the now-traditional canon.

Another common reaction was for someone to say that they don’t think about the gender of authors when they search for papers, read them, or cite them. Here it diverged: a few people said, “I don’t think about it and now I realize that I should.” Others said, “I don’t think about the gender of authors when I’m citing them, therefore I’m not part of this problem.”

I challenged this statement. Does that really mean you’re not part of the problem? Maybe not, I said – and if not, that’s great for you, good work! But instead of assuming that not consciously citing more papers by men means no harm is done, check your reference list on the paper you’re writing. What’s the author breakdown? Do you think this strategy is really working?

I don’t think I was popular for making this callout.

Rubber Meets the Road

As I mentioned in a tweet a few weeks ago, “caring means walking the walk.”

In my paper-reading project, I track the gender of the authors I read, and I found that I am reading fewer papers by women than by men. I was curious about what the ratio might be of the papers I end up citing.

I went through the references section of my current manuscript with a blue and a pink highlighter (I know, supporting stereotypes, etc., it was lazy). The results were not pretty.

I was frankly surprised at how few of the papers I had cited were by women. I knew it wouldn’t be 50/50 for a lot of reasons (discussed below), but I didn’t think it would be that bad.

I basically proved to myself that in order to cite women, you have to do it on purpose. Just not intentionally excluding them isn’t enough.

It’s like how colorblind policies don’t work. Not intentionally doing harm is not the same thing as not doing harm. As Evelyn Carter writes about claiming to be colorblind with respect to race, “if you ‘don’t see’ race, but you say you care about inclusion, how can you advance inclusion efforts that will effectively target communities of color?”

There is a lot of unconscious bias and systematic barriers that lead us to contribute to inequality. Working for equality and to recognize contributions made by women and minorities means actively working to overcome those unconscious biases and systematic barriers.

Shifting the Balance

After my discouraging experiment with the highlighters, I went through the paper and looked at the places I had cited different work. In some places, I was able to find a paper by a woman that I could cite instead – and often even one that supported my point even better than the paper I had cited originally.

That was the most delightful aspect of this task I had set out for myself: I discovered papers in my core area of study, that were by women and that I had never read. And they were really good and very interesting!

The point of reading and citing work by women isn’t just to check boxes and give women a fair shot at career metrics. The main reason is to do better science. Science is creative; it involves having ideas, being exposed to new things, thinking outside whatever box you’re in. Reading work by more different people will necessarily help that process.

I’m really glad that I found these new-to-me papers.

As I discussed with a different colleague later, a lot of this issue comes down in part to poor citation practice, which is endemic across academia. We cite something because everyone else cites it, even though maybe we haven’t read the whole paper. Or we cite something because it’s already in our reference library and we are in a hurry. I’m completely guilty of this, and I’m quite sure everyone else I work with is, too.

If we spent more time reading, and more time looking for and getting familiar with other work that’s related to what we’re doing, I think all of our papers would have much more diverse author lists – at least in terms of evenness and not being dominated by the famous people in our field.

Our papers might also just simply be better, because of all the ideas we would be having.

Don’t Worry, I’m Still Citing Men

After this citation overhaul, my reference list was still majority male first authors. You need to cite what is relevant, and in a lot of cases this work is by men. One reason is that over the history of ecology, there is a lot more work published by men than by women, especially the farther back you go. [4]

Plus, if you need to cite a classic paper, it doesn’t matter who it’s by. That’s the one you need to cite. That’s a second thing. [5]

And likewise, if you need to cite something about your specific study organism or system, there might be only a handful (or a few handfuls) of people in the world who publish on this very specialized niche. They are who they are. If you need to cite peer-reviewed literature, you may have limited choices, and you need to cite the best and/or most relevant work out of that array. [6]

So there are a lot of reasons you can’t just take your reference list and manipulate it towards 50/50 gender equality. I want to make it completely clear: I am not advocating for a departure from citing good and relevant science!

When I mentioned this idea to colleagues – that citing women was a seemingly small, basic thing that we could do in our everyday lives as scientists to make a difference in structural biases – some were deeply uncomfortable that if they sought out work by women, then they would have to leave out other papers.

Listen: there is so much research out there, it boggles the mind. It’s growing exponentially. It’s insane. And in no paper do we cite all the possibly important research on a topic. We’re going to leave things out anyway. It’s just that now, we seem to be structurally leaving out work by women. Again, to overcome this bias is going to require intentionality. This is not a problem that we can just hope will go away because we are good people and don’t mean to cause harm.

No matter what we do, we’re going to keep citing a lot of great research by men.

Tokenism?

Among women, there was another layer. Many of the women I talked to did not want to be cited just because they were women and someone needed to move their reference list towards gender parity. They wanted to be cited because someone genuinely thought their research was the best and most relevant.

And I get that. I was invited to give a talk once because the organizer was looking for a replacement female speaker after the originally-invited woman couldn’t participate. I was so excited for this opportunity, but at the same time it felt weird to get it explicitly because they were looking for a woman.

I’m not sure what to do with this concern, but I think it comes back to proper citation practice. Is your work relevant to the topic, and being cited correctly? Then you deserve to be cited. If papers by women are accepted less often and cited less often, then part of the reason you are not currently being cited might be simply because someone isn’t familiar with your work.

And The Inevitable Question, Does Quality Limit Equality

Finally, we had some conversations about whether this might be because women’s work just wasn’t as good as men’s.

It’s pretty hard to assess that (although the paper I started this blog post with tried to, by using journal impact factor as a covariate).

I have two thoughts. One is that I doubt it’s true. There was an interesting graphic and paper that went around Twitter – about economics, not ecology – showing that papers by women are better written than those by men, that women incorporate reviewer comments more often, and that they improve at presenting information over the course of their careers.

It’s provocative and I’m not sure if it’s also true in my field or not, but I believe it. I think we are culturally trained from a young age to try to please, and so we might be likely to try to pacify reviewers, and to make a revision extra perfect if it was rejected the first time around.

Also, in the sciences, women have to be better to be evaluated as being as qualified as men.

Second, about the content. If it is possible that the data and experiments presented by women are less strong, why might that be?

It is interesting to think about how structural problems would lead to this. Could it be because women get less funding to do their research, and thus might have less resources or support? Could it be that women are asked to do more departmental service and teaching, and have less time to do research?

If the research really is worse – which I’m not convinced it is, but there’s little way to objectively assess, at least not for a dataset of any size – this very well might be a result of the same structural issues that cause the citation patterns.

What Next

What do you think about this? Are there any other ways to work on this problem?

Notes

[1] Impact factor is not a perfect measure of journal quality. It is an estimate of how often work there gets cited, which is a traditional metric for how important it is. For any individual paper, the impact factor of the journal it is published in doesn’t say much about the quality of that paper. However, I think it was the best covariate the authors could find to control for the differences in quality between papers when comparing men’s and women’s outcomes. Also, for better or for worse scientists pay attention to impact factors, so it may affect citation practice even if it’s not an actual metric of “quality” per se.

[2] I have an interesting, harebrained idea about this: if papers by women are less likely to be accepted, do good papers with women as last authors end up in lower-impact journals? That could explain why the better-cited papers from those journals are by women. Totally spitballing here.

[3] Citations shouldn’t define one’s value as a scientist, colleague, or employee anyway, but… that’s a discussion for another day.

[4] Is it really though? Women probably contributed to many important ideas. They typed the manuscripts. Maybe they did some of the work. There are tons of cases in science where people had the same idea independently, and one person got famous, and the other didn’t. Sometimes both these people were white men. I’m guessing there’s a lot of times when some of these people were women, minorities, scientists from outside Europe and North America, and people who were/are otherwise excluded from the elite science community.

[5] Given what I just wrote, I think I – and we all – need to put more effort into finding papers that were written around the same time as the famous ones that “came up with” ideas, and represent contributions of the community-building of those ideas by other people.

[6] When you’re looking for very specific things, there might be a lot of important and relevant data in theses and lower-impact papers by students who did not continue in science. This is valuable literature to search for, and might expand what you think the contributions of women and minorities are, given that these people are less likely to continue in academic careers either by choice or by exclusion.

Book Review: Spying On Whales by Nick Pyenson

My aunt sends us books for Christmas every year, a.k.a. the best kind of holiday presents. In this year’s box was Spying on Whales by Nick Pyenson, and I sat down to start reading it that very night.

In some ways, I’m the perfect audience for this book. I’m interested in nature, natural history, and evolution, but I actually didn’t know much of anything about whales.

img_3489

Whale territory. You just can’t see them because they are under the water – as Pyenson discusses, this is one of the things that make these giant animals so mysterious and compelling.

In November I had gone on a whale-watching boat ride in northern Norway, and more or less everything I knew came from the Ukrainian guide on the boat. It wasn’t much, but before and after the trip, I thought whales were cool, when I thought about them at all.

If you are already a little obsessed with whales, then I think you’ll like the book too. All the facts are there for you to nerd out on.

9780735224568But Pyenson writes impressively well, and the book is never bogged down in this nerdery. Instead, it’s fun and adventurous. Chapters sometimes end in cliffhangers – I certainly couldn’t stop reading. And at times, it’s quite a moving read.

In the prologue, Pyenson describes how the Voyager 1 and Voyager 2 spacecraft are carrying whalesong into the outer reaches of the solar system. We may not understand what whales are saying, but we intuitively feel that they are intelligent and powerful – and that their communication is not only beautiful but probably important.

“They’re so compelling that we imagine aliens might find them interesting,” Pyenson writes of whales.

And yet, he continues, we know very little about whales, either the ones that live in our oceans today or the whales of the past.

I’ll admit: I didn’t actually really realize that.

Pyenson fills in details that I never knew I didn’t know, and certainly didn’t realize were so recently discovered. He discusses what things we still don’t know, and what mysteries will be exciting to solve. This book is filled with behavior, anatomy, and evolutionary history, as well as discussions of how all three connect to make a modern whale.

“Nothing in biology makes sense except in the light of evolution”, foundational biologist Theodosius Dobzhansky wrote as the title of one of his essays.

In my fields – ecology and evolutionary biology – this is tossed back and forth. “Nothing in ecology makes sense except in the light of evolution,” an evolutionary biologist will say, scoffing at ecology as boring (yes, this has happened to me, in person).

“Well, nothing in evolution makes sense except in the light of ecology,” a ecologist might retort.

Pyenson’s book is a great demonstration that both are true. In trying to piece together – literally, based on fossil fragments – how whales went from living on dry land to being the biggest animals in the oceans and indeed on the planet, ever, both ecology and evolution are needed. The two are woven together throughout the book as Pyenson describes working with one collaborator after another.

“The trip was an opportunity to push at the questions sitting on the edges of our disciplines, at the intersection of Ari’s understanding of behavior and local ecology, Jeremy’s grasp of physiology and biomechanics, and my background in paleontology and Earth history,” Pyenson writes at one point. “The basic questions about how whales do that they do require all these fields of understanding, and I’ve always thought the best way to answer them was through this Venn diagram of disciplines and personalities.”

Now that’s a scientific approach I admire, and he demonstrates that it’s fun, too. Minus the part at the whale slaughterhouse.

How does this approach deliver answers? Here’s an example question. If whales have gradually evolved to be bigger and bigger, why aren’t they even bigger still? That evolutionary question has an ecological answer.

The first two-thirds of the book are about the past and the present of whales. But my favorite part was the last third, about the future of whales, and of us. Whales live a very long time, sometimes two hundred years. There is a beautiful description of all that a 200-year-old whale has seen in its lifetime.

And then, a description of what today’s whales might see in the future, things that you and I will not live to see.

“Any bowheads living today do so in a liminal gap between a familiar past and a potentially unrecognizable future,” Pyenson writes of climate change in the Arctic. “A bowhead calf born today will live in an Arctic that, by the next century, will be a different world than that experienced by all of its ancestors.”

Maybe that sounds trivial – like, duh, climate change is happening fast – but after reading a hundred pages about all of the different environments that whales have encountered through their long evolutionary history, it is powerful when you reach this point.

And that is one of the things that kept me reading this book. It is peppered with fun facts – large baleen whales grow at a rate of a hundred pounds a day before reaching maturity, dang, that seems impossible! – but also with profound observations and good writing.

Finally, as a field ecologist, I really enjoyed Pyenson’s descriptions of fieldwork trips. He takes us to Chile, Panama, Iceland, and Antarctica, as well as more familiar-to-Americans spots like California and North Carolina. All of his descriptions of the challenges and successes of fieldwork, the camaraderie between colleagues, and even that one wonderful-but-definitely-weirdo collaborator (we all have at least one), feel authentic.

If you like science and nature writing, and compelling nonfiction in general, I think you’ll like this book. I did. Thanks Lizzie!

another paper.

In case you missed it – we had another paper published! I’m really happy that things seem to be getting out there, little by little. Of course, about the same day, I also got some comments from co-authors on another manuscript which left me with tears of frustration thinking of all the months I had spent on this darn paper, only to have to basically start all over again…. if only I had known what they thought earlier! well, such is life.

At least we have one more published to celebrate.

Here it is (open-access, free for anyone to download): ta-dah!!