Promoting Open Discussions of Scientific Failure within the Annual Society for Neuroscience Conference

Context

SfN has featured a formal storytelling session as part of the annual conference for almost a decade. These storytelling sessions are intended to humanize science and convey its importance in a compelling way. Our storytelling session “Oh Sh*t: Great Scientists Tell Stories About Their Greatest Failures” was featured as part of the 2021 SfN conference.

Inspiration

We were inspired to run a storytelling session about scientific failure at SfN by a global movement within the entrepreneurial community called the “F*ckUp Night” (https://fuckupnights.com). At these events, powerful people share their personal stories of failure to pass on wisdom to younger members and encourage them not to give up in difficult careers.

Vision

We hoped that listening to well-established scientists open up about their struggles and mistakes would reduce stigma and, ideally, initiate a ripple effect, inspiring conversations about failure throughout the rest of the conference, in socials and luncheons. We designed the event to follow a format akin to the live storytelling events hosted by The Moth, (n.d.), with each storyteller taking the stage, stepping into the spotlight, and—with just a microphone—diving into their personal story. No slide presentation, just raw honesty about their lived experience of doing science.

Logistics

The storytellers were recruited via e-mail, social media, and word-of-mouth. We targeted scientists who were either at the top of their field or rising stars, with the goal of sending a clear message to the audience that even highly successful scientists make big mistakes. Due to the nontraditional format, we also looked for individuals who were successful as scientists in part due to their storytelling ability, or who were known to be eloquent, insightful, genuine, wry, or hilarious—individuals who were secure in their own skin or known for their introspection and frank honesty. We also sought individuals who had been intensively involved in the mentorship of younger scientists, winning teaching awards or serving as directors for local undergraduate or graduate programs.

Recruitment turned out to be remarkably challenging—even the most famous and confident scientists were (understandably) uncomfortable at the prospect of sharing their most vulnerable moments in a professional setting in front of a large audience of their peers, within a nontraditional, unfamiliar format. Moreover, as a high-profile event focused on failure, it was absolutely essential that the line-up of speakers be properly balanced in terms of gender, race/ethnicity, geography, and field of study, so that it would not be implied that scientific failure (or greatness!) was dominated by a particular group of people. In the end, the finalization of a line-up of six storytellers required approaching more than one hundred prestigious, selectively-targeted, scientists.

To prepare storytellers for the event, we provided guidelines adapted from The Moth (n.d.) for constructing a good story, with additional guidance on how to avoid some potentially dangerous pitfalls while telling personal stories in front of a professional audience (Box 1). We also provided the storytellers with examples of compelling storytelling from previous neuroscience events (The Moth Presents Wendy Suzuki, 2015) and met with each storyteller individually to discuss the outline for their stories.

Impact

The storytelling proposal was accepted as a 50th anniversary event for SfN 2020. However, due to the pandemic, the event was rescheduled to 2021 and then, sadly, made remote. Despite this disruption, virtual attendance at the event numbered in the thousands and included a wide swath of the SfN population. There was a silver lining to the remote format—because the stories were pre-recorded, we were able to create a YouTube channel (Great Scientists, Great Failures, 2021b) and share the stories with a wider audience, receiving an additional ∼5,300 views and 250 subscribers. Once the channel was created, additional brave scientists were invited to record their stories, adding two more stories to the channel that have received >1,100 views.

Overall assessment

This was unquestionably a high-effort, high-impact endeavor, requiring dedication on both the part of the organizers and the storytellers. As expected, we found that a high-profile storytelling session attracted a broad audience, providing a useful starting point for initiating conversations about failure. The stories within this particular event tended to focus on experiences with professional failure—failed collaborations, misguided mentorship, rejections, difficulties starting up a lab, and struggles with funding and self-worth. This emphasis meant that the stories were most useful for promoting resilience and mental health in the neuroscience community, and for serving as memorable professional training. Future storytelling events could focus on the science itself—failed replications, heavy criticisms, being scooped, experimental or theoretical mistakes—providing lessons in experimental troubleshooting, and encouraging the sort of scientific transparency necessary to improve research reproducibility and the revisions required for scientific progress.

We also found that the storytelling format was translatable to a remote setting, although video recordings lack the collective energy and immediate feedback received while performing in front of a live audience. This remote format allowed greater worldwide access, as well as potential use of the stories in classroom and training situations. That said, as an educational tool, these stories are powerful, but the audience's role is passive, especially within a remote format, and therefore we recommend providing space for facilitated post-event discussion.

Context

We developed the “Calculate Your F-Index” game as part of the interactive break-out session that we ran remotely over zoom immediately following the storytelling session at SfN 2021.This break-out session was intended to serve as a proxy for informal post-event conversation but ended up serving only a small number (<30) of participants. During the following 2 years (SfN 2022–2023), we ran the “Calculate Your F-Index” game in person as part of the SfN-sponsored social “The Confound Hour: Let's Make Some Noise!” (100–150 participants).

Inspiration

We were inspired to create a game that reframed failures as accomplishments by the online purity tests popular in the late 1980s/1990s (“Wikipedia | Purity test,” 2025). Within these games, participants receive a point for everything that they have done wrong instead of what they have done right. We named the game “Calculate Your F-Index” as a satirical reference to the common H-Index metric used to evaluate research productivity.

Vision

Although F-Index scores could be calculated independently by each individual, we designed the game to be run in a group setting, with each question presented on a slide, often with humorous visuals or cartoons. This upbeat, irreverent group context was generated to inspire public admission of fault—laughs, head nods, cheers—creating an atmosphere of solidarity that has some parallels with the bonding created by the common drinking game “Never have I ever …” (“Wikipedia | Never have I ever,” 2024).

Logistics

Between 2021 and 2023, we generated two versions of the F-Index game: (1) The 2021 version was used in the remote break-out session for the “Oh Sh*t” storytelling session and focused on scientific failure, broadly speaking, with questions overviewing a variety of professional and experimental issues (Box 2). (2) The 2022–2023 version was used in person within the SfN-sponsored social “The Confound Hour” and had a narrower focus on experimental failure and scientific troubleshooting (Box 3).

We generated and vetted the questions included in the F-index games with input from social media, most extensively by vigorous discussion within the Academic Research Moms private Facebook group, and tailored the final list of questions to evenly encompass failures common across career stages and representing the many subfields in neuroscience (e.g., molecular, behavioral, neuroimaging, computational, electrophysiology). Importantly, the game was designed to reward points cumulatively for failures experienced over the course of a scientific career. This meant that the game was rigged—inevitably the top scorers would end up being the most senior (and therefore, probably the most successful) people in the room, simply because failure is a common and essential part of science, and senior scientists have had more time to fail repeatedly. This game mechanic was used to visually dissociate failure from youth and naiveté and ensure that the audience came away with the feeling that their own failures were not special, but both a common experience and essential for scientific success.

There were some key differences between running the game over zoom (<30 participants) and in person (100–150 participants per year). We found that keeping track of points was easier over zoom, where participants had easy access to some means of keeping notes. Within the in-person setting of an SfN-sponsored social, the game was dependent on access to a functional AV system and felt a little more chaotic, with some individuals reporting difficulty tracking scores or becoming confused after wandering into the social mid-game. We recommend that future facilitators invest in temporary access to anonymous smartphone interactive polling software for large groups, such as Poll Everywhere or Mentimeter.

At the end of the game, we provided a poll illustrating the distribution of points (over the years, we have only seen one 0!) and then asked the top scorers to reveal themselves. We have rewarded the winner in different ways—in person, this often took the form of drink tickets—accompanied by applause and cheers. We found that it was useful to further ensure that the winner was recognized for their insight and persistence by giving them with the opportunity to make a short “victory speech” offering words of wisdom for trainees (“What advice would you give to younger scientists about overcoming failure? What helped you most?”). In 2021, we have also used the F-index scores as a contributing criteria for winning the “SfN “Stuck to It” Award” and being coronated with a physical golden duct tape crown.

Impact

In both remote and in-person contexts, the F-index game seemed to be an effective ice breaker, easing participants into admitting failure and quickly demonstrating that failure is common in science. In person, the game also created an upbeat atmosphere of solidarity and bonding.

Overall assessment

The F-index game is relatively easy to run, in both remote and in-person contexts, with access to common tools (zoom, AV equipment, polling software). The questions that we used for a neuroscience audience (Boxes 2, 3) could be easily adapted to other scientific fields. Most notably, the game scales effectively—it is as easy to run the game in a small group as in a very large group. That said, the game lacks the depth of empathy generated by detailed storytelling or confessionals and may be best paired with facilitated discussion.

Context

We developed Science Failure BINGO as part of the in-person SfN-sponsored social “The Confound Hour: Let's Make Some Noise!” at the 2024 annual SfN conference (∼50 participants).

Inspiration

We found that the F-index game was good at getting participants warmed up and excited to share stories about scientific failure, but it had the downside of being mostly noninteractive. To better encourage interaction, in 2024 we adapted a common icebreaker game Human BINGO (wikiHow, n.d.) to include a wide assortment of scientific woes (e.g., confounds, artifacts, lab accidents, disastrous mistakes, scientific fiascos, etc.).

Vision

The goal was to encourage participants to mingle and find people who had experienced the assorted scientific woes listed on their BINGO cards. Participants were allowed to only mark off one box per person, forcing them to meet and talk with each other. The first participant to mark five boxes in a row (horizontal, vertical, or diagonal) would then shout “BINGO!” and win a drink ticket. Our hope was that the game would turn the norms of conference networking solidly on their head, as the game quickly makes the individuals in the room who are the most open about their failures the most socially valuable. Following the BINGO win, we also allowed participants to cash in on their own woes by giving themselves 1 point per “Yes” for all of the struggles that they had personally experienced on their own cards. The participant with the highest score was then declared a winner and given a drink ticket, allowing participants to celebrate their own failures as well as those of their peers.

Logistics

To create the BINGO cards, we adapted the content of the F-index game to an abbreviated format, making sure that there was a decent mixture of low-hanging fruit (“Cried in lab or office”), common mistakes (“Result turned out to be a bug in the code”, “Spill in the centrifuge: It's amazing what 12,000 RPM will do …”), and rarer catastrophes (“Set fire to the lab”, “Stolen laptop”), using humor when possible to maintain an upbeat atmosphere (“Spilled radioactivity, no superpowers yet”, “Labeled samples 1–20 and will definitely remember what they are”). We then used an online BINGO card generator to generate a set of 30 unique cards from our list of scientific woes (Fig. 1, a full set of the scientific failure BINGO cards can be generated and downloaded for free; myfreebingocards.com, n.d.) and printed multiple sets of the 30 cards with the assumption that the randomness of mingling would ensure that the experience of each participant filling out a card was unique. We also brought extra pens to the event but found that most participants had their own writing utensils.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Failure BINGO. Example BINGO cards resembling those used in the in-person SfN-sponsored social “The Confound Hour: Let's Make Some Noise!” (2024). A full set of BINGO cards can be generated and downloaded for free online (“myfreebingocards.com,” n.d.)

Impact

We distributed the BINGO cards when participants arrived at the door for the social and found that even before the game began, the cards were inspiring conversation. Once the game officially began, the room erupted into busy enthusiastic mingling. The game was actually so successful at encouraging interaction that the room was too loud to hear folks shout BINGO—we ended up having six people win BINGO before we were able to gain the attention of the full group to switch to the next activity!

Overall assessment

The game was easy to run, did not require AV equipment, and was effective at building rapport between participants and creating an upbeat atmosphere. The prompts that we used for a neuroscience audience could be easily adapted to other scientific fields. We only ran the game with approximately 50 participants, but the game would likely scale effectively to a larger crowd. The only downside was the duration—altogether, the game only lasted 10–15 min. The game could potentially be expanded by using BINGO cards larger than a 5 × 5 grid, replacing some of the low-hanging fruit among the answers (“Thought about quitting science”) with less common struggles, or by distributing a second set of BINGO cards.

Context

We featured confessionals as a core part of each of our four interactive events hosted within the 2021–2024 annual SfN conferences, including the remote “Oh Sh*t” Break-out Session during SfN 2021 and the in-person social “The Confound Hour: Let's Make Some Noise!” at SfN 2022–2024.

Inspiration

Following the complicated logistics of organizing the formal storytelling session, we were impressed by how freely participants in the post-event break-out session shared stories about their own experiences with failure. Of course, these spontaneous confessionals occurred after famous scientists had just modeled sharing vulnerability in a professional setting and also followed participation in the F-index game, but they implied that open discussion of failure might be easily promoted within a spontaneous, low-stakes, supportive environment. We tested out this hypothesis during the next 3 years, within the format of an in-person SfN-sponsored social (“The Confound Hour”).

Vision

We billed the social as an entertaining opportunity for scientists to discuss work that didn't make it to the main floor of SfN (“Did you have an experiment that tanked this year? Now is your time to shine!”). At the event, the audience was encouraged to cheer loudly for each confessional, in empathy and solidarity, celebrating the storyteller's insight and persistence.

Logistics

We tested out multiple formats for spontaneous confessionals. Within the remote break-out session following the storytelling session (SfN 2021), we had participants meet in small groups (zoom break-out rooms) to brainstorm ideas (“You deserved more points: What failures should have been part of the F-Index but weren't?”, “Would you be willing to be a storyteller? If so, what story would you tell?”), and then rejoin the group to share their stories and ideas either verbally or in the chat.

At the in-person social at SfN 2022–2024 (“The Confound Hour”), we started with a game (F-index or BINGO) to create a rowdy, upbeat atmosphere. In 2022, we then had participants volunteer to come up on stage and share their stories, with the first 10 volunteers receiving drink tickets. For each confessional, we encouraged the audience to cheer loudly. To avoid potential legal or professional pitfalls in a spontaneous live event, we provided a set of rules for participants (Box 4). We recommend that future facilitators leave these rules posted during the confessionals and occasionally provide reminders, as it was awkward to navigate providing a supportive (rowdy!) environment for confessionals while responding to stories about mistakes with potential legal/ethical ramifications. For example, although it is important for trainees to have opportunities to discuss failures in animal research to prevent reoccurrence, publicly discussing any mistakes that involve real distress to the animals is risky in our current polarized climate. Likewise, it is absolutely essential that confessionals remain personal, with any cameos from colleagues anonymized to avoid professional backlash.

In 2023–2024, we had participants break up into teams (6–10 individuals) and then structured the confessionals as a team-based competition following the format of the game Two Truths and a Lie (wikiHow, n.d.). Each team was given time (20–25 min) to come up with their best stories of experimental failure, as well as one fake story. We then went around the room: in each round, a representative from one team presented their three stories, and the remaining teams voted on which story was the lie. The lie was revealed, and the voting teams received points if they were correct, whereas the presenting team received points for each incorrect answer. After the reveal, audience members could demand further details about the true stories. In the future, we recommend that facilitators bring some method for publicly tracking team points (e.g., white board or large pad, projected excel sheet) and a portable microphone. The game concluded with an announcement of the winning team, as well as audience nomination and vote for the best stories. In 2022–2024, the best storyteller was coronated with a physical golden duct tape crown and awarded the “SfN “Stuck to It” Award” in celebration of their persistence in the face of adversity.

Impact

Over the past 4 years, our interactive events, which included both the F-index and BINGO games and spontaneous confessionals, have been attended by several hundred participants. In comparison to the formal storytelling session, these events have typically attracted a younger crowd—mostly students and post-docs—representing a diverse variety of backgrounds (genders, race/ethnicity, institutions, countries of origin).

Although the long-term effects of the events on scientific practice are unknown, at each event there was an immediate, visible impact on transparency, appreciation of failure, and mental health. Within the sessions, there is a palpable feeling of collective catharsis and visible bonding, with newly formed groups of participants leaving the sessions together. Afterward, participants told us that they felt less alone, more confident, more connected, or even “healed.” As one participant recently wrote “I initially felt a bit out of place at the conference, but your social hour provided an excellent platform for meaningful conversations among scientists. It was truly enjoyable and sparked insightful discussions that I carried back to my lab.” In a post-session survey, participants unanimously reported that our event made them feel like failure is a regular part of successful science, and they, in turn, viewed their own failures in a more positive light.

Overall assessment

Logistically, interactive events encouraging spontaneous confessionals seem to provide the greatest impact in proportion to the effort spent organizing them. These events worked well over a remote platform (zoom) and in person and were even feasible in a crowd of >100 with broken AV equipment. The downside is a restricted ability to scale the events—the larger the group, the harder it is to create the sort of “low stakes” atmosphere that encourages sharing stories about failure. We found that team-based activities helped encourage more active participation, allowing anxious or introverted individuals to contribute to smaller scale discussions, while extroverted individuals represented the team's stories to the larger audience. That said, even within the less structured “take the stage” format of 2022, we still had more than 25 people spontaneously share confessionals to an audience of over 100. Thematically, the use of the format of Two Truths and a Lie appears to fit well with the theme of confessions—in order to make a true story appear as a lie it should be somewhat outrageous or have some dramatic aspects—this motivates participants to share more of their “bigger” stories.

To scale up further, it might be possible to have individuals or teams share their best stories over an interactive online platform and then have voting take place online. For example, in parallel at SfN 2022, The Community for Rigor, an NIH-funded educational initiative, collected confessionals from attendees at their booth and via an online submission portal, focusing specifically on the ways that attendees had grown in terms of scientific rigor, including challenges related to p-hacking, experiment preregistration, and lab workplace dynamics. These submissions were then shared anonymously online for the next 6 months on the organization's Twitter account (@comm4rigor) following screening for inappropriate or injurious content. Similar to our experiences within “The Confound Hour”, Community for Rigor found that both the in-person interviews at their booth and online confessionals served as a rich, generative means for understanding the shape and scale of the problems that arise when science only talks about success.