Dr. Holly Garriock Oral History 

Download the PDF: Garriock_Holly_Oral_History (66 kB)

Dr. Holly Garriock

Behind The Mask

December 13, 2021

Barr: Good afternoon. Today is December 13, 2021. My name is Gabrielle Barr, and I’m the archivist at the Office of NIH History and Stetten Museum. Today I have the pleasure of speaking with Dr. Holly Garriock. Dr. Garriock is the chief cohort development officer for the All of Us research program, which is part of the Office of the Director (OD). Today she is going to be speaking on her COVID-19 initiatives and research. Thank you very much for being with me.

Garriock: My pleasure. Nice to be here, Gabrielle. Thanks for the invite.

Barr: Absolutely. To begin, will you introduce the working group who put together the COPE [Covid-19 Participant Experience] Surveys, their different areas of expertise, and your role in heading this group. It was a rather substantial effort.

Garriock: It was for sure, and it wasn’t done single-handedly. It definitely required a big team. It was largely Dr. Jordan Smoller from Harvard and I putting together a team across the consortium—groups mainly of subject matter experts. Several of us had mental health expertise. We also had survey methodologists on the group and people that were really eager, able, and willing to help us put this survey together in a really rapid fashion.

Barr: How did you choose which instruments to look at when you and others were creating your surveys?

Garriock: We mostly wanted to make sure that we used surveys and instruments that were highly validated and had good psychometric properties. We were able to leverage some of the modules that were in development. We had a mental health and well-being module that was in development, as well as a social determinants of health module—basically a survey questionnaire—that was in development. Those two separate groups from those surveys were able to come together and had already provided a lot of information on the best validated instruments in those fields. We balanced that with wanting to make sure that we had instruments that were in there that were widely used across research so as to facilitate the crosstalk between cohorts and between data collections during the pandemic.

Barr: What did you do about the questions that were COVID-19 specific, especially in your early surveys, when a lot of tests hadn’t been validated yet?

Garriock: We really adopted from other past surveys that were focused on viral infections. For some of the pandemic impact survey items, we were able to borrow from some surveys that were out there during the time of Ebola. We used those past surveys for pandemic impact. For the COVID-specific symptoms, questions, and things like that, we really relied on the CDC [Centers for Disease Control and Prevention] and the symptoms that they were reporting so that we were able to update that symptomatology over time.

Barr: Why was Ebola the infectious disease that you looked at, as opposed to other types of infectious diseases?

Garriock: We were able to find a survey instrument that really measured the impact of the pandemic during the Ebola time. We weren’t able to find something [else] that was as up to date, widely used, with good psychometric properties, and measured the impact of the pandemic on a participant’s lifestyle. That’s why we went with that one.

Barr: You and your team asked a lot of different types of questions in these COPE Surveys. Can you go over some of the main categories that were a part of the survey, and what you and your team hope to learn from them?

Garriock: Yes, for sure. We had a section of the survey focused on mental health, and that would include anxiety, depression, suicidality, loneliness, perceived stress, as well as some substance use. Then we had a section of questions around the social determinants of health. What are the neighborhood characteristics like? Do they have support? Do they experience discrimination? Then we had the COVID symptoms, testing, and treatment—and eventually vaccination—questions that we were able to add. That’s the broad swath of content for the survey.

Barr: How did you ensure that the language in the survey remained accurate with the very nuanced situations, but still really simple enough for participants to understand? You are reaching out to a lot of different people.

Garriock: We were. We do have a goal within the program to keep our reading level within the middle school range. We did a lot of user testing in the previous [focus] groups as the two previous separate surveys were being developed. We did some user-experience testing with them to make sure they understood the questions and really just tried to make them as understandable as possible. We balanced that with wanting to keep the assessment as validated as possible. We were also able to add a tip tool—this button you can hover over electronically which would provide some additional information for what that particular section of questions was asking about and the value of that section of questions to the participants and to their researchers, ultimately, with the data that the participants would be providing.

Barr: Is that a strategy that All of Us has used before with their surveys, or is that a new strategy employed for COVID-19?

Garriock: It’s a new strategy that we kind of evolved with COPE, but it’s now one evolution that happened during the pandemic that we’re going to keep within the program. We heard positive reviews of it, so that is one to stay.

Barr: That’s really great. Before you were with All of Us, you were with the National Institute of Mental Health (NIMH). Will you speak about how you were able to apply your expertise in anxiety and depression to the construction of the survey?

Garriock: Yeah, for sure. While my Ph.D. is in genetics— I’m a trained geneticist—my focus has always been on psychiatric genetics. My research was on the genetic basis of antidepressant response and, later, depressive disorder in adults. When I came to the National Institute of Mental Health in 2010, I oversaw a portfolio of research on pediatric anxiety depression and suicidality. I really got to balance my expertise in genetics with learning a whole new area around pediatric and adolescent mental health in general. At NIMH, I really served as the genomics reviewer within our branch for any [extramural] applications that came in that had a genomics component. I really got to know a lot about pediatric anxiety and depression and took that with me over to All of Us. Still, [it was useful for studies] in adults, for the COPE Survey, being able to know the very common assessment tools used within anxiety and depression and knowing that those phenotypes are very common and what can help impact them. So, we knew that those would be content areas that would fluctuate or be impacted by the pandemic. We knew we wanted to include them on the survey.

Barr: Definitely. How did you modify the questions to reflect new stages of the pandemic across the six surveys that were sent out?

Garriock: Primarily, we were able to add symptoms in or change the symptoms. As the CDC added new symptoms to their guidance or awareness levels, we were able to integrate them into the subsequent survey version. As society develops the vaccinations, we were able — in February, in our last version of the COPE Survey—to add questions around vaccination. Did they get vaccinated? And what was the likelihood that participants would get vaccinated?

Barr: It was possible for participants to respond to the surveys more than once. What percent actually participated throughout all six surveys?

Garriock: It was about 10%. It’s around 10,000 participants. Of all the responders, we had about 106K responders to any one survey across the six versions. It was about 10% that completed all six.

Barr: Wow. That’s much higher than I would have expected. That’s great.

Garriock: Yeah. We do have a very eager base of participants that are willing to engage with the program. It’s really great to see.

Barr: How did the feedback from your participants help to shape the direction of your surveys?

Garriock: One of the biggest ways their feedback was taken into account was a shortening of the survey. Our first three versions—May, June, and July—were the longer survey, the more complete, comprehensive survey. But the November-December 2020 and February 2021 versions were shortened by about half. We took a look at the questions that participants were completing—which ones were performing well, which ones were not performing well, if there was variation among them, or which ones caused a lot of questions coming into the support center. We cut some of those out. Also, for some of the mental-health content ones, assessment tools can be longer—like nine items. Or there’s another equivalent that could be two or three items. For the second version of the COPE Survey, we went to the shorter versions of those assessment tools rather than the more comprehensive, complete, and longer nine-version items, as an example.

Barr: What about the questions where participants could write in their response freestyle? How widely were they responded to? What was the response like? Was that something you kept throughout all six iterations?

Garriock: We didn’t have any questions that were solely write-in, but we did have a lot of questions that had a response option where they could press another button and then have a free-text field. We were able to take in some free-form texts. The biggest difficulty here is, of course, that you get a lot of different responses and different spellings. You have to go through and analyze it, so the consolidation process was quite challenging. The benefit is that you get a lot of information. These participants are really gifting us with how they feel and helping us inform the next iteration. We were able to go through those responses. We thought it was going to be these top ten reasons why participants wouldn’t get vaccinated, but actually they’re telling us another fifteen. So, we need to add that to the next version so that the participants can choose it, or other participants can choose that as a reason why wouldn’t they get vaccinated just yet.

Barr: What were some of the reasons that were not on your original list?

Garriock: Pregnancy was one of them. There were just a lot of people waiting for other people to get it and see how it goes. Remember, for the COPE Survey we first had vaccination questions just in February, so it was still a little early for some participants. There were a fair number of people sitting back and seeing how it evolves—or they just weren’t even eligible. But there was a lot of pregnancy questions or breastfeeding questions. Participants just saying they weren’t sure yet. They wanted to let other people try it.

Barr: I can understand that. What were some of the other challenges that you and your team had in crafting, disseminating, and analyzing the data from all these surveys?

Garriock: Primarily, this was the first survey that we had only out in the field for a short period of time. All of our other surveys are released to the participant and available forever for them to complete. We had this as our first longitudinal set of surveys. For some of them, they were only open a month total for the survey. We had to get the communication out there to participants and let them know that the survey was there, and then we had to let them know it was only going to be there for a short period of time. Then, when the next one came, we had to let them know why we’re asking them again. We had to explain the scientific value of knowing the change over time—the trajectory-based assessments that we made possible by having those fixed versions of the survey. I would say the challenges we had really were communications based. There’s a lot of work with our communications team to identify effective communications to our participants, to drive all of our participants back into the participant portal and to ask them to complete the survey. One of the things that we were able to do on the sixth iteration is really provide this direct link for participants within the communication. The participant would get a text message or an email, and within that message there’s a link that brings them directly to that survey that we’re asking them to complete. So they don’t have to log into their participant portal. That helped a lot. About 75% of our participants that completed the surveys used a direct link, so we found that was also something that is going to stay. For our participants, it seems to be a very effective strategy.

Barr: That’s really great. Are you aware of any of the findings from these surveys? I know it’s quite a lot of data to analyze.

Garriock: We’re not quite sure yet what all the findings will be. On our side of things, we really look at operational data and survey completion. We try to break that up by participants who are historically underrepresented in biomedical research and participants who are not. Then we really make sure the survey is performing as we would expect. We leave all of the other analyses and scientific findings for the researchers to do. We hope that providing this infrastructure of a data set within our researcher workbench will allow our researchers the ability to answer some of those questions that they might have once all the code data is ready and available.

Barr: You spoke about two strategies that you did with the COPE Survey that you’re going to employ for other kinds of service. Did you learn any other lessons from working on these surveys that you would apply to your other work?

Garriock: The other one is the resources. We provided a lot of resources to participants over time—in the middle of the survey as well as at the end of the survey. We made that information available to them even outside of the survey experience. We tried to do that for the subsequent surveys, beyond COPE, so that participants can have some resources available to them that are not dependent on the survey experience. We try to have additional return of information to participants as well, so that we can show them some data insights on how many participants within the cohort are answering similarly to them, or what the distribution is within the cohort for a particular question. For all of these questions, we try to make sure that the participant gets some value out of the time that they spent completing the survey. We truly believe that the participants are our partners, and we ask them what would be valuable to them and what they think about the proposed return of information for the assessment that we’re putting in. We use the feedback from the participants in order to inform next steps and decision making.

Barr: Will you speak about some of the other ways you’ve contributed to combating COVID-19, including your detail with HHS on its mental-health response during the pandemic?

Garriock: Sure. In the summer of 2020, I was deployed to the Healthcare Resilience Working Group (HRWG) as part of the HHS [United States Department of Health and Human Services]/FEMA [Federal Emergency Management Agency]/ASPR [Administration for Strategic Preparedness and Response] efforts to respond to COVID-19. I was there, and it kind of took me away from my All of Us activities in order to contribute to that working group. Really, in this capacity I served as subject-matter expertise for behavioral health and really tried to focus the efforts for our health care providers in order to build up resilience resources for the first responders to the COVID-19 pandemic. That was my primary responsibility.

Barr: What kind of resources did you create? Online? Or pamphlets?

Garriock: Some of it was a catalog of available resources—a catalog of meditation and well-being resources available for healthcare responders and healthcare workers. Most of it was digital-asset based for the providers.

Barr: That must have been a really interesting experience.

Garriock: It was great. Everybody was in line; everybody was energized and mission-oriented—solely focused on making sure that the resilience of our healthcare workers was maintained. We were able to take their mental health and well-being into consideration and prepare them as well as possible during a really unprecedented event.

Barr: Did you focus on certain types of healthcare or first responders? Were certain resources for nurses or physicians? Did you specialize to that level?

Garriock: For some assets we were able to specialize at that level. There’s a great website—the ASPR TRACIE [Technical Resources, Assistance Center, and Information Exchange]—which really catalogs all of the efforts from the Healthcare Resilience Working Group. It was maybe less by specialty and more by rural healthcare workers, urban healthcare workers, EMS, emergent responders versus what’s happening with the telemedicine response we can help provide. Those were the kind of divisions that were happening during the time when I was deployed to that group. It was a very active group throughout the summer, and there were a lot of contributors to it.

Barr: Did you provide resources for mental-health providers? I’ve heard from some people that they’re also being hit hard during the pandemic with all types of issues from hearing people all day long talk about their experience.

Garriock: Exactly. It’s funny: those providers are probably the most knowledgeable about strategies and skills for coping and keeping yourself well, but they’re maybe the worst users of [that knowledge]. It’s important to provide a peer-to-peer platform for them to interact with. There’s still a lot of stigma around mental health and well-being, especially around healthcare workers. So it was important to make sure that we developed, or contributed to the development of, something that provided a peer-to-peer network for them to engage with. We heard this same thing about how hard it is to hear the stories of participants across the country when we paused in-person activities within the consortium. Our staff went from primarily in-person enrollment—very interactive interactions with our potential and current participants—and went fully remote. They became all phone based. It was great: they were doing a lot of warm reach-outs from the program to the participants, trying to see if they’re okay and keep them engaged. They just keep hearing that not only are they experiencing the pandemic and their own stresses of their life, but then they’re hearing on the other end of the line the challenges and tribulations that our participants are going through. Actually, we put together a couple webinars for our frontline staff so that we’re able to help build strategies for them and give them skills. We’ve had a specific seminar on how to meditate and shared some scanning you can do to help calm yourself—even to the extent of how to talk to your supervisor about how you’re feeling and what you may need during this point in time. Everything I learned during my deployment I was able to transfer over to All of Us and really benefit the greater good for this program.

Barr: That’s great. How else has COVID-19 impacted your regular duties?

Garriock: Like everybody, full-time teleworking and not being in the office has the biggest impact. Onboarding staff has been quite hard. I haven’t quite figured out the best way to onboard staff fully remotely in an effective way. Of course you can do it. It’s just harder and not as good, in my opinion. I would say that’s the biggest impact that this fully virtual environment has had on me. You just miss the regular human check-ins with your team. It’s really hard to get the information from someone all through digital means without getting the context and without knowing people that well. You can read things in different ways, so that has been quite challenging—to really know what that comment actually means from that person. That has been hard. It’s a universal struggle for workers that have gone fully remote.

Barr: Definitely. In addition to being a scientist, you’ve also been a person who’s living through the pandemic. What have been some personal challenges and opportunities that COVID-19 has presented for you?

Garriock: For me, it comes back to the fully remote. This lack of a boundary between work and home—those spatial boundaries are quite valuable, in my mind. Having the separate office from home is not only a physical reminder but a mental reminder of what mode you should be in, and it allows me to shut off the other side of the brain more easily if I’m spatially in a different environment. With more activities canceled because of the pandemic, there was more time to work. I chose working more rather than engaging with my family at times—because [the work] needed to be done. It was a pandemic, and we were trying to respond to that accordingly. Now that all the activities are largely back on the books, we’ve had to shift—and I have to say, “Okay, I can’t actually work that much.” I need to move my kids around. I need to engage with them and maybe even attend those activities. I would say that’s been the biggest challenge for me and my family: shifting from being fully home and having all the time because nothing else is going on. Once you add things back in, it becomes more challenging. In terms of an opportunity, we did leverage the opportunity—since we were home and were able—to rescue and adopt a dog. We adopted him from Mississippi. He’s a boy so we call him Mister Sippy. That’s been our newfound opportunity that we were able to do because of the pandemic that I’m not sure we would have otherwise done.

Barr: Oh, that’s wonderful! Is there anything else that you would like to share about your COVID-19 work or experiences?

Garriock: I just want to re-emphasize that it definitely was very much a team environment and a team effort. We had great team members contributing to COPE across the consortium. It was just a very great example where once one thing is prioritized and everybody is mission-oriented and aligned on what we wanted to do as a program, we were unstoppable. We were able to make some great innovations within the program.

Barr: That is great! I wish you and your team and your family all the best, continued success and health, and very happy holidays.

Garriock: Thank you Gabrielle. It was such a pleasure.