Since our program was formalized in 2014, we have worked to better understand the impact of our grantmaking. What began as an informal process of cataloguing grantee outputs evolved into an internal database that tracks publications, conferences, students supported, and additional funding raised through Sloan-supported research. The database serves as a light-touch tool for monitoring measurable outcomes while helping us identify broader trends over time, and has also informed our strategic planning process.

We are pleased to share the lessons we learned along the way about incorporating assessment into routine operations, using the information strategically, and adapting the process over time.

Time is Precious, for Both Grantees and Funders

We initially needed something we could build and maintain with limited staff time and resources available. A simple Excel spreadsheet has worked well for us because it is organized, straightforward, and adaptable. We focused on tracking categories most relevant to our program strategy of generating new knowledge, supporting early-career scholars, and connecting research to practice. Starting simple was valuable, allowing us to iterate based on our needs and priorities. For example, we initially wanted to track conference attendees, but it quickly became apparent that would become intractable, so we removed that category.

The process was still time-consuming; establishing the tracker tool took about four months of dedicated time from a single team member, but starting with the simplest option got us insight sooner, prevented us from getting locked into a more onerous process, and gives us the flexibility to move to a more advanced system in the future if our needs change. Ultimately, it is important to be honest about the capacity you have available, but getting going is the most important step.

Hot Tip: Standardize Information Flows but Allow Narrative Freedom

When beginning the tracking effort, our grantee reporting guidelines asked for a loose narrative structure that included general descriptions of outputs and progress updates relative to grant metrics and goals. The level of detail varied greatly between reports, sometimes requiring us to fill in the gaps where information was not provided.

To facilitate the tracking process without adding undue burden on grantees, we limited the amount of new information we asked for and then clarified and updated our guidelines to better align with how we extract information. We now ask for information on written outputs and students in table format with categories that match those used in the database while also retaining the overall narrative structure of the reports to allow grantees freedom in their responses and ensure we don’t miss key qualitative updates.

Nothing Replaces Reading the Reports

In order to understand the full impact of a funded project, quantitative outputs require qualitative context to fully situate outcomes within the broader research landscape. That remains true. We learn so much from reading those report narratives, from cutting-edge research results to contemporary policy impacts. To fully connect outputs to impact, it is critical to have context that can only be provided by making sense of full reports.

Final Thoughts

Retaining flexibility in the reporting process is important so funders can be responsive to not only their needs but also those of their grantees. Perhaps the most important lesson we learned is that output tracking is just one component of a well-stocked portfolio of impact assessment techniques and approaches.

The American Evaluation Association is celebrating RTD TIG Week with our colleagues in the Research Technology and Development TIG. All of the blog contributions this week come from our RTD TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

I’m Andrew Feig – Senior Program Director at Research Corporation for Science Advancement (RCSA). At RCSA, we have two major programs, Cottrell Scholars and Scialog (as well as a nascent RCSA Fellows Program), but for this discussion, I will focus on Scialog. Scialog is a contraction of Science + Dialog. The program brings together groups of scientists (Scialog Fellows) who do not yet know each other but should. After a structured, data-driven process to introduce Fellows to one another and 1.5 days of facilitated discussions, participants spend the last half-day forming teams and writing proposals. Themes typically run for three years, and Fellows return year-on-year for the duration of the initiative, with additional rounds of grant-making each year. RCSA’s goal is to fertilize new areas of science, seed new interdisciplinary collaborations, and help early-career faculty become leaders in the thematic area, where their bold ideas change the direction of scientific thinking. 

At one level, our Board wants big outcome assessments. What were the ground-breaking scientific advances that derived from these grants? How did these grants change the field? Unfortunately, since we fund early-stage ideas, it often takes 5, 10, or even 15 years for the most successful projects to evolve and yield that level of impact. These stories do not provide actionable insights to guide the program and thus have only modest value in my opinion. Collect them, but don’t spend too much time or resources here.

Hot Tip

Far more important are the early hallmarks of success that we can use as proxies for what may eventually come from these teams. Do the collaborations persist? What occurred during Scialog that helped the teams gel? Was there sufficient early success that the team sought or acquired additional funding based on data our funding helped them obtain? We collect early hallmark data from structured grant reports, and supplement those reports with annual analyses of every past Scialog – back to its inception more than 15 years ago. Using APIs, we pull publication histories of every participant (more than 2000 to date). We tag papers co-authored by 2 or more Fellows who met at Scialog, noting whether or not they received funding or wrote a proposal together. We use these papers to track the long-term network effects of the community we seeded. It turns out that many of these papers come from teams that never received a grant and thus would never have submitted a progress report. They liked the idea they proposed and chose to pursue it with their colleagues anyway.

Even more important, however, are the questions about the activities during Scialog. What activities effectively build trust between Fellows? What motivates two individuals to choose to collaborate just 24 hours after having met? Through a project with Prof. Danny Abrams (Applied Mathematics @ Northwestern), we have measured the interpersonal interactions during our meetings, the nature and role of facilitation in discussion groups, and the dynamics of group conversations that break homophily and drive creative thinking. These studies have led to changes in our conference schedule, modifications of our facilitator training, and optimization of the event in transformative ways. This is the type of assessment I lean towards: INTENTIONAL – based on data collections we build into the planning and execution of our meeting; it is part of what we do not an add-on;  ACTIONABLE – allowing us to use it to refine our convening formats; and IMPACTFUL – providing a feedback loop that lets us tell the story of how we use assessment in the day-to-day work of our foundation and pursue continuous program improvement.

The American Evaluation Association is celebrating RTD TIG Week with our colleagues in the Research Technology and Development TIG. All of the blog contributions this week come from our RTD TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

NIH’s central responsibility is to support the scientific community, primarily through competitive funding for research. That work is done in partnership with experts pulled from the community itself: previously funded researchers who review grant applications and advise NIH institutes and centers. Complementary and integrative health (CIH) investigators, a subset of scientific researchers holding specialized certifications (osteopathy, acupuncture, naturopathy, and others), do not receive NIH funding or participate in these activities at the same rate as non-CIH investigators. The CIH community was advocating for increased participation, and leadership at the National Center for Complementary and Integrative Health (NCCIH) needed to understand why the deficit existed before deciding how to address it.

Developing a composite metric of success

CIH research has long carried a perception in some quarters that it isn’t as “successful” as mainstream research, which some would argue explains the deficit. To rule out the research-quality explanation, I needed to compare how CIH and non-CIH investigators performed once they were in the NIH system.

Success for a scientific investigator hinges on NIH funding, publications, and impact on subsequent studies or clinical trials. A true success metric combines all of these and accounts for career stage. That metric didn’t exist in the data, so I built one. I used my subject matter expertise to combine the underlying variables (grant review metrics, publications, scientific impact, time) and sort investigators into five success levels (Unsuccessful, Too Early, Mildly Successful, Moderately Successful, Highly Successful). This was time-consuming and depended heavily on my expertise, which limited its transferability. So, I used the labeled dataset to train a Random Forest classifier, a machine learning model that learns your definition once and applies it consistently to new data.

Changing perceptions to identify the underlying issue

I assembled ten years of NCCIH application, funding, publication, and impact data. I sorted CIH and non-CIH investigators into the success buckets and compared the distributions. They were almost identical. Once funded, CIH investigators were performing at the same level as their peers. The gap wasn’t about the quality of the research.

So, I looked earlier in the pipeline and found that fewer CIH investigators were submitting applications to NCCIH in the first place, by a wide margin. This mattered because grant applications are how an investigator gets into the NIH system. If you don’t apply, you aren’t in the system, and you can’t be selected for funding, review panels, or advisory committees. That explained the deficit the community was noticing and gave leadership a clearer picture of how to address it. The conversation shifted from dedicated funding mechanisms for already-competitive researchers to supporting an application pipeline: recruitment, first-time applicant support, and removing barriers to submitting.

Lesson Learned: solve the real problem, not the assumed one

Stakeholders bring evaluators their problems, but they also bring perceptions about what’s causing them. Sometimes the perception is right and the evaluation confirms it. Sometimes it isn’t, and the evaluation’s job is to find what’s actually going on. Doing that work well usually means three things: testing the perception against the data instead of accepting it as given, building new metrics when the data doesn’t answer the question directly, and following the data to the real issue rather than the one you started with. When evaluation does all three, the solution gets aimed at the problem that actually exists.

The American Evaluation Association is celebrating RTD TIG Week with our colleagues in the Research Technology and Development TIG. All of the blog contributions this week come from our RTD TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Research, technology, and development evaluation sits at the intersection of evidence and decision-making. Whether we are assessing grant programs, examining research portfolios, understanding scientific impact, or informing strategy, our goal is ultimately the same: helping organizations make better decisions based on evidence rather than assumptions.

That sounds straightforward, but anyone who has worked in evaluation knows the challenge. We rarely have perfect data. Stakeholders often arrive with strong beliefs about what is happening and why. The systems we study are complex, interconnected, and constantly changing. And sometimes the questions we are asked are not the questions that most need answering. As we launch RTD TIG Week, it is worth reflecting on a few practices that consistently strengthen evaluation, regardless of the setting, methodology, or sector.

Lessons Learned

1. Start with the question behind the question

The first problem presented to an evaluator is not always the real problem. Before collecting data or selecting methods, spend time understanding what decision needs to be informed and what assumptions may be shaping the conversation. Some of the most valuable evaluations emerge when evidence challenges prevailing beliefs and redirects attention toward a more important issue.

2. Design for learning, not just reporting

Evaluation is most useful when it supports continuous improvement. Consider how information will be used before deciding what to measure. Data collection that is built into programs and organizational processes often produces more actionable insights than assessments conducted only at the end of a project.

3. Build systems that can grow with your needs

Organizations frequently begin with ambitious plans for data collection and assessment. In practice, sustainable systems are often the ones that start simple, focus on the most important information, and evolve over time. A modest but consistently maintained tracking system can generate far more value than a sophisticated system that becomes too burdensome to sustain.

4. Look for patterns, connections, and context

As research portfolios and programs grow in size and complexity, evaluators increasingly need ways to understand relationships across projects, disciplines, organizations, and outcomes. Quantitative metrics remain important, but understanding how activities connect and where areas overlap can reveal opportunities that individual measures alone might miss.

5. Keep your focus on impact

Metrics are tools, not goals. Publication counts, funding totals, participation rates, and other indicators can provide useful information, but they are ultimately proxies for something larger. Effective evaluation continually asks whether the measures being used align with the outcomes and impacts that matter most to stakeholders and society.

These ideas are not new, but they remain central to effective evaluation practice. Throughout RTD TIG Week, we will explore how evaluators and research organizations are applying these principles in different contexts, using a variety of approaches and methods. While the settings may differ, a common thread runs through all of them: a commitment to using evidence thoughtfully, asking better questions, and helping organizations learn.

We invite you to join us throughout the week as we examine the evolving tools, challenges, and opportunities shaping research evaluation today. We hope the discussions spark new ideas for your own work and encourage you to share your experiences with the RTD TIG community.

The American Evaluation Association is celebrating RTD TIG Week with our colleagues in the Research Technology and Development TIG. All of the blog contributions this week come from our RTD TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi, I’m Jacqueline Corum (she/her), a Cooperative Extension evaluator at the University of Kentucky and graduate student in the Educational Research Methods and Policy Studies program.

In a previous role, I facilitated a grassroots community and leadership development program at the Brushy Fork Leadership Institute called “People Ready Communities.” This program supported diverse teams of community members as they led projects aimed at improving their communities’ “readiness” to be places where all residents experience safety, acceptance, inclusion, and belonging. We did this through capacity building; enhancing local leaders’ ability to make data-informed, collaborative decisions that supported the planning and implementation of short-term community development projects based on their unique local needs and context.

This grassroots model informed my work this semester as I conducted interviews and focus groups exploring how LGBTQIA+ folks in Lexington, Kentucky experience and navigate feelings of safety and belonging in their daily lives. I used this qualitative research development opportunity to improve both technical skill and academic knowledge through incorporating theories on intersectionality, microaggressions, belonging, and aging in place to better understand contributing factors. Despite growing availability and pressures to use AI to support research and data analysis, I coded the data without using AI, reflecting throughout the coding process on my positionality as a researcher and member of the community I am studying. I also think I captured thematic elements relating to queer culture that AI may have missed or represented less authentically. I found Dedoose to be a helpful tool as I went through iterations of open coding, describing and organizing themes, and creating a codebook with excerpts.

My next step is to continue exploring the connections between belonging and community capitals. The relationship between belonging, community capitals and economic impact appears to be a gap in literature. I believe the economic justification for fostering LGBTQ+ safety and belonging effectively compliments the qualitative data on the topic.

Reflection

How can we leverage evaluation to foster communities where all members feel safe and that they belong?

Hot Tips

• RESIST outsourcing qualitative data analysis to AI.
  •  In an interestingop ed from Inside Higher Ed, Andrew Gillen points out four main points: the researcher is just as important as the research; AI is not neutral; adoption of AI tools can have a negative impact on the training of new researchers; and AI can’t safeguard data like human researchers.
  • Claire Moran advised that ethical awareness requires oversight. “AI can hallucinate citations, reproduce bias, and generate content that sounds right but is wrong.”  They added Roberts et al. “caution that LLMs imitate language patterns, not truth, with AI’s ‘human like’ qualities resulting from statistical predictions and not comprehension. Without human review, misleading or unethical outputs can slip through.”

Rad Resources

• Dedoose is a low-cost, user-friendly tool for manually coding qualitative data themes, creating theme visuals, and data reporting.
• HRC’s Municipal Equality Index provides a rubric and analysis of city’s LGBTQIA+ related policies
• PFLAG’s Guide to Being an Ally to LGBTQ+ People
• AARP Livable Communities resources support communities to increase their accessibility and safety for ALL, especially for the disability and aging communities.
• Welcoming America is a national nonprofit with resources focused on supporting communities to become more welcoming and inclusive of immigrant populations.

If you are interested in learning more or joining my work, please reach out to me. I would like to identify potential collaborators interested in supporting a joint research proposal, data collection and analysis, and/or publication development.

The American Evaluation Association is hosting LGBTQ+ Voices in Evaluation TIG Week with our colleagues in the LGBTQ+ Voices in Evaluation Topical Interest Group. The contributions all this week to AEA365 come from our LGBTQ+ Voices in Evaluation TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.

Hi, I’m Laura Pinsoneault and Kristen Gardner-Volle from Evaluation Plus. We served as the evaluator on a PCORI award that brought community leaders, clinicians, and researchers together to understand and reduce cancer disparities among transgender, nonbinary, and gender-diverse (TGD) people. We worked with the Community and Cancer Science Network (CCSN), which starts with a simple belief: how we connect drives change.

Lessons Learned: Collaboration Is Health Equity

When we began exploring why TGD people experience worse cancer outcomes, we quickly realized the issue wasn’t just missing data or bias, it was also how knowledge itself is produced. The CCSN model treats relationship-building as research infrastructure. We brought together TGD community leaders, clinicians, and scientists as equals to map root causes of cancer disparities from systemic discrimination to hormone access.

To make that collaboration real, we started with team design. We used a MoSCoW matrix (“must have, should have, could have, wish for”) to identify whose perspectives were essential—community advocates, researchers, clinicians, and people with lived expertise. This structure balanced expertise, experience, and identity before the project began. For evaluators, this means that partnership design is part of an equity practice: it shapes what knowledge emerges and whose wellbeing is centered.

Hot Tips: Use Creative Tools to Transform How People Think Together

We mixed community members, clinicians, and researchers into small groups to design an Album Cover—titling it to show what the world would be like if it was free from disparities and three “title tracks” that described how to get there.

It was playful and energizing—but also deeper. As people built their albums, they began to see their expertise in new ways. Scientists talked about belonging. Community partners talked about systems change. The exercise didn’t just give everyone a voice; it reshaped how we understood our knowledge and its limits. It moved us from parallel expertise to shared insight and bonded the team in ways no formal meeting could.

Cool Tricks: Move from Vision to Action

Once our shared vision took shape, we used a Consensus Workshop to turn ideas into action. The process blends the Focused Conversation method with five stages of collective synthesis. It balanced dialogue, surfaced wisdom and turned inspiration into action.

Here’s what emerged:

Rad Resources

• Our CCSN Toolkit includes tools, team-building guides, and templates to help others design transdisciplinary partnerships grounded in deep equity.
• As evaluators for the SAGE LGBTQI+ Advancing Elders Project, we learned from the Global Story Archive where LGBTQ+ elders share personal stories about health, identity, and resilience. For evaluators, these questions matter because they expand what counts as evidence and remind us that the pursuit of LGBTQ+ health equity is inseparable from the pursuit of human dignity. Story-based evidence helps us see the systems that harm and the possibilities that heal.

Reflection

Health equity grows in the spaces where science and community meet, listen, and learn together. Relationship-building is equity work. Creative structure simply gives it the shape it needs to thrive.

This work was developed by a transdisciplinary team including Chandler Cortina, MD; Tobi Cawthra, MPH; michael munson; Caleb Weinhardt, Andrew Petroll, MD; and Melinda Stolley, PhD, in partnership with FORGE and the Medical College of Wisconsin Cancer Center.

This project was funded through a Patient-Centered Outcomes Research Institute (PCORI) Eugene Washington PCORI Engagement Award (EA #25591). The views presented in this article are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.

The American Evaluation Association is hosting LGBTQ+ Voices in Evaluation TIG Week with our colleagues in the LGBTQ+ Voices in Evaluation Topical Interest Group. The contributions all this week to AEA365 come from our LGBTQ+ Voices in Evaluation TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.

I am Veronica S. Smith, MS (she/they), a community data and evaluation guide and solopreneur of data2insight.

I partner with LGBTQ-serving organizations doing community-driven data collection, analysis, and storytelling that centers lived experiences and everyday realities. This work is informed by The Coalition of Communities of Color’s research justice strategy guidelines:

• Community members are experts.
• Communities are positioned as researchers not just subjects of research and inquiry.
• Communities can conduct critical and systemic inquiry into their own lived experiences.
• QTBIPOC knowledge and expertise can counter dominant cultural narratives

Hot Tips

A community data party is an activity we use in our work to grow trust across ecosystem actors and capacity to conduct critical and systemic inquiry.  A data party is an in-person event where diverse community members collectively analyze and make sense of data, equitable research method for community engagement.

A data party’s purpose is to:

• encourage people to make sense of data together,
• center community members’ in use of data to make policy, law, and/or provide resources to benefit a community,
• gives the community informed decision making.

The potential benefits of a data party are:

• increased community research knowledge and interpretation of data,
• data sense making happens within a community,
• community-backed recommendations to inform the community, policymakers, lawmakers, and funders.

Lessons Learned

Kory Higgs, and I co-facilitated a community data party for The Collaborative, made up of Queer Power Alliance, POCAAN, Entre Hermanos, and Pacific Northwest Black Pride. Our focus was gathering community members to make sense of survey results about economic stability for South King County LGBTQIA+ residents. About 25 folks joined for food, music, community, and conversation.

The roundtable discussions were the event highlight. Attendees reviewed data placemats featuring information and questions about the three focus areas of transportation, healthcare, voting and civic engagement. Conversations reflected on experiences, challenges, and possible solutions.

One party goer said they enjoyed the conversations and want more time to continue those conversations. Another appreciated that findings were shared in plain language and easy to make sense of compared to some research findings that “make me feel dumb.”

The Collaborative is building on this work to co-create a 2027 legislative agenda to improve economic stability for South King County LGBTQIA+ residents.

Rad Resources

These great resources are from Corey Newhouse and Dana Benjamin, leaders of Public Profit and Back of the Napkin (respectively).

• Dabbling in the Data: A Hands-On Guide to Participatory Data Analysis
• Back of the Napkin Downloadable Resources

This Stanford Social Innovation Review article explains how overreliance on dominant data fails communities and how community data provides a solution.

• Community Data Is Trusted Evidence by Mira Mohsini & Andres Lopez

Culturally Responsive and Equitable Data Parties: A Method for Participatory Analysis and Sense-Making in Virtual Spaces

The American Evaluation Association is hosting LGBTQ+ Voices in Evaluation TIG Week with our colleagues in the LGBTQ+ Voices in Evaluation Topical Interest Group. The contributions all this week to AEA365 come from our LGBTQ+ Voices in Evaluation TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.

Hello! We’re McKenzie Javorka, Patrick Boise, and Kaitlin Fertaly, evaluators at the University of Montana’s Rural Institute for Inclusive Communities.

Montana has major gaps in pre-exposure prophylaxis (PrEP) uptake among people who could benefit from it, especially men and people assigned male at birth (AMAB) who have sex with men/AMAB individuals. To understand why, we partnered with our state public health department to conduct a statewide PrEP needs assessment.

Lessons Learned

We’ve done plenty of needs assessments in Montana, but this one challenged us in new ways, especially when it came to recruiting participants.

Our original plan was straightforward: run a statewide online survey, then invite a subsample of respondents to do follow-up interviews. Given Montana’s size and rurality, an online survey seemed like the most efficient and affordable option. But soon after launching, our survey was flooded with bots and fake responses. Sorting through what was real versus fake or invalid became a major task.

Even the next step, inviting survey participants to follow-up interviews, was tricky. Some scammers went as far as completing phone interviews with false information to earn incentives. It quickly became clear we needed a new approach.

We pivoted and instead began recruiting through community-based organizations, health clinics, and word-of-mouth networks. We distributed flyers, partnered with trusted LGBTQ+ and harm reduction groups, and tabled at a drag show to connect directly with potential participants. It took longer than expected and skewed more urban than we hoped, but these efforts finally got us the participation we needed.

Hot Tips

• Expect bots, AI, and scammers in online surveys. We built in checks like requiring a valid Montana zip code and logic questions, but they weren’t enough. AI-generated and falsified responses can easily slip through. In the end, we had to apply multiple filters after data collection and still had lingering concerns about data validity.
• Be cautious about advertising incentives. Incentives can help with recruitment but publicizing them online can also attract bots and scammers. If you must mention them, consider mailing physical incentives (which require a real address in the study area) or using verification systems before sending digital rewards.
• Partner with trusted organizations and “study champions.” Our most effective recruitment came through LGBTQ+ and harm reduction organizations that directly shared our study with clients. We also relied on highly connected community members who spread the word through social media and personal networks. However, these tactics may not be the most effective for reaching people in the most remote areas or those who aren’t openly LGBTQ+.
• Offer flexible participation options. There is no one-size-fits-all approach for interviewing – different groups prefer different formats. For example, people who use injection drugs were easiest to reach in person at harm reduction sites, while most other participants preferred scheduled Zoom or phone interviews. Flexibility helped us meet people where they were, literally and figuratively.

Despite the bumps along the way, we ultimately met our study goals by staying persistent, flexible, and community-centered. However, reaching LGBTQ+ participants in rural areas may take more time- and resource-intensive methods, such as traveling to recruit at in-person events or building relationships with trusted community gatekeepers. These methods were beyond our budget and timeframe for the current PrEP needs assessment; however, we now know to plan for more intensive outreach in future projects focused on rural LGBTQ+ populations.

What strategies have worked for you in reaching LGBTQ+ people in rural areas? We’d love to hear your thoughts in the comments!

The American Evaluation Association is hosting LGBTQ+ Voices in Evaluation TIG Week with our colleagues in the LGBTQ+ Voices in Evaluation Topical Interest Group. The contributions all this week to AEA365 come from our LGBTQ+ Voices in Evaluation TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.

Hello! We are Gregory Phillips II, PhD, and Nate Varnell, MPA, members of the LGBTQ+ Voices in Evaluation Topical Interest Group. This Pride Month, we want to name something directly: the losses and restrictions of sexual orientation, gender identity, and expression (SOGIE) data collections at the federal level are putting evaluators in conflict with our own professional principles. But, there are many ways the community can advocate for itself and find a path forward.

Implications for LGBTQ populations and evaluators

Grant terminations on the basis of “DEI” and “gender ideology” disproportionately impacted LGBTQ+ populations, costing not only data but a generation of researchers who reflect the populations they study. Consistent SOGIE data collection is how we track HIV risk, substance use, mental health disparities, and many more factors impacting LGBTQ+ communities. The Youth Risk Behavior Surveillance System (YRBSS) is a concrete example: a transgender status question was pilot tested by the CDC in 2017, implemented nationally in 2023, and gave us the first nationally representative data on trans youth health. The administration’s decision to not process the 2025 data means we may never know which jurisdictions assessed it, and if they will ever release these data.

Executive Order 14168 redefined sex based on gamete size and removed gender from federal data collections, contradicting decades of research on the distinctiveness of gender from sex. For evaluators, this forces a tension between following federal guidelines or AEA’s Guiding Principles. Systematic Inquiry requires we adhere to sound standards; how should evaluators work with gender minority populations when they are told to classify by sex rather than gender? How does this violate the Respect for People principle? Integrity requires evaluators to be honest and transparent. How do we reconcile the fact that some evaluators will choose to follow their conscience and science, and others will follow the EO? Finally, decades of research have highlighted the harms that are caused by deadnaming and misgendering—how do we justify violating the Common Good and Equity principle by refusing to recognize people as they identify within our evaluations?

Rad Resources: Upholding our principles through data rescue and advocacy

We must all be data stewards and keep ourselves, our data, and the communities we serve safe. The data lost and new restrictions will be felt for a long time, but we can act in line with our principles by building resilience and the capacity to advocate.

• Proactive Networking: A core principle of LGBTQ Evaluation is that it demands community ownership. There is immense value in building a rapid response network before the next crisis. Your expertise is an asset. You know the data that you need and care about, and should make it known to the community. Example: The Federal Data Forum.
• Knowledge Bases: There’s no guarantee of institutional longevity with government, so we need to build dual power with real infrastructure, like community owned repositories. With or without dedicating funding or institutional backing, there are many low-cost, open source tools available. The hard part is the time and energy to curate and update these resources—it takes a village. Example: The Public Environmental Data Partners hosts a digital archive and GitHub pages to preserve environmental data.
• Build Your Advocacy “Muscles”: Staying informed is only as important as the ability to act when opportunity knocks. LGBTQ+ evaluators and SOGIE data professionals need to share resources on how to write letters and public comments to congresspeople, agencies, and the media; and to be coordinated with engagement and the messages we share. AEA held a Town Hall on April 14 about exactly this.

The American Evaluation Association is hosting LGBTQ+ Voices in Evaluation TIG Week with our colleagues in the LGBTQ+ Voices in Evaluation Topical Interest Group. The contributions all this week to AEA365 come from our LGBTQ+ Voices in Evaluation TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.

I am Dr. Angelica Thompson (she/her), Founder & CEO of Data Works, Inc., an applied research and evaluation consulting firm that helps organizations use data, evaluation, and strategic insight to improve outcomes and drive meaningful impact.

Organizations often talk about inclusion as a value, but employees experience inclusion through action, culture, and feelings of belonging. One of the most meaningful ways organizations can foster a sense of belonging is through Employee Resource Groups (ERGs) or Employee Interest Groups (EIGs).

Lessons Learned

At their best, ERGs are far more than social groups or symbolic initiatives; they create intentional spaces where employees can connect through shared identities, experiences, and goals while helping organizations better understand and support the people who make up their workforce.

Employees perform at their best when they feel psychologically safe, respected, and seen. Yet Gallup research has found that only 20% of employees strongly agree that they feel connected to their organization’s culture. That means that most employees are showing up to work without a strong sense of belonging or connection.

For LGBTQ+ employees in particular, the stakes can be even higher. Research shows that approximately one-third of LGBTQ+ professionals report feeling exhausted from hiding or masking aspects of their identity in the workplace. Nearly 40% have also experienced workplace harassment or discrimination at some point in their careers. When employees do not feel safe enough to show up authentically, organizations not only lose employee morale but also trust, engagement, creativity, collaboration, and retention.

People should not have to compartmentalize who they are to succeed professionally. ERGs help address that gap by creating community and visibility within organizations. They provide spaces where employees can share experiences, mentor one another, discuss challenges, celebrate culture, and build meaningful support systems. For many employees, particularly those from historically marginalized communities, simply knowing they are not alone can significantly improve their workplace experience.

But the value of ERGs extends beyond individual belonging. These groups can also serve as powerful organizational partners. Employees closest to a particular community often have important insights about workplace culture, policies, communication practices, recruitment, retention, accessibility, and customer/client experiences. ERGs create structured pathways for organizations to hear these perspectives directly, rather than making assumptions about what employees or communities need.

ERGs also create opportunities for employees to grow as leaders, advocates, and change agents. Employees often gain experience in strategic planning, facilitation, mentorship, and organizational advocacy through their involvement in these groups. In turn, organizations benefit from stronger leadership pipelines and more engaged employees.

However, ERGs cannot thrive without meaningful organizational support. Inclusion work should not rest solely on the shoulders of employees volunteering their emotional labor. Leadership buy-in, visibility, dedicated resources, and a willingness to act on employee feedback are essential. Employees need to see that their voices are not only welcomed but also valued enough to influence change.

As workplaces continue evolving, organizations have a responsibility to create environments where employees can show up fully as themselves while helping shape stronger, more inclusive cultures.

Hot Tips

If your organization does not currently have an ERG or EIG, consider advocating for one! Start conversations, build community, and identify shared needs and opportunities for support. Change often begins with employees willing to speak up and create space for others to feel seen, heard, and valued.

Belonging is not a workplace luxury. It is foundational to healthy organizations, effective leadership, and sustainable impact.

Rad Resources

• What Percentage of Employees Feel Connected to Their Workplace Culture?
• Over Half of LGBTQ+ Americans Are Bisexual. Many Don’t Feel Safe Being Out at Work

The American Evaluation Association is hosting LGBTQ+ Voices in Evaluation TIG Week with our colleagues in the LGBTQ+ Voices in Evaluation Topical Interest Group. The contributions all this week to AEA365 come from our LGBTQ+ Voices in Evaluation TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.