LEO + Partners

If you were to create an organization from scratch, you probably wouldn’t design it to be like LEO. It’s mission is noble and it’s “customers” are inspiring, but LEO only succeeds if it can convince its customers—nonprofits who work with those in need—to open themselves up for evaluation. Rigorous evaluation not only takes effort, but it’s outcomes can be existential; some of the programs aren’t having the impact everyone involved had assumed. 

Over the course of almost a decade, LEO has worked with partners from New York to Sacramento, on everything from foster care to housing to college completion. LEO’s papers have gone on to attract funding for these causes, changing government policy, and catalyzing expansion for some of the best performers. But almost none of these studies were possible without the openness and bravery—as well as significant effort—from LEO’s partners. 

Photo Steve Donisch Chicago Jesuit Academy

In this quarter’s issue, we dive into the nature of this partnership in a way you might not expect. Instead of what qualities make a good partner for LEO, we look at LEO through the lens of our nonprofits themselves. How does it feel to be evaluated, and is there a way to do it more equitably, given the differences in identities, education levels, geography, and incomes between the evaluators and those being evaluated? 

To understand how LEO works with nonprofits, it helps to know Bill Evans. Calling Bill a co-founder would be factually correct, but it’d be more accurate to describe him as one of LEO’s vital organs. But if you had to pick whether he’s the brain, soul, or heart of LEO, you’d be hard-pressed to choose just one. 

Over the course of dozens of LEO partner interviews, the term “teddy bear” has come up too many times to ignore.* Of course, when people describe someone as a teddy bear, they’re often doing so because there’s something about them that first screams its opposite: grizzly bear. 

For Bill, it’s not his physique, but his overflowing intellect that intimidates. His voice booms and cuts, he disagrees quickly and liberally, and he’s often a step or two ahead of the conversation. Luckily for his colleagues, Bill’s somewhat abrasive style is tempered by his youthful curiosity, an endearing restlessness, and a penchant for breaking into song during meetings. Luckily for the world, Bill’s gifts are focused on solving poverty.

Bill’s also a steadfast man of faith, whose story about coming to Notre Dame features the Holy Spirit as a protagonist. (The acronym “LEO” stuck once Bill realized that Saint Leo of Catania was famed for his work with the poor.) Bill’s faith also seems to map on how he chooses partners to work with. He relishes discovering a David—for example, a small collective of nonprofits in a tiny South Dakotan town—that courageously takes on Goliath. But it also informs how he engages with partners, channeling another term taken from the Catholic tradition: accompaniment. 

“Where most researchers go wrong,” Bill says, “is thinking that they don’t stand to learn more from the nonprofits than they do from us.” 

Accompaniment refers to the act of solidarity with those experiencing poverty. To quote an influential figure in Catholic social teaching and Notre Dame Professor Emeritus Gustavo Guittierez, “Solidarity with the poor means not to try to be the voice for the voiceless, rather that those who have no voice today will have a voice and be heard.” The concept of accompaniment exists in many religious traditions, and the message is the same: to serve others, you must walk with them, not ahead of them. 

I didn’t want to open the piece with the term partnership, or worse, accompaniment. Every research organization fancies itself a brilliant “partner” to the organizations it evaluates. Just as a hotel might refer to its staff as “ambassadors,” or an airline’s passengers as “guests.” It smacks of marketing and condescension. And most of the nonprofits we work with have a horror story or two about an evaluation they went through before working with LEO. 

The worst-case scenario when working with academics, as one of our partners’ head of impact—who asked for their identity to be disguised so as not to offend anyone—says is “feeling like you’re a lab rat.” Once or twice before, their organization has partnered with academics who “show up with an idea they want to study, an outcome they want to measure, and a laundry list of to-do’s” that they and their staff are expected to add onto their existing workload. 

The project ended up having three negative outcomes for the organization: it demotivated the staff, already overworked before layering on another task unrelated to their clients. Additionally, it soured them on measurement and evaluation, a bias they had to work hard to overturn once their work with LEO was on the table. Worst of all, the results weren’t particularly helpful to the organization—while the study no doubt added to the academic literature on poverty, it had little relevance for program design. They felt used and left empty-handed. 

Carrie Cihak has spent much of her career thinking about evaluation. As a Fellow at Stanford’s Causal Inference for Social Impact Lab, she looks for ways for academics and government officials to collaborate to create evidence-based policy. As Head of Learning and Impact, Strategy, and Partnerships for the transit division in King County Washington—home to the Seattle metro area and one of the largest transit agencies in the country—she seeks to lead by example, designing the county’s policies with evaluation in mind from the start. 

Carrie and her colleagues also hold an unofficial record for having the most LEO projects under her belt: King County has done multiple projects with LEO, spanning from housing and homelessness to transportation. Over the years, Carrie has been a part of dozens of evaluations on King County programming, and in her words “partnership doesn’t even begin to describe the relationship we’ve had with LEO.” What’s different about the way King County experiences LEO as a partner? Carrie describes three examples which highlight what makes a LEO partnership special.

1. Letting partners set priorities. According to Carrie, “LEO wants to know our outcomes of interest, versus coming in with their own intent.” For example, in their ORCA LIFT income-based discounted transit fare program, LEO and King County not only found that people who were provided fully subsidized fares used the transit system twice as much, they also discovered people who received the pass and lived in neighborhoods without a transit stop had no impact on their transit use. These results are informing how King County Metro is scaling up availability of fully subsidized passes for lowest-income people in the region, as well as other interventions to help people access transit.  

2. Listening versus telling. “It’s one thing to say you value the expertise of the partner and the clients they serve, but it’s another entirely to integrate with the team.” Carrie recalls that working with LEO isn’t a one-off conversation meant to extract data, but an enmeshing with the stakeholders; LEO and Carrie’s team meet weekly for the duration of the studies—which often take years, not months.

3. Partnership at all levels and from both directions. LEO’s involvement doesn’t just stop with research; it spans across other functions, such as grant-making, and staff exchanges. Carrie continues: “King County has benefited tremendously from people like Melissa and Katie at LEO helping us apply for grants and guaranteeing project start-up funding before we identify external funders. But there’s more—we’ve even housed LEO interns for the summer and research associates on several occasions.”

Carrie’s organization struggles with many of the same dynamics that LEO faces, in particular: how can you balance the differences in privilege, race, education, and income that exists between those helping and those experiencing poverty? While Seattle is one of the whitest large cities in the U.S., the larger King County metro area includes some of the most diverse zip codes in the country. Seattle is one of the least racially diverse major metropolitan areas in America; Carrie’s team targets the areas where the needs are greatest. To do so, King County calls upon LEO, not just to evaluate, but to partner though the evaluation life cycle—from design to execution to measurement. 

LEO believes its approach—call it partnership, or accompaniment—stands out from many of its peers in the space. In the words of LEO’s partners, its defining characteristics are respect, humility, and a willingness to meet them where they are. In Bill’s words: “It’s being smart enough to realize you don’t know enough.” 
 

*For the record, I also heard more than one partner refer to Bill as a “research god.”