We discuss specific steps in results-driven contracting with Lisa Morrison Butler, the former Commissioner of the Chicago Department of Family and Support Services (DFSS) and now the Executive Vice President and Chief Impact Officer at Results for America. She served as Commissioner of DFSS — the largest funder of human and social services in the city — for more than five years across two mayoral administrations. Under her leadership, she led an organizational transformation aimed at achieving greater overall impact, including overhauling the department’s approach to contracting.

The five main steps in DFSS’s path to results-driven contracting that we discuss are:

1. Build consensus around the need for change related to contracting.
2. Choose a pilot division.
3. Develop a calendar to roll out reform across the department.
4. Build clear goals and new expectations into each new RFP, among participating divisions.
5. Adopt active contract management, including developing learning cohorts that met quarterly.

Learn more:

• To hear other Gov Innovator podcast interviews on results-driven contracting, click this link.
• Read Lisa’s “Innovator Interview” with the Harvard Kennedy School’s Government Performance Lab on her efforts to improve outcomes at Chicago’s DFSS.

The post How results-driven contracting improved outcomes for Chicago’s most vulnerable: An interview with Lisa Morrison Butler, former Commissioner, Chicago Department of Family and Support Services – Episode #202 appeared first on Gov Innovator Podcast.

Rabbi Margolius presents the three step process used by IJS:

• First, adopt a stance of non-judgmental curiosity by observing one’s own mind and reactions. This can be done by simply pausing.
• Next, be aware of having the possibility of choice, realizing that we have a range of options in responding to a stimulus.
• And finally, choose the value or set of values that one wants to use in responding.

Together, these steps can help us to have a response rather than a reaction. The approach draws at times from Jewish spiritual practice, but it can be useful for anyone, whatever one’s faith background or connection (or not) to a religion.

A transcript of our conversation is available here.

Related resources: An article co-authored by Andy Feldman and Rabbi Margolius in Government Executive on the value of mindfulness practices is here. The magazine also interviewed the co-authors for their podcast, available here.

The post Using mindfulness practices for greater self-awareness and self-improvement: Rabbi Marc Margolius, Senior Program Director, Institute for Jewish Spirituality – Episode #201 appeared first on Gov Innovator Podcast.

Dr. Goldstein (@NaomiGoldstein1) is the Deputy Assistant Secretary for Planning, Research, and Evaluation at ACF. She will retire from federal service this week after a 21-year tenure at ACF, including becoming Director of the Office of Planning, Research and Evaluation (OPRE) in 2004 and Deputy Assistant Secretary in 2015. Beyond those important career achievements, it is her kindness, humility, candor and commitment — and, I would add, all with a sense of humor — that have made her such a valued colleagued and friend to so many over the years.

A transcript of our conversation is available here.

Learn more:

• Recent OPRE blog post by Dr. Goldstein: “How Do I Love My Job? Let Me Count the Ways“
• Previous podcast interviews:
  • Why evaluation policies are useful to results-focused federal agencies (with Molly Irwin)
  • Strengthening evaluation capacity within agencies

The post Career reflections from an evidence leader: An interview with Naomi Goldstein, Administration for Children and Families, U.S. Department of Health and Human Services – Episode #200 appeared first on Gov Innovator Podcast.

• State agencies are required by the governor’s budget office to show the evidence behind requested budget changes.
• In doing that, agencies use a framework called the “Evidence Continuum” that describes the stages of building and assessing program information.
• Also, the legislature now requires agencies to use consistent definitions of evidence-based programs in their budget requests.
• And Colorado is one of only a handful of states to have a dedicated fund to support program evaluations.

A recent report by the The Pew Charitable Trusts’ Results First initiative, titled “Colorado’s ‘Evidence Continuum’ Promotes Efficient, Effective Public Programs,” discusses the state’s progress in these areas. To get an overview, we are joined by one of its authors, Sara Dube who directs the Results First initiative at the The Pew Charitable Trusts.

The post Colorado’s nation-leading practices in using evidence to guide budget decisions: An interview with Sara Dube, Director, Results First Initiative, The Pew Charitable Trusts – Episode #199 appeared first on Gov Innovator Podcast.

To do that, we are joined by Dr. Michelle Miller-Adams (@mmilleradams). She is the co-director of the Upjohn Institute’s place-based research initiative and is a national expert on tuition-free college initiatives. Her latest book is The Path to Free College: In Pursuit of Access, Equity, and Prosperity.

The post How state tuition-free college programs are spanning political divides: An interview with Michelle Miller-Adams, Senior Researcher, Upjohn Institute – Episode #198 appeared first on Gov Innovator Podcast.

To learn more about these important policies and the evidence behind them, we are joined by someone who has examined them closely, Dr. Elizabeth Bell (@PoliSciBell). She is an Assistant Professor at the Askew School of Public Administration and Policy at Florida State University and also serves as an Academic Affiliate for the Office of Evaluation Sciences within the federal government.

The post How Promise Programs are expanding college access, affordability, and degree attainment: An interview with Elizabeth Bell, Florida State University – Episode #197 appeared first on Gov Innovator Podcast.

To hear lessons from that work and also broader observations from someone working to advance evidence-based policy for almost two decades, we are joined by David Anderson. He is the Vice President of Evidence-Based Policy at Arnold Ventures and previously served as the Vice President of the Coalition for Evidence-Based Policy.

The post Insights about building credible evidence in social policy: An interview with David Anderson, Vice President of Evidence-Based Policy, Arnold Ventures – Episode #196 appeared first on Gov Innovator Podcast.

One important piece of that story is how the state launched an Impact Evaluation Unit within  Minnesota Management and Budget (MMB). The Impact Evaluation Unit helps lead the development of program inventories used for evidence-based budgeting and also conducts rigorous program evaluations to help the state learn what works.

To learn more, we are joined by Dr. Weston Merrick (@WestonMerrick). He is a senior manager at MMB and leads the Impact Evaluation Unit.

The post How Minnesota’s impact evaluation unit supports evidence-based budgeting: An interview with Weston Merrick, Minnesota Management and Budget – Episode #195 appeared first on Gov Innovator Podcast.

In our interview, she discusses the mission of her former office and then provides three pieces of advice based on her experience: 1) Use evidence to create a cycle of continuous improvement; 2) Understand that using evidence can be messy; and 3) See the use of evidence as a team effort. She also provides advice for state education departments that are earlier on the path to doing this type of work.

The post Insights on evidence use from Ohio’s Department of Education: An interview with Heather Boughton, Results for America – Episode #194 appeared first on Gov Innovator Podcast.

To learn more, we’re joined by Elizabeth Funsch. She is the founding manager of ArudelStat and, prior to that, served as a senior performance manager in Prince George’s County in Maryland.

The post Developing and using agency performance plans: An interview with Elizabeth Funsch, ArundelStat Manager, Anne Arundel County in Maryland – Episode #193 appeared first on Gov Innovator Podcast.

Since 2016, Dr. Jenkins has been the Director of the Office of Performance and Evaluation at the Administration for Community Living (ACL) at the U.S. Department of Health and Human Services. She also teaches program evaluation at American University and, prior to ACL, served on the evaluation staff of the Peace Corps.

ACL’s mission is to maximize the independence, well-being, and health of older adults, people with disabilities, and their families and caregivers. Both ACL and Dr. Jenkins have been recognized by Results for America for their leadership in creating a culture of evidence-based decision making.

The post Three keys to a successful agency evaluation office: An interview with Susan Jenkins, Director of the Office of Performance and Evaluation, Administration for Community Living, HHS – Episode #192 appeared first on Gov Innovator Podcast.

While the program ended more than 20 years ago, its lessons, including from a five-year randomized evaluation by MDRC and from longer-term research on impacts for children and youth, are especially important today as the nation struggles to tackle racial and economic inequalities exacerbated by the pandemic.

A new brief published by Georgetown’s Center on Poverty & Inequality reexamines the lessons from New Hope and we’re joined by two of its authors. Julie Kirksick (@kerksick) served as New Hope’s Executive Director from 1997 to 2008 and as Associate Director prior to that. Since New Hope, she has held senior roles in human services agencies in both Wisconsin and Colorado. And Kali Grant (@kaligrant) is a Senior Policy Analyst at the Georgetown Center on Poverty and Inequality.

The post Lessons from New Hope for reducing poverty today: An interview with Kali Grant, Georgetown University, and Julie Kirksick, former New Hope Executive Director – Episode #191 appeared first on Gov Innovator Podcast.

• Sharing an example of how credible research influenced policymaking on a key issue in New Mexico, pre-school education.
• How an important piece of good-government legislation — the New Mexico Evidence and Research-Based Funding Requests Act of 2019 — has spurred the development of program inventories in high-priority policy areas that support evidence-based budgeting.

The post Evidence-based budgeting in New Mexico: Part 2 interview with Charles Sallee, Deputy Director, Legislative Finance Committee, New Mexico – Episode #190 appeared first on Gov Innovator Podcast.

New Mexico’s LFC adapted PerformanceStat to a legislative setting, dubbing it “LegisSTAT.” To learn more, we’re joined by Charles Sallee. For more than a decade he has served as the LFC’s Deputy Director.

Learn more: An op-ed in Governing, written by Charles Sallee, LFC Chair Patricia Lundstrom and Andy Feldman discusses the piloting of LegisSTAT in New Mexico.

The post Launching LegisSTAT in New Mexico: Part 1 interview with Charles Sallee, Deputy Director, Legislative Finance Committee, New Mexico – Episode #189 appeared first on Gov Innovator Podcast.

Alexander Eaton, director of Retina Health Centers, said of its final approval. STP is believed to act by two main mechanisms: (1) direct GABAergic effect [ Quilichini et al. One eye from each subject was chosen as the study eye, and only data for the study eye were used for the efficacy analysis apteekki netissä Suomessa. This can block blood flow, leading to a pain crisis.

Any serious active infection including an illness with a fever is a reason for delaying use of the hepatitis B vaccine, recombinant except when in the opinion of the prescriber, withholding the vaccination entails a greater risk. Oligospermia may occur at high doses. Their side effects are different from chemotherapy (sometimes less severe) and many are taken as a pill https://apotek-sverige.org/cialis-original/. Tranexamic acid is not recommended in children with severe renal impairment (see Section 5.

Clinical monitoring Before starting treatment with clopidogrel, your doctor may do a genetic test to check your CYP2C19 genotype. Wellbutrin Trintellix and Wellbutrin (bupropion) are both prescription antidepressant medications. It is a steroidal antimineralocorticoid of the spirolactone group and a selective aldosterone receptor antagonist (SARA) http://www.sayadlia24.com/. New insights into the burden and costs of multiple sclerosis in Europe.

A vegetarian diet which contains no animal products (including milk products or eggs) does not supply any vitamin B 12. Priming a new bottle removes trapped air so you get the correct amount of medicine in each dose. Biomaterials 35, 456–465 (2014) Further evaluate patients who develop exertional chest pain, unexplained syncope, or arrhythmias during treatment with DYANAVEL XR, Quillivant XR, and QuilliChew ER.

The most frequently reported adverse reactions resulting in clinical intervention (for example, discontinuation of AVONEX or the need for concomitant medication to treat an adverse reaction symptom) were flu-like symptoms and depression. Blood pressure should be rechecked supine at a 30-degree angle if increased doses are required. Your doctor may tell you to take acetaminophen (Tylenol), an over-the-counter pain and fever medication, to help with these symptoms . Return to Pharmacotherapy Update Index References • Product labeling.

Your doctor might change the dosage of your heart rhythm drugs. The adverse event most frequently associated with discontinuation (occurring in three patients) was somnolence. The shift from a low or normal value at baseline to an increased CPK value was greater for DUOPA-treated patients (17%) than for patients treated with oral immediate-release carbidopa-levodopa (7%) . Do not inject this medicine into skin that is damaged, tender, bruised, pitted, thickened, scaly, or has a scar or hard lump.

Paroxetine comes as a tablet, a suspension (liquid), a controlled-release (long-acting) tablet, and a capsule to take by mouth. Those applying for access must be in professional medical employment or study. It is used to distinguish a pituitary source of excessive ACTH secretion from a different source https://pharmacie-enligne.org/super-kamagra/. This activity describes the indications, action, and contraindications for benzocaine as a valuable agent in managing pain control.

Many of you might know his book Leading Change, which is a classic on the topic and describes eight steps for successfully leading change initiatives. Those steps include: 1) Create a sense of urgency; 2) Build a guiding coalition; 3) Form a strategic vision and initiatives; 4) Enlist a volunteer army; 5) Enable action by removing barriers; 6) Generate short-term wins; 7) Sustain acceleration and 8) Institute change.

His new book, coming out this month, is called Change: How organizations achieve hard to imagine results in uncertain and volatile times. It covers many topics, including how to make strategic planning more successful and impactful, as we discuss in the interview.

The post Leading successful change initiatives: An interview with John Kotter, professor emeritus at the Harvard Business School and founder of Kotter International – Episode #188 appeared first on Gov Innovator Podcast.

To learn more, we’re joined by Rakesh Mohan (@RakeshMohanEval). He’s been the director of the office since 2002. OPE was the recipient of the American Evaluation Association’s Outstanding Evaluation Award and Rakesh himself was the recipient of the Outstanding Practitioner Award from the American Society for Public Administration.

Web extras: We discuss additional details about OPE and its work, including the use of contractors to support the work [click here] and the steps from project selection to report creation [click here].

Learn more: A longer interview with Rakesh Mohan hosted by James Pann is available here.

The post Insights from Idaho’s Office of Performance Evaluations: An interview with Rakesh Mohan, Director – Episode #187 appeared first on Gov Innovator Podcast.

Our interview focuses on five questions that organizations can ask, using the logic models they have created:

1. Are we doing the right work?
2. Are we doing the work right?
3. Are our products or services accessed and used as intended?
4. What difference are we making?
5. What will it take to sustain our work? And, relatedly, what have we learned?

The post Five questions public agencies should ask to put their logic models to work: An interview with Cynthia Phillips, former Acting Chief Evaluation Officer, National Science Foundation – Episode #186 appeared first on Gov Innovator Podcast.

• confirmation bias
• status quo bias
• bandwagon bias

Understanding those biases — which have been well-researched in the past, but are newly being applied to the context of evidence use by decision makers — can help public leaders make better decisions. To learn more, we’re joined by Dr. Ariel Kalil (@ariel_kalil). She’s a developmental psychologist and a professor at the University of Chicago’s Harris School of Public Policy and directs the school’s Center for Human Potential and Public Policy and co-directs the Behavioral Insights and Parenting Lab. She’s also an adjunct professor in the Norwegian School of Economics in Bergen, Norway.

The post Three cognitive biases that can influence decision makers’ use of evidence about what works: An interview with Ariel Kalil, University of Chicago – Episode #185 appeared first on Gov Innovator Podcast.

Additional resources:

• The budget forms that includes questions about evidence are available from Tennessee’s Department of Finance and Administration website here. Archived copies are also available here and here.
• The evidence framework is available from the Office of Evidence and Impact website here. An archived copy is also available here.

The post Insights from Tennessee’s Office of Evidence and Impact: An interview with Christin Lotz, Director – Episode #184 appeared first on Gov Innovator Podcast.

To learn more, we’re joined by two of the brief’s authors. Carly Robinson (@Carly__Robinson) is a postdoctoral research associate at Brown University and Matthew Kraft (@MatthewAKraft) is a professor of education and economics at Brown. Both are former public-school teachers.

Learn more: The brief is part of a series called EDResearch for Recovery produced by Brown, UVA and Results for America that’s designed to provide K-12 education decision makers and advocates with credible evidence help students respond to and recover from the COVID-19 pandemic.

Related initiative: The newly launched National Student Support Accelerator provides comprehensive resources for those interested in implementing high-impact tutoring.

The post Boosting student achievement with high-dosage tutoring: An interview with Carly Robinson and Matthew Kraft, Brown University – Episode #183 appeared first on Gov Innovator Podcast.

One way to help support the use of behavioral insights within an agency is to create a community of practice. To learn more, we are joined by Melissa Leal, a change management specialist at the Colorado Department of Public Health and Environment (CDPHE). In 2019 she led the formation of a community of practice around behavioral insights within the the department.

Related interview: CDPHE is using the Behavioral Insights Team’s EAST framework in its current project, aimed at improving the health of its employees. The EAST framework was discussed on the podcast here.

The post Launching a behavioral insights community of practice within a public agency: An interview with Melissa Leal, Colorado Department of Public Health and Environment – Episode #182 appeared first on Gov Innovator Podcast.

Among the topics he’s explored is how to improve written communications with practical purposes. His recent op-ed in the Boston Globe, co-authored with Jessica Lasky-Fink, is called “Write shorter messages.” As a preview of their insights, they find that keeping communications short, simple and skimmable makes it more likely that people will respond or take action.

The post How to communicate so that busy people respond and take action: An interview with Todd Rogers, behavioral scientist and professor, Harvard Kennedy School – Episode #181 appeared first on Gov Innovator Podcast.

To learn more, we’re joined by Janey Rountree, the Executive Director of the California Policy Lab at UCLA, and Evan White (@EvanBWhite) the Executive Director of the California Policy Lab at UC Berkeley. They work in close collaboration with the Lab’s faculty directors, Till von Wachter and Jesse Rothstein.

The post How the California Policy Lab is helping state and local agencies tackle homelessness, poverty and other key policy challenges: An interview with Janey Rountree and Evan White, Executive Directors – Episode #180 appeared first on Gov Innovator Podcast.

This example has a specific policy focus — higher education access – but the steps that HUD has taken to build evidence provides lessons for any public agency, whether federal, state or local, that is working to tackle important policy issues and using evidence to inform their decision making.

To learn more, we’re joined by Dr. Calvin Johnson. He’s the Deputy Assistant Secretary for Research, Evaluation and Monitoring within the Office for Policy Development and Research (PD&R) within HUD. He is a leader within the federal evidence community and a fellow of the National Academy of Public Administration.

Additional information: Read about HUD’s work with the Office of Evaluation Sciences to design low-cost behaviorally informed interventions — letter, in this case — to encourage FAFSA enrollment [link].

The post Lessons from HUD about building evidence on important social policy issues: An interview with Calvin Johnson, Deputy Assistant Secretary for Research, Evaluation and Monitoring at HUD – Episode #179 appeared first on Gov Innovator Podcast.

To learn more, we are joined by Dr. Dayanand Manoli. He’s an economist and a professor at the McCourt School of Public Policy at Georgetown University and has collaborated with the IRS on several research projects, as we highlighted on this podcast in 2016. Recently, he co-hosted, with Kathy Stack, a workshop at the Partnership for Public Service on the topic of IPAs within federal agencies.

The post How federal agencies can use IPAs to bolster evidence capacity and help implement the Evidence Act: An interview with Dayanand Manoli, Professor, Georgetown University – Episode #178 appeared first on Gov Innovator Podcast.

To learn more, we are joined by the office’s founding director, Jenni Owen (@jenniowen). She was previously the Governor’s Policy Director and, before that, was on the faculty and the director of policy engagement at the Sanford School of Public Policy at Duke University.

The post The role of North Carolina’s Office of Strategic Partnerships: An interview with Jenni Owen, Director of Strategic Partnerships, North Carolina – Episode #177 appeared first on Gov Innovator Podcast.

Web extras: Mr. Padrino discusses how the efforts have continued and advanced in the new administration, under Governor Polis. [click here]. He also discusses the connections and differences between performance measurement / management and evidence-based policy. [click here]

Learn more: Colorado is featured in Results for America’s Blueprint for Delivering Results in State Government, released in 2020. Also see the related Gov Innovator interview with Henry Sobanet about Colorado’s lean initiative.

The post Colorado’s performance management strategy under Gov. Hickenlooper: An interview with David Padrino, former Chief Performance Officer, Colorado – Episode #176 appeared first on Gov Innovator Podcast.

To learn more, we are joined by Arun Sambataro, the Senior Equity and Social Justice Policy Advisor within the Office of Equity and Social Justice.

The post King County, Washington’s Equity and Social Justice Strategic Plan: An interview with Arun Sambataro, King County – Episode #175 appeared first on Gov Innovator Podcast.

To learn more about the findings and where we are today with evidence-based policymaking on the cusp of a new presidential administration, we are joined by Jed Herrmann (@HerrmannJed). He is the Vice President for State and Federal Policy Implementation at Results for America. He previously served at both the federal and local levels, including as a Senior Advisor to the CEO at the Corporation for National and Community Service and Chief of Staff and Deputy Commissioner at the New York City Department of Consumer Affairs.

The post Investing in what works at the federal level: An interview with Jed Herrmann, Vice President for State and Federal Policy Implementation, Results for America – Episode #174 appeared first on Gov Innovator Podcast.

To learn more about the county’s journey with data-driven decision making and what lessons we can draw for other jurisdictions, we’re joined by Dave Gottesman. He’s been the CountyStat manager since 2012. He also co-leads the Mid-Atlantic Stat Network, a group of PerformanceStat staff from the region that convenes periodically to share their insights. Dave was previously on the podcast in 2013.

The post Insights from nearly a decade of CountyStat: An interview with Dave Gottesman, CountyStat Manager, Montgomery County, Maryland – Episode #173 appeared first on Gov Innovator Podcast.

Dr. O’Hara is a Research Professor in the Massive Data Institute and Executive Director of the Federal Statistical Research Data Center at the McCourt School for Public Policy at Georgetown University. She was previously a senior executive at the U.S. Census Bureau where she founded the Center for Administrative Records Research and Applications (CARRA), the Bureau’s administrative data unit.

The post Unlocking the power of big data to catalyze evidence-based policy: An interview with Amy O’Hara, Georgetown University – Episode #172 appeared first on Gov Innovator Podcast.

As background to our interview, a growing number government agencies and jurisdictions are taking an experimental approach to driving better results. For example, in making operational improvements to programs, they’re choosing an approach or strategy by rigorously testing different approaches and seeing which one works best. (That’s in contrast to choosing an approach based on best guesses or anecdotes or the opinion of the most senior person in the room.) For examples at the federal level, for instance, see those catalogued by the Office of Evaluation Sciences. An experimental mindset, however, is still nascent in most of government. Looking to leading examples and lessons learned from public and private organizations–including the companies studied by Dr. Thomke–can help more government leaders build an experimental culture.

The post Creating a culture of experimentation within organizations: An interview with Stefan Thomke, Harvard Business School – Episode #171 appeared first on Gov Innovator Podcast.

The Behavioral Insights Team, in conjunction with the What Works Cities Initiative, set out to work with more than 20 police departments to build credible evidence on that topic, including running randomized trials to test out different recruiting messages and other steps. The results are described is the report Behavioral Insights for Building the Police Force of Tomorrow.

To learn more, we are joined by Dr. Elizabeth Linos (@ElizabethLinos) who was one of the authors of the report. She is a professor of public policy at University of California, Berkeley, and is the former vice president and head of research and evaluation at the Behavioral Insights Team in North America.

The post Using behavioral insights to help recruit diverse police candidates: An interview with Elizabeth Linos, University of California, Berkeley – Episode #170 appeared first on Gov Innovator Podcast.

To learn more, we are joined by Jason Bossie, who serves as the Director of SBA’s Office of Performance Management, serving under CFO Tim Gribben.

Web extra: Jason discusses the SBA’s acquisition vehicle for program evaluation and why it’s been useful. [click here]

The post How the Small Business Administration became a leader in evidence-based decision making: An interview with Jason Bossie, Director, Office of Performance Management, SBA – Episode #169 appeared first on Gov Innovator Podcast.

The State of Oregon was the first out of the gate to do that. In 2017, it launched OregonSaves, a savings plan that covers private sector workers who do not otherwise have access to a savings plan in their workplace. Eight states have similar programs in the works, including California and Illinois, which are expected to start their versions in 2018. To learn more about OregonSaves, we are joined by its founding executive director, Lisa Massena.

The post How Oregon’s auto-enrollment IRA program, OregonSaves, helps state residents save for retirement: An interview with Lisa Massena, Executive Director, OregonSaves – Episode #168 appeared first on Gov Innovator Podcast.

We profile two such programs that are making an important difference, as shown by rigorous program evaluations. They are the City University of New York’s (CUNY’s) Accelerated Study in Associate Programs (ASAP) and the Dell Scholars program. We are joined by two researchers who helped lead the respective evaluations of these programs. Lindsay Page (@linzcpage) is a professor at the School of Education at the University of Pittsburgh and Michael Weiss (@MDRC_News) is a Senior Associate at MDRC.

Additional resources: Learn more by accessing MDRC’s reports on CUNY ASAP [click here] and the study of the Dell Scholars Program by Lindsay Page and her co-authors, Stacy Kehoe, Benjamin Castleman and Gumilang Sahadewo [click here].

The post Two promising strategies to promote college success for disadvantaged students: An interview with Lindsay Page, University of Pittsburgh, and Michael Weiss, MDRC – Episode #167 appeared first on Gov Innovator Podcast.

To learn more, we are joined by Will Yang. He has been an Innovation and Design Consultant at HHS for more than five years and now leads the CoLab.

Additional resource: A recent FedScoop article discusses Will Yang’s reflections on the first cohort of the CoLab [click here].

Related interview: An earlier Gov Innovator podcast interview with Bryan Sivak, then CTO of HHS, discusses implementing a department-wide innovation strategy [click here].

The post How HHS’s Data Science CoLab catalyzes employee innovation: An interview with Will Yang, CoLab Director, U.S. Department of Health and Human Services – Episode #166 appeared first on Gov Innovator Podcast.

To walk us through the most notable examples, we are joined by Nick Hart (@NickRHart), the director of the Evidence-Based Policymaking Initiative at the Bipartisan Policy Center (BPC).

Additional resources: For more information, see overviews by BPC [click here], Results for America [click here] and three posts from Feb.  9th from the Social Innovation Research Center [click here].

The post Wins for data and evidence-based policy in the bipartisan budget deal: An interview with Nick Hart, Bipartisan Policy Center – Episode #165 appeared first on Gov Innovator Podcast.

To learn more about evaluation policies and why they are useful to federal agencies, we are joined by Naomi Goldstein, the Deputy Assistant Secretary within the Office of Planning, Research, and Evaluation (OPRE) at the Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services; and Molly Irwin, the Chief Evaluation Officer at the U.S. Department of Labor.

Additional resources: ACF’s evaluation policy is available on the OPRE website [click here]. DOL’s evaluation policy is available on the department’s Chief Evaluation Office website.[click here].

The post Why evaluation policies are useful to results-focused federal agencies: An interview with Naomi Goldstein, HHS, and Molly Irwin, DOL – Episode #164 appeared first on Gov Innovator Podcast.

A good example comes from South Carolina. Its Medicaid program is administered through Managed Care Organizations, which offer different health care plans to Medicaid beneficiaries. What happens when people don’t choose a plan? In those cases, the state has begun randomly assigning those individuals to plans. It’s not only a fair way to make those assignment decisions, but it also allows researchers to build credible evidence about the different Medicaid plans — and about the state’s star ratings of those plans.

To learn more, we are joined by Matthew Notowidigdo, and an affiliated professor at J-PAL. With Craig Garthwaite of Northwestern, he has been studying South Carolina’s Medicaid system as part of JPAL-North America’s State and Local Innovation Initiative.

The post How program managers can use random assignment (i.e., a lottery) to build evidence that can improve customer service: An interview with Matthew Notowidigdo, Northwestern University – Episode #163 appeared first on Gov Innovator Podcast.

To learn more, we are joined by two of the co-authors, Amy Laitinen (@amylaitinen1) and Clare McCann (@claremccann), who are respectively the director and deputy director for higher education with the Education Policy program at New America.

The post Reforming the federal Experimental Sites initiative to better learn what works in higher ed: An interview with Amy Laitinen and Clare McCann, New America – Episode #162 appeared first on Gov Innovator Podcast.

To learn more, we are joined by Laura Wentworth (@laurawent). Since 2009 she has worked for California Education Partners as the director of the partnership. Ed Partners is the third party organization that supports collaboration in the education sector across California. Laura is a former public school teacher and earned her PhD in administration and policy analysis in education from Stanford.

The post Insights from a leading researcher-practitioner partnership, between Stanford University and San Francisco’s school district: An interview with Laura Wentworth, California Education Partners – Episode #161 appeared first on Gov Innovator Podcast.

Some of the Commission’s key recommendations focus on making the most of the data the government already collects by giving qualified researchers—including academics as well as evaluation experts within government—greater access to data from government programs and surveys. At the same time, the CEP calls for strengthening privacy protections to ensure that those data are not misused. It also recommends ways that departments can increase their evidence capacity, meaning their ability to use and build evidence about what works.

To learn more, we are joined by Nick Hart (@NickrHart) who served as the Policy and Research Director for the Commission. Today he is the director of the Bipartisan Policy Center’s Evidence-Based Policymaking Initiative.

The post A primer on the Commission on Evidence-Based Policymaking’s recommendations: An interview with Nick Hart, Bipartisan Policy Center – Episode #160 appeared first on Gov Innovator Podcast.

• Skill building: Identifying key skills and building them in an intervention, producing impacts into the future. That might include analytical thinking, delayed gratification delay or grit
• Foot in the door: Designing the right intervention at the right time to help a child or adolescent through a period of risk or opportunity, such as interventions that keep young people from repeating grades.
• Sustaining environments: Providing additional interventions that build on the gains of the initial intervention, essentially creating “recharging stations” to sustain initial gains.

To learn more, we are joined by Greg Duncan. He is a Distinguished Professor in the School of Education at the University of California, Irvine.

Related interviews: Also see Greg Duncan’s interviews on how states can optimize their pre-K programs [click here] and how successful school systems are closing achievement gaps [click here].

The post Strategies to sustain program impacts for children and adolescents: An interview with Greg Duncan, Professor, University of California, Irvine – Episode #159 appeared first on Gov Innovator Podcast.

To learn more about global trends in impact bonds, we are joined by Emily Gustafsson-Wright (@EGWBrookings), a Fellow at the Center for Universal Education at the Brookings Institution. She is the co-author of the recent report, The Potential and Limitations of Impact Bonds: Lessons from the First Five Years of Experience Worldwide.

The post The use of impact bonds around the world: An interview with Emily Gustafsson-Wright, Fellow, Center for Universal Education, The Brookings Institution – Episode #158 appeared first on Gov Innovator Podcast.

The bipartisan Every Student Succeeds Act (ESSA) was enacted in December 2015. ESSA gives states more opportunities to design their own educational systems, while also encouraging and sometimes requiring them to use evidence-based approaches that can help improve student outcomes.

Our guest today, Thomas Kane, joins us for part two of our conversation about how states can use ESSA to focus education spending on what works. Our earlier conversation on the podcast focused on how states can use “efficacy networks” to test strategies for school improvement. Today we focus on another important strategy for state education leaders. Tom Kane is a professor of education and economics at the Harvard Graduate School of Education and faculty director of the Center for Education Policy Research at Harvard.

The post How states can use ESSA to focus education spending on what works: An interview with Tom Kane, Professor, Harvard Graduate School of Education – Episode #157 appeared first on Gov Innovator Podcast.

To learn more, we are joined by Kelly Fitzsimmons, the founder and the managing director for network and strategy of Project Evident.

The post Helping nonprofits build and use evidence through Project Evident: An interview with Kelly Fitzsimmons, Founder and Managing Director, Project Evident – Episode #156 appeared first on Gov Innovator Podcast.

For its achievements, the Green Bank won the Innovations in American Government Award for 2017, sponsored by the Ash Center for Democratic Governance and Innovation at the Harvard Kennedy School. It plans to use the prize money to relaunch the Green Bank Academy to help more jurisdictions launch green banks.

To learn more, we are joined by Bryan Garcia, the president and CEO of the Connecticut Green Bank.

Additional resources: Mark Muro of the Brookings Institution and colleagues provide information on green banks here and here.

The post How the Connecticut Green Bank is catalyzing green energy infrastructure: An interview with Bryan Garcia, President and CEO, CT Green Bank – Episode #155 appeared first on Gov Innovator Podcast.

To get a C-Stat update, we are joined by Reggie Bicha, who has led CDHS since being appointed by Governor John Hickenlooper in 2011. Prior to his current role, he was the Secretary of the Wisconsin Department of Children and Families under Governor Jim Doyle — the first person to hold that title in the newly created department.

Related interviews:  Ki’i Powell, then-Performance Management Director at CDHS, spoke in detail about the C-Stat process in her 2014 podcast interview. [click here] Also, Reggie Bicha spoke about KidsStat at the Wisconsin Department of Children and Families in his 2013 podcast interview. [click here]

The post Insights from C-Stat in Colorado at year 5: An interview with Reggie Bicha, Executive Director, Colorado Department of Human Services – Episode #154 appeared first on Gov Innovator Podcast.

To learn more, we are joined by the Lab’s founding director, David Yokum. Prior to his work with DC, he was one of the founders of the Social and Behavioral Sciences Team at the White House, which applied insights from the behavioral sciences to improving federal agency operations.

October 2017 Update: As an example of the Lab @ DC’s work, it recently released its study on the effect of police body worn cameras. Learn more on its website and in media coverage by the New York Times. The study is a partnership between the DC police department and The Lab.

The post Launching an applied research team within city government: An interview with David Yokum, Director, The Lab @ DC – Interview #153 appeared first on Gov Innovator Podcast.

To learn more, we are joined by Kelvin Watson, the Deputy Chief Executive for Data Services at Stats NZ.

The post How New Zealand links data from public data sets to address important policy challenges: An interview with Kelvin Watson, Deputy Chief Executive, Statistics New Zealand – Episode #152 appeared first on Gov Innovator Podcast.

How was the department able to link data across programs and with providers, given what often seems like insurmountable privacy laws that can make data sharing difficult? We get insights from Erin Dalton, the Deputy Director for the Office of Data Analysis, Research and Evaluation (DARE), and Brian Bell, a supervisor within DARE and also the privacy officer at the department.

Our interview builds on an earlier conversation with Erin Dalton that provides an overview of the Data Warehouse.

The post Linking data to improve human services while working within privacy laws: An interview with Erin Dalton and Brian Bell, Allegheny County Department of Human Services – Episode #151 appeared first on Gov Innovator Podcast.

What principles can help guide public leaders—whether policymakers or public managers—in their use of evidence-based policy to improve results? Howard White (@HowardNWhite) of the Campbell Collaboration joins us to share four fundamental principles:

1. Use the right evidence to answer the right question. Different types of evidence — e.g., monitoring, process evaluation, impact evaluation and systematic reviews — all can produce useful information for decision makers. But each type of evidence should not be used to answer questions that are beyond its usefulness.
2. Don’t rely on single studies. When possible, leaders should avoid making important funding decisions based on single studies, especially those done in one site. That’s because the findings from one study are often different from those of further studies. The best approach is to use systematic reviews (where they exist), meaning syntheses of multiple high-quality studies.
3. Context matters for transferring evidence. Why do findings from one study often not replicate in another? A key reason is that context matters. For example, when a home visiting program found to be effective in the U.S. was tested in Britain, it produced no impact. Why? Likely it was the different context: Britain already provides services to low-income parents that are quite similar to the home visiting program in the U.S. It is why leaders should test out, with rigorous evaluation, programs and initiatives in their own setting, particularly if previous research was conducted in a different context.
4. Evidence-based policy is not a blueprint (aka cookie cutter) approach. This is a way of summarizing the previous two principles. While architects can take the blueprints for one building and build the same building elsewhere, and chefs can take a recipe from one restaurant and cook it in another, public leaders need to be careful when applying research from one place or setting to another.

Dr. White is the Chief Executive Officer of the Campbell Collaboration, a nonprofit best known for its use of systematic reviews to help policymakers and others make well-informed decisions. Previously he was the founding Executive Director of the International Initiative for Impact Evaluation (3ie) and led the impact evaluation program at the World Bank’s Independent Evaluation Group.

Photo credit: European Union

The post Four fundamental principles of evidence-based policy and practice, drawing from U.S. and European experience: An interview with Howard White, Executive Director, Campbell Collaboration – Episode #150 appeared first on Gov Innovator Podcast.

To get insights, we’re joined by Bob Behn. Although it is too early to comment on the Trump administration’s reorganization in particular, since it is still being formulated, he draws on lessons from other reorg efforts to provide advice to public leaders.

Bob Behn is one of the nation’s leading experts on leadership and performance management in government. He is a lecturer at the Harvard Kennedy School and the faculty chair of its executive program called Driving Government Performance. He also publishes a free monthly newsletter called Bob Behn’s Public Leadership Report.

The post The opportunities and pitfalls of government reorganization: An interview with Bob Behn, Professor, Harvard Kennedy School – Episode #149 appeared first on Gov Innovator Podcast.

To learn more, we are joined by Thaddeus Ferber, a Vice President at the Forum for Youth Investment, an organization that advocated for and helped bring about the P3 authorization.

The post An overview of Performance Partnership Pilots for Disconnected Youth, with lessons for other areas of social policy: An interview with Thaddeus Ferber, Vice President, Forum for Youth Investment – Episode #148 appeared first on Gov Innovator Podcast.

RIIPL’s goal is to use data and science to improve policy, alleviate poverty and increase equity of opportunity. To do that work, it has created a new linked database of public programs, connecting more than 100 previously independent data sets.

To learn more and hear about some example projects, we are joined my Justine Hastings (@JHastings_Econ). She is an economist at Brown University and the founding director of RIIPL.

The post How the Rhode Island Innovative Policy Lab (RIIPL) works: An interview with Justine Hastings, Director, RIIPL – Episode #147 appeared first on Gov Innovator Podcast.

In the U.K., the department of revenue and customs, in conjunction with the U.K. Behavioral Insights Team (BIT), tested new versions of the letter sent to people who were late in paying their taxes. The new versions simply added ones additional sentence, a behavioral “nudge” that drew on the power of social norms. As the New York Times explained:

One nudge was a sentence telling recipients that a majority of people in their community had already paid their taxes. Another said that most people who owe a similar amount of tax had paid. Both messages bolstered tax collection, and combining them had an even stronger effect. Over the last financial year, the letters brought forward £210 million of revenue, Britain’s revenue and customs department says — money that otherwise would have had to be chased in costly court procedures and failed to earn interest for the government.

A few years later, the U.S. Treasury Department, in conjunction with the U.S. Social and Behavioral Sciences Team (SBST) — the Obama Administration’s version of the BIT — decided to test a similar approach. They updated the letters sent to people who owed non-tax debt to the Federal government, simplifying and personalizing the letters and adding similar behavioral nudges as in the U.K.. The results showed no effect on payment rates, underscoring how context matters.

To learn more, we are joined by Tammy Chang, a Senior Economist at U.S. Department of the Treasury within the Bureau of the Fiscal Service, and Nathaniel Higgins, a co-founder and Fellow on the former SBST.

The post The importance of replication and validation in evidence-based policy: An interview with Tammy Chang, U.S. Treasury Department, and Nathaniel Higgins, formerly U.S. Social and Behavioral Sciences Team – Episode #146 appeared first on Gov Innovator Podcast.

Massachusetts is known as a leader in providing education policymakers with research findings that they can use to improve policy and practices. The state’s Office of Planning and Research, within the Department of Elementary and Secondary Education (ESE), takes a multifaceted approach:

• Design a proactive research agenda, also known as a learning agenda
• Increase internal capacity for research
• Create partnerships with external researchers

To learn more, we are joined by Carrie Conaway (@clconaway), the Chief Strategy and Research Officer of the ESE. She leads the 15 person Office of Planning and Research.

The post How Massachusetts provides education policymakers with research insights: An interview with Carrie Conaway, Chief Strategy and Research Officer, Massachusetts Department of Elementary and Secondary Education – Episode #145 appeared first on Gov Innovator Podcast.

Our focus today is Seattle, Washington, whose Human Services Department worked with the Government Performance Lab (GPL) at the Harvard Kennedy School to improve outcomes for its homeless population. The changes included:

• Consolidating contracts and allowing for more flexibility
• Establishing goals for homeless service providers and tracking progress
• Using active contract management

To learn more, we are joined by is Jason Johnson, the Deputy Director of Human Services Department in Seattle.

Additional information: Check out the brief by the GPL about Seattle’s contracting reforms. [click here].

The post How Seattle used results-driven contracting to improve homeless services: An interview with Jason Johnson, Deputy Director, Human Services Department, City of Seattle – Episode #144 appeared first on Gov Innovator Podcast.

Recently, a group of leading pre-K researchers set out to find consensus about what we know about pre-K education. In April 2017 the Pre-Kindergarten Task Force presented their findings. To learn more, we are are joined by a member of the task force, Greg Duncan. He is a Distinguished Professor in the School of Education at the University of California, Irvine.

In the interview, Duncan summarizes the research evidence around four key decisions (policy levers) facing state policymakers related to pre-K: (1) Whether to fund more or fewer pre-K slots; (2) Whether and how to regulate classroom quality; (3) Whether and how to prescribe curriculum; and (4) How to support the gains of pre-K after children leave pre-K, i.e., in the elementary school years.

The post How states can optimize their pre-K programs: An interview with Greg Duncan, Professor, UC Irvine, and Member, Pre-Kindergarten Task Force – Episode #143 appeared first on Gov Innovator Podcast.

The new federal education law, the Every Student Succeeds Act (ESSA), emphasizes the importance of evidence, including defining four levels of evidence-based practices. The law, however, leaves it to states to decide how much they want to build an evidence base and how much to nudge districts toward choosing more effective strategies. So what should state education leaders do who want to leverage the new law and encourage districts to learn and do what works for students?

Tom Kane joins us for a two-part series to provide suggestions. In this podcast episode, he discusses how states could use the authority and resources provided by ESSA to launch a system of “efficacy networks,” meaning collections of local agencies committed to measuring the impact of the interventions they’re using. As he notes, “An overlapping system of efficacy networks working with local [education] agencies would create a mechanism for continuous testing and improvement in U.S. education. More than any single policy initiative or program, such a system would be a worthwhile legacy for any state leader.”

He also describes how the Proving Ground initiative run by the Center for Education Policy Research (CEPR) at Harvard is demonstrating the value of having an efficacy network. CEPR is working with 13 school agencies to develop a model to easily conduct low-cost, local pilots.

Tom Kane is a professor of education and economics at the Harvard Graduate School of Education and faculty director of CEPR. His recent article in the journal Education Next is called, “Making Evidence Locally: Rethinking education research under the Every Student Succeeds Act.”

The post How states can use “efficacy networks” to test strategies for school improvement: An interview with Tom Kane, Professor, Harvard Graduate School of Education – Episode #142 appeared first on Gov Innovator Podcast.

As a result, increasing the effectiveness of procurements offers an enormous opportunity today for state and local governments to improve their overall performance. As part of Bloomberg Philanthropies’ What Works Cities initiative, the Government Performance Lab has been working with a set of city governments to adopt results-driven contracting strategies and to transform the culture of procurement.

To learn more, we are joined by the Lab’s director, Jeffrey Liebman. He is a professor at the Kennedy School and is also the Director of the Taubman Center for State and Local Government. His recent article published on the website Route Fifty is, “Business as Usual Can Be the Riskiest Procurement Approach.”

The post Transforming the culture of procurement in state and local government: An interview with Jeffrey Liebman, Director, Harvard Kennedy School Government Performance Lab – Episode #141 appeared first on Gov Innovator Podcast.

What lessons does having a second Deputy Secretary provide for other public agencies, whether federal, state or local? To get insights, we speak with Heather Higginbottom (@hhigginbottom). She served as the Deputy Secretary of State for Management and Resources from 2013 to the end of the Obama administration. Before that role she served as Deputy Director of the White House Office of Management and Budget. She currently serves as the Chief Operating Officer of CARE, the global humanitarian organization. We recorded this conversation in 2016 while she was still in her Deputy Secretary role.

Update: The Trump administration appears to be combining the two Deputy Secretary roles at State into one, reversing the structure during the Obama administration.

The post Insights from the only Federal department with two Deputy Secretaries: An interview with Heather Higginbottom, former Deputy Secretary for Management and Resources, U.S. Department of State – Episode #140 appeared first on Gov Innovator Podcast.

We get insights from Jeffrey Neal (@JeffNealHR), a leading expert in human resources issues. He served for 33 years in the Federal government, including as the Chief Human Capital Officer for the 23,000-employee Defense Logistics Agency (DLA) from 2000 to 2009. At DLA, he helped the agency identify more than $50 million in administrative and operational savings. He later served as the Chief Human Capital Officer for the Department of Homeland Security, which has more than 240,000 civilian and military employees. Today he is a Senior Vice President at the consulting firm ICF and he also runs the ChiefHRO blog.

The post Transforming support services in Federal agencies: An interview with Jeffrey Neal, Former Chief Human Capital Officer, Defense Logistics Agency and Department of Homeland Security – Episode #139 appeared first on Gov Innovator Podcast.

In particular, the Nutter Administration established strategic goals with measurable targets; launched PhillyStat, Philadelphia’s performance management system; established Philadelphia’s open data policy in 2012 and launched an open data portal in 2015; and launched Philly 311, the city’s online customer service system.

Today Michael Nutter is a CNN political commentator, a professor at Columbia University, a fellow at the University of Chicago’s Institute of Politics and a senior fellow with the What Works Cities initiative, among other roles.

The post Creating a results-focused city government: An interview with Michael Nutter, former Mayor of Philadelphia – Episode #138 appeared first on Gov Innovator Podcast.

Our focus today is new software, called RCT-YES, designed to help public leaders (and the researchers who work with them) answer that question. It was funded by the Institute of Education Sciences, the statistics, research, and evaluation arm of the U.S. Department of Education, and developed in partnership with Mathematica Policy Research. The software, available free to download online, is based on new statistical methods for analyzing data from randomized controlled trials (RCTs).

To learn more, we are joined by Peter Schochet. He is a nationally known methodological expert in program evaluation and a Senior Fellow at Mathematica. He led the team that developed RCT-YES.

Web extra: For those with deeper expertise in evaluation, Peter Schochet gives an overview of how the RCT-YES software is designed to conduct a wide range of analyses using RCT or QED data and how the software uses new statistical methods for analyzing those data. [click here]

The post Making rigorous program evaluation easier with RCT-YES software: An interview with Peter Schochet, Fellow, Mathematica Policy Research – Episode #137 appeared first on Gov Innovator Podcast.

To get an overview and hear implementation lessons for human services agencies that might want to use these types of interventions — or “nudges,” as they are often called — we are joined by two researchers from the social policy research firm MDRC, which was a partner on the BIAS project. Lashawn Richburg-Hayes is a Director and Nadine Deshausay is a Research Associate at MDRC.

More information: For more information on the 15 projects, including their goals, strategies, results and costs, see MDRC’s PowerPoint presentation presented at the BIAS Capstone Convening in April 2016 [click here].

The post Lessons in applying behavioral insights to human services from the Behavioral Interventions to Advance Self-Sufficiency (BIAS) project: An interview with Lashawn Richburg-Hayes and Nadine Deshausay, MDRC – Episode #136 appeared first on Gov Innovator Podcast.

A growing number of states and localities, however, are developing Integrated Data Systems by linking their program data, also called administrative data, across multiple agencies to monitor and track how services are being used and to what effect. These systems can also be used to test social policy innovations through quick, low-cost randomized control trials and quasi-experiments, as well as to do continuous quality improvement efforts and benefit cost analysis.

To learn more, we’re joined by Dennis Culhane. He’s is a professor in the School of Social Policy and Practice at the University of Pennsylvania and the co-principal investigator of the nonprofit Actionable Intelligence for Social Policy (AISP) which has built a network of jurisdictions using Integrated Data Systems.

Credit: The graphic above is from AISP’s overview of Integrated Data Systems.

The post How states and localities are improving the quality of education, health, and human services through integrated data systems: An interview with Dennis Culhane, Professor, University of Pennsylvania – Episode #135 appeared first on Gov Innovator Podcast.

In short, access to administrative data is critical to making Federal programs and policies more effective and efficient. Today, however, access to data can be so restricted that conducting research and analysis can be very difficult.

Our guest today has a vision for how that could be different and why greater access to data is important. Maria Cancian is a Professor of Public Affairs and Social Work at the University of Wisconsin–Madison and the former Director of the Institute for Research on Poverty. From 2015 to 2016, she served as the Deputy Assistant Secretary for Policy at the Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services.

Additional resource: The White House Office of Management and Budget recently created a set of background briefs on using administrative data, prepared for the Commission on Evidence Based Policymaking [click here].

The post Why broadening access to Federal administrative data is critical for improving government services and increasing taxpayer value: An interview with Maria Cancian, Professor, University of Wisconsin-Madison – Episode #134 appeared first on Gov Innovator Podcast.

To get insights into all those topics, we are joined by Adam Sacarny (@asacarny). An economist by training, he is a professor of health policy and management at Mailman School of Public Health at Columbia University. He has been working with both the U.S. Department of Health and Human Services (HHS) as well as the State of Colorado on communications-related experiments. Both HHS and Colorado are taking a “test, learn and adapt” approach by testing out certain interventions using relatively quick, low cost randomized controlled trials, learning from the results, and refining the strategies to be tested again.

Web extra: Adam Sacarny discusses the communications-related experiment he conducted with the State of Colorado around helping Colorado citizens choose a health plan on the ACA marketplace that best fits their needs. [click here]

The post Test, learn and adapt – How public agencies can use researcher-practitioner partnerships to test low-cost, light-touch interventions: An interview with Adam Sacarny, Professor, Columbia University – Episode #133 appeared first on Gov Innovator Podcast.

• A requirement from the Governor’s budget office that agencies seeking new funding provide evidence of program efficiency and effectiveness and, for new programs, describe their program evaluation strategy.
• The Herbert administration’s use of a performance management framework for agencies.
• A requirement by the Legislature that proposals for new or significantly expanded programs require a performance note that describes how the program will measure its success (with followup by legislative auditors to track results).
• A statewide registry of evidence-based prevention interventions that guides the Utah Division of Substance Abuse and Mental Health in contracting decisions.
• The use of a comprehensive cost-benefit model in juvenile justice to help lawmakers identify evidence-based policies that provide the best return on taxpayers’ investment.

To learn more, we are joined by Kristen Cox, the Director of the Office of Planning and Budget for Governor Gary Herbert, and Jonathan Ball, the Director of the Utah Fiscal Analysts Office for the Legislature.

The post How Utah became a leader in evidence-based policymaking: An interview with Kristen Cox, Director, Governor’s Office of Planning and Budget, and Jonathan Ball, Director, Utah Fiscal Analysts Office – Episode #132 appeared first on Gov Innovator Podcast.

To learn more, we’re joined by someone who has been closely involved in these efforts, Toby Barker (@toby_barker). First elected to the legislature in 2007 at the age of 25, today his is the Chairman of Performance-Based Budgeting Committee, which launched in 2016. He also sits on several other committees, including Appropriations. He is a Republican state representative whose district covers Central Hattiesburg.

More information: Mississippi is part of the Pew-MacArthur Results First Initiative, which Toby Barker references in the interview. For more information, see the Gov Innovator podcast interview with Gary VanLandingham of Results First. [click here]. Also see the initiative’s Guide to Evidence-Based Budget Development.

The post How the State of Mississippi uses evidence-based budgeting to increase return on investment and improve program outcomes: An interview with Toby Barker, Mississippi State Representative – Episode #131 appeared first on Gov Innovator Podcast.

Remarkably, today about one in four schools in the UK (7,600 schools, involving more than 750,000 students) is taking part in some type of EEF-funded randomized controlled trial to learn what works in education policy and practice — or to learn how best to convey evidence-based approaches to teachers and encourage their use. To date, EEF has funded 130 projects; awarded £75 million in funds; partnered with 26 independent evaluation teams; published 60 reports; and launched the Teaching and Learning Toolkit and Early Years Toolkit.

To learn more, we are joined by EEF’s founding Chief Executive, Sir Kevan Collins. He has worked in the public sector for over 30 years, including serving as Chief Executive of the London Borough of Tower Hamlets and, before that, as Director of Children’s Services for that borough.

The post How the UK’s Education Endowment Foundation (EEF) is building rigorous evidence about how to close education achievement gaps: An interview with Sir Kevan Collins, Chief Executive, EEF – Episode #130 appeared first on Gov Innovator Podcast.

We get insights from Robert Gordon who held top leadership roles at the White House Office of Management and Budget and the U.S. Department of Education — and was one of the architects of the Obama Administration’s evidence agenda. He’s also the co-author with Ron Haskins of a bipartisan agenda for strengthening the use of data and evidence, published in the book Moneyball for Government. He is currently a Senior Vice President at the College Board.

In the interview, he discusses three of the grant-related strategies presented in the “Moneyball” chapter. They are for Federal agencies to:

• conduct grant-program “look backs” to replace mandates for processes with incentives for outcomes;
• transform existing formula and competitive grants to use more evidence;
• create new flexibility to test new approaches to fighting poverty.

Web extra: Robert Gordon discusses how evidence-based policy can be an area of agreement between leaders from different political parties around the goal of spending smart. [click here]

The post Transforming Federal grant programs from compliance driven to results focused: An interview with Robert Gordon, former Acting Deputy Director, White House Office of Management and Budget – Episode #129 appeared first on Gov Innovator Podcast.

To get insights into what it takes to create a successful researcher-practitioner partnership, we’re joined by Dayanand Manoli. He is an economist at University of Texas at Austin whose research interests include policy related to social security and retirement policy, income tax policy and education policy. He has collaborated with the IRS on several research projects.

The post Creating successful researcher-practitioner partnerships at the Federal level: An interview with Dayanand Manoli, Professor, University of Texas at Austin – Episode #128 appeared first on Gov Innovator Podcast.

• How can we fix the misalignment between the skills of Americans without college degrees and the workforce needs of well-paying industries that do not necessarily require a college degree?
• And how can we create a learning strategy where we incentivize states to produce better outcomes for community college graduates and then learn what works among state approaches?

We get insights into those questions from Harry Holzer, a leading thinker on workforce issues. He is the former Chief Economist at the U.S. Department of Labor and is today a professor at the McCourt School of Public Policy at Georgetown University.

In the interview, we discuss Harry Holzer’s policy recommendations, including 1) Providing more resources to community colleges but also creating incentives and accountability by basing state subsidies on student completion rates and earnings of graduates; 2) Expanding high-quality career and technical education plus work-based learning models like apprenticeship; and 3) Assisting and incentivizing employers to create more good jobs.

The post Raising job quality and skills for American workers through more effective education and workforce development within states: An interview with Harry Holzer, Professor, Georgetown University – Episode #127 appeared first on Gov Innovator Podcast.

To learn more about the trends in school-based health centers and the evidence of their impact, we’re joined by a leading expert on the topic, Olga Acosta Price. She is an Associate Professor at the Milken Institute School of Public Health at The George Washington University and director of its Center for Health and Health Care in Schools. Her recent paper published by the Brookings Institution focuses on school-centered approaches to improving community health.

Web extra: Olga Price discusses how the recent clarification of the free care rule issued by the Federal government helps facilitate broader use of school-based health centers. [click here]

The post Using school-based health centers to address the health needs of low-income youth: An interview with Olga Acosta Price, Professor, The George Washington University – Episode #126 appeared first on Gov Innovator Podcast.

To learn more, including advice for other Federal agencies, we are joined by Diana Epstein who is a manager in the Office of Research and Evaluation at CNCS and Carla Ganiel who is a Senior Program Specialist with the AmeriCorps program.

The post How one Federal agency, the Corporation for National and Community Service, strengthened the role of evidence in a key grant program, AmeriCorps: An interview with Diana Epstein and Carla Ganiel, CNCS – Episode #125 appeared first on Gov Innovator Podcast.

In particular, he recommends twelve practices or leadership skills that can help organizations strengthen their performance. Our discussion draws on his original report on the topic, published by the IBM Center for the Business of Government, which discusses eleven of those practices.

To give us an overview, we’re joined by Bob Behn, speaking with us (probably with a baseball tie on) from Boston. A professor at the Kennedy School, he’s the faculty chair of the executive program called Driving Government Performance. He also publishes monthly insights though his Public Leadership Report, available free online.

Web extra: Bob Behn describes the connection between the these twelve practices and the PerformanceStat approach to public leadership, which was the focus of his most recent book. [click here]

The post Twelve “better practices” that can help public leaders tackle key organizational challenges and boost results: An interview with Bob Behn, Professor, Harvard Kennedy School – Episode #124 appeared first on Gov Innovator Podcast.

The Wake County Public School System (WCPSS) — North Carolina’s largest school district — faced exactly those questions when it implemented a district-wide reform initiative. The initiative is called Multi-Tiered System of Supports, or MTSS, and is designed to increase academic achievement and reduce behavioral problems, although the specifics of MTSS are not the focus of our interview.

WCPSS was able to implement a rigorous evaluation of the initiative using a phased-in design, with 88 schools being randomly assigned to one of two groups: One group of 44 schools implemented MTSS first, while the other group of 44 schools will implement it two years later. That allowed district leaders to compare the outcomes for children in each set of schools to determine the impact of the MTSS initiative.

To learn more, we’re joined by Matthew Lenard. He has served as Director of Data Strategy and Analytics for WCPSS since 2012 and is the co-lead researcher on the MTSS evaluation.

The post How school districts can use rigorous program evaluation to test new education reforms: An interview with Matthew Lenard, Director, Data Strategy and Analytics, Wake County Public Schools – Episode #123 appeared first on Gov Innovator Podcast.

It’s a question about which organizational leaders may want hard evidence, either to take stock and help improve program results, or to satisfy their authorizers or funders who may be asking for rigorous evidence of impact. Either way, how can you determine the impact of your program? And which strategies may sound useful but are unlikely to produce accurate answers?

To examine these these questions and get a “101” on impact evaluation, we’re joined by David Evans (@tukopamoja). He is a Senior Economist at the World Bank and the co-author, with Bruce Wydick, of a recent post on the Bank’s Development Impact blog on this topic.

The interview covers:

• The concept of impact
• Ways that organization could try to estimate impact that generally won’t be accurate
• Three strategies to more accurately estimate program impact:
  • Using a lottery, aka a randomized experiment
  • Using an eligibility cutoff, aka regression discontinuity design
  • Using before and after data for both participants and nonparticipants, aka a differences-in-differences approach
• Factors to guide the choice of one impact evaluation strategy over another

The post Determining if your program is having a positive impact (i.e., impact evaluation 101): An interview with David Evans, Senior Economist, The World Bank – Episode #122 appeared first on Gov Innovator Podcast.

A recent study (and related Hamilton Project policy proposal), however, suggest that this conventional wisdom is wrong. It uses a rigorous evaluation design — a randomized controlled trial — to examine the effects of intensive, individualized (two students to one tutor) math tutoring among 9th and 10th grade boys in twelve Chicago public schools.

To learn more, we are joined by one of the study’s nine authors, Jonathan Guryan. He is a professor of human development and social policy at Northwestern University and a fellow at Northwestern’s Institute for Policy Research.

The post Using intensive, individualized math tutoring to boost academic outcomes of disadvantaged youth: An interview with Jonathan Guryan, Professor, Northwestern University – Episode #121 appeared first on Gov Innovator Podcast.

In our interview, Gordon Berlin reflects on the Rikers Island SIB as well as broader lessons from SIB projects to date, including:

• What types of SIB projects are likely to work best
• The role of evaluation in SIB projects
• Why philanthropy could and should play a central role in SIBs

The post Lessons from the nation’s first Social Impact Bond, aimed at reducing recidivism among adolescent offenders at Rikers Island: An interview with Gordon Berlin, President, MDRC – Episode #120 appeared first on Gov Innovator Podcast.

To learn more, including lessons for other public agencies, we’re joined by Russ Whitehurst. He was the first director of IES and served in that role from 2002 to 2008. Today he is a Senior Fellow at the Brookings Institution, including serving as editor of the Evidence Speaks series.

Web extra: Russ Whitehurst describes the origins of IES, including some of the key people involved in its creation and launch. [click here]

The post How the Institute of Education Sciences at the U.S. Dept. of Education is helping the education field to learn and do what works: An interview with Russ Whitehurst, Senior Fellow, The Brookings Institution – Episode #119 appeared first on Gov Innovator Podcast.

Milwaukee’s Teen Pregnancy Prevention Initiative, launched in 2008, is a citywide effort led by the United Way of Greater Milwaukee and Waukesha County. It has been recognized as a model of community collaboration, including by the White House Council for Community Solutions.

The three main prongs of the strategy are:

• An aggressive advertising campaign targeted to teens
• The use of evidence-based sex education
• The involvement of community partners to support the strategy

The initiative set a goal in 2008 to reduce Milwaukee’s teen births by 46% over 10 years. It exceeded that goal, three years early, with a 50% decline in teen births by 2012. During the same period, national teen birthrates also declined sharply, although Milwaukee’s decline slightly outpaced the national average. More broadly, the initiative is a leading example of an “all-hands-on-deck” community partnership focused on a key community challenge.

To learn more, we’re joined by two people who have been at the center of the initiative. Bevan Baker has been the Commissioner of Health in Milwaukee since 2004. And Nicole Angresano is the Vice President of Community Impact at the United Way of Greater Milwaukee and Waukesha County.

The post Milwaukee’s three-pronged strategy to reduce teen pregnancy: An interview with Bevan Baker, Commissioner of Health, City of Milwaukee, and Nicole Angresano, United Way of Greater Milwaukee – Episode #118 appeared first on Gov Innovator Podcast.

A rigorous evaluation (using a randomized controlled trial) of a student-aid initiative, called Aid Like a Paycheck, recently took three additional steps, beyond typical program evaluation, to ensure that the study produces information that is relevant to end users. The strategies will be of interest to other program evaluators, but also to foundations and other funders who want to support rigorous and relevant program evaluations. The strategies are:

• Implementing a pilot phase — in fact, one that ran longer than most (about 2 1/2 years);
• Forming an advisory group of stakeholders to provide input into the design of both the intervention and the research study; and
• Doing outreach to other stakeholders about both the preliminary intervention design and research design to get additional input.

To learn more, we’re joined by the evaluation’s lead researcher, Evan Weissman. He is a Senior Associate at the nonprofit research firm MDRC and has over 15 years of experience at MDRC directing projects, providing technical assistance, conducting qualitative research, and disseminating findings in a wide range of education and social policy settings.

The post Three strategies to promote relevance in program evaluations so that findings are useful to policymakers and practitioners: An interview with Evan Weissman, Senior Associate, MDRC – Episode #117 appeared first on Gov Innovator Podcast.

To learn more, we’re joined by the study’s author, Peter Bergman (@peterbergman_). He is an professor of economics and education at Columbia University’s Teachers College. His research uses randomized controlled trials to find low-cost, scalable interventions that improve education outcomes.

The post Improving student outcomes by giving parents detailed information about their child’s academic progress: An interview with Peter Bergman, Professor, Teachers College, Columbia University – Episode #116 appeared first on Gov Innovator Podcast.

To learn more, we are joined by Betsy Fretwell (@BetsyFretwell), the City Manager of Las Vegas. She has been in that role since 2009, overseeing a city workforce of nearly 3,000 and a budget of $1.2 billion per year. She has won several awards for her work, including a National Public Service Award.

The post Las Vegas’s data-driven effort to improve traffic safety at its most dangerous intersections: An interview with Betsy Fretwell, City Manager, City of Las Vegas – Episode #115 appeared first on Gov Innovator Podcast.

To learn more, we are joined by Oliver Wise (@ojwise). He is the founding director of the Office of Performance and Accountability.

The post Insights from the City of New Orleans’ analytics unit, NOLAlytics, about using data to improve city services: An interview with Oliver Wise, Director, Office of Performance and Accountability, City of New Orleans – Episode #114 appeared first on Gov Innovator Podcast.

In the U.S., more than a third of elderly patients discharged from hospitals are re-admitted within 90 days, often needlessly. An intervention that is helping change that is the nursing-led Transitional Care Model (TCM), pioneered at the University of Pennsylvania. It been the focus of four large scale NIH-funded clinical trials, including three RCTs, all finding consistent positive health and economic effects, including reduced re-hospitalization and health care expenditures. The savings, in fact, are equivalent to about $10 billion if the approach were implemented nationwide. Today TCM is being used in a range of health systems in the U.S., although data suggest that the model’s uptake is sporadic and slow relative to its promise.

To learn more, we’re joined by the principal investigator of this research, Mary Naylor. She is a Professor in Gerontology and Director of the New Courtland Center for Transitions and Health at the University of Pennsylvania School of Nursing. Our conversation covers the TCM model, the research findings, as well as some of the challenges of getting evidence-based innovations like the TCM into wider usage by practitioners.

The post Improving health outcomes of older adults while reducing costs through the nursing-led Transitional Care Model: An interview with Mary Naylor, Professor, University of Pennsylvania School of Nursing – Episode #113 appeared first on Gov Innovator Podcast.

As evaluation experts (including RCT proponents) will note, RCTs are one tool within public managers’ analytical tool boxes, along with performance measures, process evaluation, cost-benefit analysis or cost analysis, well-designed quasi-experiments and other approaches. The goal is to use the most rigorous method possible for the question at hand.

To learn more about the value of RCTs, as well as to address some of the concerns or criticisms of the approach, we are joined by Dean Karlan (@deankarlan), a leading expert in using randomized evaluations in social policy. He is a professor of economics at Yale University and the president and founder of Innovations for Poverty Action (IPA), a non-profit that has conducted over 500 evaluations in more than 50 countries to build evidence about effective solutions to global poverty problems. His most recent book, co-authored Jacob Appel, is titled, “More Than Good Intentions.”

The post Using randomized evaluations to address global poverty and other social policy challenges: An interview with Dean Karlan, Professor, Yale University, and President, Innovations for Poverty Action – Episode #112 appeared first on Gov Innovator Podcast.

To learn more, we are joined by one of the founders of the program, Dr. Freeman Hrabowski, who is today the President of UMBC. He has received numerous accolades for his work, including more than 20 honorary degrees and a profile on 60 Minutes. President Obama named him chair of his Advisory Commission on Educational Excellence for African Americans. His new book is titled, “Holding Fast to Dreams” and includes a chapter on Meyerhoff Scholars Program.

Web extras: President Hrabowski discusses the role of ongoing monitoring and evaluation of the program [click here] as well as some highlights in terms of results [click here]. (The evidence of effectiveness of the program is also reviewed in this 2011 journal article.) He also describes the importance of first-year student performance [click here]. And finally, he discusses the ongoing replication of the program at two other universities [click here].

The post Increasing diversity in the sciences through the Meyerhoff Scholars Program: An interview with Freeman Hrabowski, President, University of Maryland, Baltimore County – Episode #111 appeared first on Gov Innovator Podcast.

To learn more, we’re joined by Erin Dalton. She is the Deputy Director for the Office of Data Analysis, Research and Evaluation within DHS.

Web extras: Erin Dalton discusses what beneficial capabilities are not possible if a county does not have integrated data [click here]. She also explains what new features of the Data Warehouse are being developed. [click here].

The post How Allegheny County’s Data Warehouse is improving human services through integrated data: An interview with Erin Dalton, Allegheny County Department of Human Services – Episode #110 appeared first on Gov Innovator Podcast.

Michael Nutter served as Mayor of Philadelphia from 2008 to 2016. During his eight years in office, the city became a leader in the use of data, evidence and evaluation to improve outcomes for city residents. In 2014, Governing Magazine named the Mayor one of the Public Officials of the Year, noting, “Philadelphia isn’t an easy place to govern. But Mayor Michael Nutter has undoubtedly made an outsized impact on the city, creating a Philadelphia that’s cleaner, safer, smarter and more fiscally sound than the city he began leading in 2008.”

To gain insights into some of the steps the city took to be more results-focused and effective, we’re joined by Maia Jachimowicz. She served as Deputy Director for Policy and, starting in 2013, as Policy Director for the Mayor until 2016. She recently became ‎Vice President for Evidence-Based Policy at the nonprofit Results for America.

Web extra: Maia Jachimowicz provides an additional example of a city agency that became more results focused during the Nutter Administration. [click here]

The post How Philadelphia became a leader in the use of data and evidence: An interview with Maia Jachimowicz, V.P. for Evidence-Based Policy, Results for America, and former policy director to Mayor Michael Nutter – Episode #109 appeared first on Gov Innovator Podcast.

A new study published in the British Medical Journal examines the effects on purchases of beverages in Mexico during the first year after implementation of the tax. To learn more, we’re joined by one of the articles co-authors, Barry Popkin. He is a Distinguished Professor of Nutrition at the University of North Carolina at Chapel Hill School of Public Health.

This is the second interview in our two-part series about Mexico’s soda tax. In part one, I spoke with Tina Rosenberg about how the tax came about.

The post The first-year effects of Mexico’s soda tax: An interview with Barry Popkin, Professor, University of North Carolina at Chapel Hill School of Public Health – Episode #108 appeared first on Gov Innovator Podcast.

October 2017 update: Problems have been reported in the research conducted by researchers at the BEN Center. For an overview, see “The Cookie Crumbles: A Retracted Study Points to a Larger Truth” in the New York Times.

The post Using behavioral insights to design smarter school lunchrooms: An interview with David Just, Co-Director, Cornell Center for Behavioral Economics in Child Nutrition Programs – Episode #107 appeared first on Gov Innovator Podcast.

To gain insights, we’re joined by Tina Rosenberg (@tirosenberg), a Pulitzer Prize winning journalist. Her recent article in The Guardian is titled, “How one of the most obese countries on earth took on the soda giants.” She is the author of the “Fixes” column in the New York Times and also a co-founder of the Solutions Journalism Network.

As Tina Rosenberg explains, there were several factors that helped pass the soda tax. One was a smart media campaign by the Nutritional Health Alliance in Mexico to raise awareness about the impact of soda. As example ad, at right, is titled “12 spoonfuls.” It asks: “Would you give them 12 teaspoons of sugar? Then why give them soda?”

This is the first interview in our two-part series about Mexico’s soda tax. In part two, I speak with Professor Barry Popkin of UNC on the estimated first year impacts of the tax.

The post How Mexico took on the soda industry and won, passing a soda tax: An interview with Tina Rosenberg, New York Times and Solutions Journalism Network – Episode #106 appeared first on Gov Innovator Podcast.

To learn more, we’re joined by Patrick McCarthy, who has led the Casey Foundation since 2010 and has been at the foundation since 1994. Prior to that he held a range of positions focused on youth and families, including division director within Delaware’s human services agency.

The post Calling on states to close their youth prisons: An interview with Patrick McCarthy, President, Annie E. Casey Foundation – Episode #105 appeared first on Gov Innovator Podcast.

The City University of New York (CUNY) launched the Accelerated Study in Associate Programs (ASAP) in 2007 with the goal of doubling the graduation rates of community college students as well as encouraging timely graduation within three years. A rigorous, random assignment evaluation by MDRC found that ASAP nearly doubled the percentage of students needing developmental courses that completed an associate’s degree (40% versus 22% for the control group), by far the largest effects MDRC has found for a community college intervention. And CUNY’s own evaluation of the overall program (not just for those needing remediation) found that the program more than doubled graduation rates.

To learn more, we’re joined by Donna Linderman. She is the Dean for Student Success Initiatives at CUNY and the Executive Director of ASAP.

The post Doubling community college graduation rates through CUNY’s ASAP program: An interview with Donna Linderman, Dean for Student Success Initiatives, City University of New York – Episode #104 appeared first on Gov Innovator Podcast.

An important state initiative to do that is Colorado’s Family Planning Initiative. Launched in 2009, it has provided 36,000 long-acting reversible contraceptives (LARCs) to low-income women through family planning health centers, while also increasing health care provider education and training. Since it’s launch, Colorado’s teen birth rate has been cut nearly in half: Both the birth rate and abortion rate for women ages 15-19 fell 48 percent from 2009 through 2014. Moreover, for every dollar invested in the LARC program, an estimated $5.85 was avoided within a three-year period by the Colorado Medicaid program.

To learn more, we’re joined by Dr. Larry Wolk, the Executive Director and Chief Medical Officer of the Colorado Department of Public Health and Environment.

The post Colorado’s Family Planning Initiative: An interview with Larry Wolk, Executive Director, Colorado Department of Public Health and Environment – Episode #103 appeared first on Gov Innovator Podcast.