by Jane Haines, Senior Associate, EBP 

Financial and other incentives have been a common tool used to spur investment and economic development for decades. However, there is constant competition between states, provinces, and metropolitan regions to provide more and more lucrative programs to attract best-in-class business.  Therefore, it is essential to recognize that these incentives are not standalone solutions but rather tools within a broader economic strategy. 

Jurisdictions typically create incentives to address a shortcoming in their business environments, or catalyze something—investment, training, innovation, employment— that wouldn't otherwise happen. Only through regularly evaluating incentive programs using empirical evidence, is it possible to determine whether they accomplish these goals. 

To ensure incentive programs are efficient and effective, it’s important to ask the right questions when evaluating their impact. Here are three key questions to consider: 

1.) What is the program trying to accomplish, and how does that contribute to state or municipal economic development goals? 

Incentive programs are tools for achieving the economic future a state or municipality envisions for itself. If a state or municipality wishes to attract investment from businesses in target industry clusters outlined in its strategy, it can design incentive programs that improve the business operating environment for that industry and its suppliers. 

For example, a state may offer a tax credit to offset higher costs of doing business compared to a nearby competitor. Capital investment, labor, and tax costs vary by industry and location. Thus, incentive programs are typically tailored to specific target industries and should be grounded in a core understanding of a location’s value proposition. 

Most incentives sunset after 5-10 years and are not meant to be permanent solutions to long-term barriers to doing business in a community. However, they can help offset initial investment costs and signal to investors that a jurisdiction is serious about stewarding their investment. Additionally, incentives can help address short-term economic challenges such as a tight labor market or high interest rates. 

For a state or municipality, offering incentives to businesses in the industries that align with community goals and priorities can support inclusive and sustainable economic development.

Economic development plans and strategies are documents that typically lay out this vision and can guide design and implementation of incentive programs. Periodically evaluating how well incentive programs align with a jurisdiction’s strategy can help ensure leaner and more inclusive use of taxpayer resources. 

2.) Does the incentive program create more value than it costs to implement? 

A popular critique of economic development incentive programs is that they cost taxpayers more in government spending than a state or municipality gains by offering them. 

One way to answer this question is by conducting Return-on-Investment (ROI) analysis. “Investment” is the incentive offered in the form of a grant, loan, tax credit, or tax increment financing to a business. “Return” is the additional tax revenue that the business then pays to the state or municipality that may not have otherwise occurred. 

A critical question in conducting ROI analysis is estimating the number of businesses that would have located or expanded in a location “but for” the incentive award. Incentive program attribution levels are difficult to ascertain because incentives are just one of many factors that impact business decisions. 

For example, if a business would have invested in a location regardless of incentive availability, it means the program was not necessary to realize the benefit to the state or municipality. To combat this, many agencies require that companies certify that they would not have made the investment in question but for the incentive award, making companies reluctant to report otherwise. 

An incentive program can be deemed effective if it “tips the scale” on an investment decision. However, understanding for what portion of businesses incentives work in this manner is a difficult task. One researcher via meta-analysis estimates that incentives tip the scale on investment decisions for 2-25% of firms receiving awards. 

3.) Is sufficient data available to evaluate a program’s return-on-investment? 

ROI analysis can be difficult for state agencies to routinely conduct because of lack of available data or low-quality data on businesses participating in incentive programs. This can stem from data confidentiality concerns between a state tax agency and an economic development agency that prevent data sharing, but also from lack of reporting on how businesses grow and contribute to the economy over time. 

One way to address both challenges is through quasi-experimental evaluation design. A quasi-experimental design is a research method used to study the impact of an intervention by comparing groups without randomly assigning participants to each group. This method is useful when random assignment is not feasible—such as for businesses receiving and not receiving incentive awards—to examine the effects of a policy or program. 

The limitation of this method is that it is not possible to establish a cause-and-effect relationship because there is no random assignment to treatment groups. 

A critical challenge states face in evaluating their incentive programs is collecting high quality data on business outcomes from recipients without overburdening them with reporting requirements. If reporting data to state agencies takes an undue amount of time and personnel resources, businesses are more likely to submit inaccurate data, decline to submit data, or opt out of participating in the program altogether. Burdensome reporting requirements can also limit small business participation and hinder progress toward inclusive economic growth. 

One way to address this limitation is by requesting access to Quarterly Census of Employment and Wages (QCEW) from the state agency that reports this data to the U.S. Bureau of Labor Statistics. 

In a recent program evaluation, EBP isolated increased payroll taxes as the main benefit that incentive programs create for the state where they’re implemented. To understand how payroll taxes paid over a five-year period differ between participating and non-participating businesses, we requested access to QCEW data. We were provided with average annual employment, average annual payroll, and total payroll figures for both participating and non-participating businesses. To protect data confidentiality among non-participating businesses, data was aggregated by six-digit NAICS code. 

Our analysis showed percent differences over five years in employment and payroll between businesses that participated in incentive programs, and those that did not participate but belonged to the same industries. 

Our findings suggest that if 23% or fewer companies that received incentives would have expanded or located in the state regardless, the incentive program creates a positive return on investment. In the case that zero companies would have invested in the state without the incentive, the program creates a 464% ROI. This drops to 72% ROI if an estimated 10% of companies would have located or expanded regardless. 

Because we isolated participating and non-participating businesses, the difference in employment and payroll between the two groups could be attributed to the incentive program. However, because treatment groups were not randomly assigned, we must consider the sensitivity rate—or percentage of businesses receiving incentives but would have invested in the state regardless—to interpret the gray zone. 

Conclusion 

Evaluating the efficiency and effectiveness of economic development incentive programs is a complex task. One way to improve accuracy and efficiency of these analyses is to rely on data sources such as state-level QCEW reports that already exist. EBP understands how to request and use this data for streamlined incentive program evaluation. 

Additionally, to better understand how well incentives work to improve the business operating environment in a state or municipality, it is necessary to conduct interviews, roundtable discussions, and surveys to establish attribution. Participants in these discussions may include peer economic development agencies, as well as companies that receive incentives in various locations across the country. Conducting these interviews often requires a reliable and objective third-party consultant with existing, trusting relationships with private businesses and economic development agencies.

Finally, to understand how small- and medium-sized businesses engage with incentives, it is critical to foster deep, ongoing engagement at the local level. Local business and community engagement require a distinct and important set of tools to foster inclusive, sustainable, and community-led economic development. 

Combining quantitative and qualitative methods to evaluate incentives in this way produces robust, defensible understanding of spending on these programs and the value they create in communities that fund them.