GAO: Insufficient Data on Success of 1115 Medicaid Demonstrations

February 27, 2018 - State and federal evaluations of 1115 Medicaid demonstrations have insufficient data to determine demonstration success because states tend report fragmented program results, according a new GAO report.

Currently, there are thirty-four 1115 demonstrations nationwide which are funded by $300 billion in federal Medicaid dollars. In 2017, HHS expressed strong support for states to use 1115 demonstrations as a way to address  healthcare issues unique to each state.

GAO found that many state-led evaluations of 1115 demonstrations were flawed because of a lack of key performance data and poor follow-up on critical hypotheses about potential demonstration outcomes.

CMS only received interim performance reports based on the early years of demonstrations rather than a comprehensive review after the end of a demonstration cycle. An average 1115 demonstration cycle is 3 to 5 years, which should be enough time for CMS to fully assess demonstration outcomes, GAO added.

“CMS officials told us that, historically, state-led evaluations have generally provided descriptive information but lacked evidence on outcomes and impacts,” GAO said. “As a result, officials noted that they consider the data reported in the evaluations but, generally, state-led evaluations have not been particularly informative to their policy decisions.”

READ MORE: How Payer Philanthropy Can Address Social Determinants of Health

At least three state demonstrations exhibited significant shortcomings, GAO found during its review.  

In Arizona, the state was required to evaluate if providing managed long-term support services led to quality of life improvements for individuals with developmental disabilities

The state’s report from October 2016 lacked key data points, including hospital readmission rates, quality of life measurements, health plan satisfaction, and provider satisfaction rates. The October evaluation data was the only dataset submitted during the five-year demonstration cycle.

Arkansas used a 1115 demonstration to provide premium assistance for 200,000 Medicaid beneficiaries that were eligible for subsidized premiums under the ACA.

The program aimed to determine if premium assistance allowed members to seek out higher quality healthcare services than in Medicaid fee-for-service. Arkansas Medicaid administrators also designed the demonstration to increase the number of insured individuals in the state. The state submitted data two-and-a-half years into the demonstration cycle, but the report did not provide data on continuity of coverage.

READ MORE: Medicaid Analytics Support Social Determinant Incentive Payments

Massachusetts received $690 million in funding for a 1115 demonstration that offered delivery system reform incentive payments (DSRIP) to seven hospitals.

The terms and conditions of the demonstration stated that Massachusetts had to determine if incentive payments led to lower per-capita spending and improved quality of care. The evaluation report submitted after the five-year demonstration did not collect enough data to conclude if the program created expected spending and quality improvements.

GAO additionally found that CMS lacked consistent data for demonstrations including DSRIP, premium assistance, and beneficiary engagement programs.

CMS only received enough data to evaluate DSRIP programs in  California, Texas, and New Jersey for an interim evaluation.

“In 2 of the 3 states, complete and usable inpatient encounter records for adults were not available,” GAO added.

READ MORE: Medicare Advantage Evaluation Requires Transparent Claims Data

CMS had difficulty collecting data about premium assistance demonstrations because the agency transitioned to a new Medicaid data system in 2015. CMS had to request data directly from state agencies, which is a time-intensive process after the transition. Obtaining even the limited data about Arkansas’s demonstration took over a year, GAO found.

Additionally, CMS has experienced trouble collecting quality information about beneficiary engagement programs.

CMS has not been able to collect data on cost-sharing and administrative costs for Indiana’s demonstration, which is intended to lower costs and improve health behaviors. GAO added that data for this demonstration is critical since the Indiana demonstration has some of the strictest beneficiary engagement requirements in the country.

GAO provided CMS with three recommendations to improve 1115 evaluation and data collection processes.

Firstly, the agency suggested that CMS Administrator Seema Verma should establish written procedures for implementing federal policies that require states to submit a final evaluation report after the end of each demonstration cycle.

GAO said that CMS should also provide guidelines for when states need to provide limited evaluations for non-complex demonstrations. CMS should also clearly define “non-complex” demonstrations for states, GAO added.

Finally, GAO recommends that CMS should implement a policy for publicly releasing federal evaluations of 1115 demonstrations during iterm and final reporting periods. The policy should include a standard for timely release so that experts and stakeholders can analyze the findings.

HHS concurred with the three recommendations and said that CMS has already taken steps to improve the oversight of state demonstration programs.  

“CMS has begun making changes to how it sets due dates for final evaluation reports,” GAO said.

“CMS officials told us that in spring 2017, CMS began requiring states to submit a comprehensive evaluation report for demonstrations in its high priority policy areas for evaluation at the end of each demonstration cycle, rather than after the expiration of the demonstration.”