How to Use Evaluation to Ensure Success and Sustainability
Program evaluation serves many functions, one of which is verifying that your efforts are producing the desired result of more students enrolled in health insurance. It is essential to the future sustainability of your efforts. Although increased health insurance enrollment is a critically important outcome, the ultimate goal of this work is to ensure that students are healthy and ready and able to achieve academically. An evaluation of your program provides the way to use data to talk about and measure your successes and challenges.
To monitor progress, CDF and AASA encouraged action planning and evaluation to help districts determine actionable goals and objectives, to plan and implement this initiative, and capture school district successes, challenges and lessons learned. In the Telling Your Story (TYS) Template and Monthly Reporting Template, district teams were provided a means for tracking and reviewing program implementation or process information — the ‘how,’ as well as enrollment outcome information, and the number of children insured. Program process information is particularly helpful in this relatively new endeavor of connecting students to health insurance through schools. Process evaluation allows staff to determine if there are areas that need to be modified, kept the same or completely changed because the effort is not achieving the desired result.
Process Data: The process data are the culmination of information gathered in this comprehensive toolkit. In this section we highlight some of the most valuable data captured in the TYS Template.
- Administrative: School districts identified the forms they use to ask the health insurance question, “Does your child have health insurance?” As noted in IDENTIFY, some of the forms are:
- Student Health Inventory and Insurance Screening Forms
- Health Collaborative Referral Forms
- Student Emergency Cards
- Homeless Youth Needs Assessment Form
- Sport Enrollment Forms
- School Promotion: Districts were asked to detail steps the team had taken to ensure the sustainability of disseminating educational materials. Districts are regularly posting information on websites as well as educational flyers. Not only is the information regularly being distributed in print and social media, but health insurance has become part of the agenda at parent and public meetings. Some districts have even added health insurance information to the report cards, emphasizing the strong link between education and health.
- Community and Family Outreach: Districts have taken steps to ensure outreach efforts are sustainable by designating personnel to assist with the project and compensating them with stipends for their extra efforts. In addition, in some districts the front office administrators and health room employees are now routinely asking, “Does your child have health insurance?” School districts are also engaging students in poster competitions for getting the word out about children’s health insurance. Some have enlisted the local newspapers to cover stories about efforts to enroll students and families in health insurance.
- Sustainability: School districts have been able to institute many practices into routine school procedures and through school policies. District Improvement Plans have been revised to include children’s health outreach activities and student databases have been updated to include mandatory fields for students’ health insurance information. Key personnel have been identified and trained to assist with health education and insurance enrollment, and districts have capitalized on strategic planning with community partners to minimize duplication of outreach and enrollment efforts and to expand resources.
Challenges: The district teams have had great success connecting children to health insurance, but the successes have not come without lessons learned and challenges. Educating and enrolling children in health insurance through schools is new territory and requires a multidimensional approach involving multiple school personnel and community partners. Some of the challenges our district teams encountered were:
- Carving out the time necessary to do this important work with other job duties,
- Training staff to assist families at community education and enrollment events,
- Having enough staff available for evening community events,
- Publicizing and having good turn-outs at outreach and enrollment events,
- Tracking and reporting education and enrollment numbers,
- Inputting student health insurance data into the district database,
- Generating reports related to insurance status from the database, and
- Coordinating with community partners.
Keeping enrollment data, following up with parents, knowing if the child enrolled, and if they did not enroll and why, ensures we can continue to provide the resources and assistance needed to support the child and family.
Outcome Data: To track outcome measures between 2013 and 2016, district teams reported monthly to their team leaders using the Monthly Reporting Template. Within this template, districts reported on the following:
- Number of uninsured students in their district at the beginning of each school year,
- Estimated number of students and families reached with awareness or outreach activities,
- Estimated number of referrals for insurance enrollment, and
- Total number of newly enrolled or re-enrolled children in health insurance.
The primary outcome measure for this initiative was the number of children enrolled in health insurance. Between 2011 and 2016, fifteen school districts large and small, urban and rural in California, Georgia, Louisiana, Mississippi, and Texas enrolled 12,917 children in health insurance. In addition, districts in California and Texas shared children’s health insurance information with an estimated 300,000 families and made nearly 4,000 referrals for health insurance enrollment.
Monthly Calls: Throughout the project, team networking and sharing was encouraged on monthly calls. Emphasis was placed on connecting with people and building relationships. Teams were not just asked to report on ‘data,’ but also to capture and share real life stories. CDF and AASA understand and value the importance of capturing and sharing stories because it is often through sharing a story that policies are changed.
The Telling Your Story Template and the Monthly Reporting Template: Designed to gather detailed and informative data over time, these templates track progress as your district works toward getting all students insured. Each is meant to be used to help identify what is working and what is not working. Information from the templates should be reviewed at least every quarter throughout the year. This will help gauge program progress and guide efforts. Evaluation tools such as the Action Planning Template, the TYS Template and the Monthly Reporting Template will help you and your team communicate, define your path and monitor each step along the way.
The Children’s Defense Fund Leave No Child Behind® mission is to ensure every child a Healthy Start, a Head Start, a Fair Start, a Safe Start and a Moral Start in life and successful passage to adulthood with the help of caring families and communities.
CDF provides a strong, effective and independent voice for all the children of America who cannot vote, lobby or speak for themselves. We pay particular attention to the needs of poor children, children of color and those with disabilities. CDF educates the nation about the needs of children and encourages preventive investments before they get sick, drop out of school, get into trouble or suffer family breakdown.
CDF began in 1973 and is a private, nonprofit organization supported by individual donations, foundation, corporate and government grants.
AASA, the School Superintendents Association, represents, works alongside, supports, and is the voice of superintendents and education leaders across the United States. Thirteen thousand strong and 151 years old, we remain committed to excellence and equity for each and every child in public schools. Leveraging our 49 state affiliates and national partners, we focus on developing school system leaders who can meet the challenges facing 21st century students and beyond.