Data and Accountability « Advance CTE Blog

2022 Advance Fall Meeting Recap – Forward Together: Amplifying Voices of CTE Leaders

Wednesday, October 26th, 2022

Advance CTE’s 2022 Fall Meeting included five breakout sessions that connected timely topics of advocacy of CTE, equipping attendees with strategies to elevate their voice and the voices of learners and partners to advance high-quality, equitable policies.

This post provides a recap of top strategies and resources shared in each breakout, which featured speakers from state teams, national non-profits, philanthropy and education research and communications.

Engaging Policymakers to Create CTE Champions

With over 30 gubernatorial elections and legislative elections in all but four states, 2022 will likely lead to major changes and transitions in the year ahead for education leadership. Speakers from Education Commission of the States, Advance CTE and the state of Washington provided guidance and tips to successfully prepare for and navigate these transitions. In particular, state CTE leaders should focus on identifying their core messages on the value of CTE, conducting research on what newly elected leaders care most about (and what CTE success stories can be shared from their home districts or communities), and leveraging ambassadors and partners to help carry forward priorities and get the ear of new leaders and their transition teams.

Moving Beyond a Seat at the Table: Advancing CTE through Partnerships and Collaboration

The intersectionality of CTE leads the creation of high-quality partnerships to be not just good strategy but a necessity. The Bill and Melinda Gates Foundation and Achieving the Dream, both national partners of Advance CTE, shared best strategies to create partnerships that will better deploy and connect federal state and philanthropic resources, and ultimately address gaps that impede the success of learners. Speakers emphasized that successful partnerships must have shared objectives and mutual needs. When asked about the conditions philanthropic organizations seek when investing in a new partnership, three main areas are identified: flexibility, having the right individuals at the table and bought in, and an understanding that the partnership is not a product bought but rather an opportunity to create connections.

Connecting Data and Learner Voice for Next-Level Storytelling

This session featuring Advance CTE and Edge Research inspired attendees to amplify learner outcome data and testimonials to create impactful data stories. Attendees were reminded to be in the driver’s seat on creating data ‘headlines’’ and to be proactive in addressing assumptions others might make about data. First step routines were also provided, such as mapping schedules for quantitative and qualitative data collection, quantitative data reporting and qualitative data access opportunities and comparing timelines with storytelling opportunities.

Edge Research provided storytelling tips — one included creating three to five headlines that encompass a data point audiences might not expect, highlight the most urgent findings, and provide a call to action. It is important to humanize qualitative data through phrases such as ‘1 out of X families’ or ‘X number of children need Y’.

Advancing Your State’s CTE Goals through State Legislative Policy

Georgia and Advance CTE discussed effective processes and tools to leverage state legislative policy to advance CTE initiatives. You can find the high-level trends of 2021 state CTE policies documented in Advance CTE and ACTE’s Year in Review. Georgia shared the importance of having a concrete policy agenda and consistently involving stakeholders including the state’s ACTE chapter and state CTSO leadership, in its formulation and execution. They also emphasized the importance of leveraging the learner voice in demonstrating CTE’s value to legislators: “Figure out what they want, and show it to them.”

Supporting Visionary Local CTE Leadership

“Relationship building is the foundation of a robust CTE leaders onboarding system,” stated Dr. Valeria Williams, Director of CTE at the Mississippi Community College Board when explaining the key to successful support of local CTE leaders. Dr. Williams and Cathie Raymond, the State CTE Director in Arizona, provided insight on developing strong onboarding programs for local CTE leaders. Attendees were encouraged to plan early for onboarding to start before the new school/academic year and remain accessible to CTE leaders year-round through intentional technical assistance and convenings. Although CTE leader turnover is high, best practices were shared with the reminder of how retention starts early through identification and empowerment of outstanding teachers to stay in the field and eventually transition into CTE leadership roles, with a critical focus on diversifying leadership pipelines.

Advance CTE provides a plethora of resources to support CTE leaders in amplifying their voice and impact, including tools to communicate data, communicate with families and engage with policymakers and employers.

Stacy Whitehouse, Senior Associate Communications and State Engagement

By Stacy Whitehouse in Advance CTE Fall Meeting
Tags: Arizona, Communicating CTE, Data and Accountability, Georgia, Mississippi, partnerships, professional development, state policy, Year in Review

Vision Spotlight: Implementing CTE Without Limits at the Local Level

Tuesday, September 27th, 2022

In March 2021, Advance CTE released Without Limits: A Shared Vision for the Future of Career Technical Education (CTE Without Limits) to empower “leadership at all levels to reflect on, refine and even rebuild many of the systems and structures that are limiting learner success.” Later that year, we subsequently released Pushing the Limits: A Roadmap for Advancing CTE Without Limits to help leaders assess, prioritize and implement strategies for one or more vision principles.

One school district – Utah’s Davis School District (DSD), just north of Salt Lake City – has taken both CTE Without Limits and Pushing the Limits to heart. DSD is recognized across the state for strong pathway programs. After their Comprehensive Local Needs Assessment (CLNA) process revealed achievement and enrollment gaps for special populations in CTE, DSD administrators sought bold solutions and turned to CTE Without Limits as a framework to close these gaps. Leveraging these resources, DSD supported professional district training and strategic school plan development with the express purpose of increasing equitable enrollment in CTE programs districtwide.

Implementing CTE Without Limits

Once DSD administration began conversations with educators and staff, they recognized the need to create schoolwide “Without Limits Teams,” comprised of a school administrator, special education staff, school counselors, and the school’s CTE coordinator. Last spring, school staff from each of the 25 DSD schools were each brought together in full day district-wide workshops structured on the five CTE Without Limits principles and designed using Advance CTE’s CTE Without Limits implementation assets. After the training, the teams were asked to create a Without Limits plan to implement these principles in their building; 19 of the 25 have already submitted their plans as a result of the training.

The plans were created using a district-wide rubric to help sharpen and focus action areas. Learner voice was also incorporated into the conversation, with the district conducting twelve focus groups with 54 students, asking similar questions as the workshops for district staff. The district also conducted pre- and post-surveys to help develop priorities for these action plans and help administrators learn where schools need the most support.

Collaboration is Key

In a recent interview with Advance CTE staff, Davis School District CTE Supervisor Tim Peters and Without Limits Project Lead Melanie Allen reiterated the value that the collaborative nature of these Without Limits Teams has for CTE district-wide and with state and community partners. Because of the composition of the Without Limits Teams, the cross-departmental work has become a key strength of this initiative. The Layton High School team gave high praises to the Without Limits initiatives and said, “The impact of everyone being on the same page can’t be overestimated for the teachers and the students. We are already seeing growth in course requests for welding tech, accounting and computer programming.” Local CTE Coordinator Kristen Davidson noted, “We’re having conversations now that we’ve never had before, and it’s changing the way we serve students.”

Utah State CTE Director Thalea Longhurst is a strong supporter of DSD’s model. “Davis School District is leading our state in focusing on truly embedding CTE Without Limits in their daily work,” she shared. “Their work is led by strong leaders who understand the importance of collaborative conversations and truly using data-driven decision making to improve programs. They have shown that even small changes can be incredibly impactful while still considering and implementing larger systemic changes. Their work is exactly what we hope others will follow as we all strive to implement this vision for the future of CTE.”

Lessons Learned

After their first iteration of workshops, DSD shared a few lessons learned that they hope other Utah school districts or other districts nationwide consider before starting this work in earnest.

Plans don’t have to make world-changing shifts. Instead, these workshops and subsequent action planning helped the district see the small, most impactful changes that could happen first while working toward larger systemic changes.
Workshops also revealed the necessity of reviewing data as a part of the process. Because the district considered their data and opportunity gaps in conjunction with action planning, they could focus their efforts on learner populations that require the most support.
A shared framework and regular collaborative conversations served as a forcing event to share important, consistent information with school educators and staff, disseminate resources, and provide updates about CTE programs. This allowed for opportunities to address myths about CTE programs and initiatives.

Next Steps

As a result of the workshops, plan feedback, and learner focus groups, the district is working on a toolkit to support the school plan action steps. DSD administrators also hope to make this an iterative process, conducting ongoing training and coaching to help their Without Limits Teams revise and refocus their plans on a yearly basis.

For more information about CTE Without Limits as well as communication and implementation resources, please visit the CTE Without Limits page on our website.

Learn more about Career Technical Education at Davis School District in Farmington, UT.

Dan Hinderliter, Senior Policy Associate

By Stacy Whitehouse in CTE Without Limits, Uncategorized
Tags: CTE Without Limits, Data and Accountability, Equity and Access, Utah

Advance CTE’s Opportunity Gap Workshop: Connecting Data to Action

Tuesday, August 30th, 2022

In 2020, Advance CTE developed and piloted a virtual workshop to help Career Technical Education (CTE) leaders at the state and local level identify and address access and equity gaps using data. This initiative supports the realization of Principle 2 of the CTE Without Limits vision where each learner feels welcome in, is supported by and has the means to succeed in the career preparation ecosystem.

The workshop includes five major components that build upon the requirements — as well as opportunities — laid out in the Strengthening Career and Technical Education for the 21st Century Act (Perkins V) to help state and local leaders operationalize the commitments set in their state plans:

Raising awareness,
Data analysis
Root cause analysis
Identification of practical strategies, and
Action planning.

The first workshop cohort of 10 states launched September 2021 after successful pilots were conducted in New Jersey, Florida and Minnesota the previous year. Initially a one-day, five-hour training, the workshop now provides two four-hour training sessions on the materials created for state CTE leaders to in turn train their staff, regional coordinators and local education agencies. Participant receive the following materials:

Facilitator Guide – Virtual or in-person instructions.
PowerPoint Deck Template – Customizable slide deck for participant use.
Participant Workbook – Activities and additional step-by-step instructions.
Opportunity Gap Analysis Dashboard Template – Fillable pre-formatted Excel spreadsheet with graphs, a heat map, and instructions on data entry and comparison groups.
Practical Strategies Resource – Warehouse of evidence based strategies to align to identified root causes.
Action Planning Template – Facilitation tool to encourage stakeholders connection and collaboration.

Upon completion of the workshop, participants are required to plan and deliver the workshops in their states; evaluate the effectiveness of the workshop using a provided survey for their in-state participants to share with Advance CTE; and complete a six-month and 12-month survey to report any changes in CTE policies and procedures.

Workshop Benefits and Outcomes

Collaborative Spaces and Technical Assistance: The effects of the coronavirus pandemic on the education system has led to a reduced capacity of state CTE teams, which prompted Advance CTE to increase the intensity of technical assistance provided to states. Advance CTE staff launched a series of monthly community of practice calls to connect states with common challenges and to workshop solutions. To provide states with more time to plan and deliver their own in-state trainings, the cohort’s timeline was extended from three to six months.

Diversity, Equity and Inclusion: The workshop provides participants with foundational knowledge of equity prior to delving into data analysis. For example, the first objective — raising awareness — consists of developing an equity baseline for participants by reviewing the definitions for diversity, equity and inclusion; examining equity vs equality; and addressing the concept of deficit based thinking in comparison to strength based analysis. The workshop format allows for participant processing time on equity concepts, gives time for states to fully meet all five objectives, and allows the incorporation of another Advance CTE resource, Brave Dialogues: A Guide to Discussing Racial Equity in Career Technical Education.

Data Analysis Training and Dashboard Development: Mock data (from the fictitious Heartland Public Schools and Heartland Community College) is used to mitigate the often difficult conversations associated with participants reviewing their data and keeps the focus on workshop training concepts and applications. The workshop facilitators intentionally meet states where they are in their data journey. Some states have data disaggregated by student sub-groups, but may not have data elements available for programs of study level analysis. The fillable Excel spreadsheet provided to participants is customizable and allows for Career Cluster level analysis. The identification of an opportunity gap narrows the scope of the analysis; however, a statewide, regional, institutional, school district, or campus analysis is possible using the dashboard template. An additional work-based learning (WBL) dashboard was developed upon request and is available for states to examine equitable access to and completion of industry-recognized certifications.

Analysis and Action Planning: For data to be an effective storytelling tool, participants have to recognize and be willing to interpret a point gap analysis through an equity-minded lens using data disaggregated by race, gender and ethnicity as well as the special population categories identified in Perkins V.

Providing workshop participants with a list of practical strategies, which are then selected based on a root cause analysis, increased the availability of resources states could offer local education agencies due to in-state dissemination of workshop materials. The accompanying action planning template and design allows state and local leaders to engage stakeholders with a structured process for developing solutions to gaps in access to high-quality CTE programming for marginalized learners.

Next Steps

The launch of cohort two in June of 2022 offered an opportunity to overlap communities of practice and allowed cohort two participating states to collaborate with states from the inaugural cohort. Currently Advance CTE is accepting applications for cohort three of the initiative and anticipates actively overlapping cohorts two and three into collaborative communities of practice. The cohort three application closes September 2, 2022.

Dr. Kevin Johnson, Sr., Senior Advisor

By Stacy Whitehouse in Advance CTE Resources
Tags: Access and Equity, dashboards, Data and Accountability, Florida, Minnesota, New Jersey, Opportunity Gap, Perkins V, Strengthening Career and Technical Education for the 21st Century Act

CTE Research Review: Q&A with Shaun Dougherty on the Impacts of CTE Access on Learner Outcomes in Connecticut

Wednesday, June 15th, 2022

This research review series features interviews with three CTE researchers— Julie Edmunds, Shaun Dougherty, and Rachel Rosen — to highlight new and relevant Career Technical Education (CTE) research topics being pursued and discuss how state CTE leaders might leverage these to make evidence-based decisions. This series is conducted in partnership with the Career and Technical Education Research Network, which is providing new CTE impact studies and strengthening the capacity of the field to conduct and use rigorous CTE research.

For the final post in this series, Advance CTE caught up with researcher Shaun Dougherty to learn more about the outcomes of his study, The Effects of Career and Technical Education: Evidence from the Connecticut Technical High School System. This study took place at The Connecticut Technical High School System (CTHSS), an independent district of choice composed of 16 high schools where all students who attend the school participate in some form of CTE. The technical high schools in this study offer between 10 and 17 programs of study compared to the two to four typically offered at traditional comprehensive high schools. These findings suggest that student participation in CTE programs can improve graduation and college enrollment rates, and labor force participation, especially for male students. These findings should be meaningful to state leaders who are interested in improving CTE data quality to improve programming and learner supports.

How do your research findings advance the CTE field’s understanding of ways to better serve learners?

My findings in Connecticut and an earlier study in Massachusetts suggest that increasing access to high-quality CTE-focused schools can improve high school graduation rates, test scores, and early workforce experiences (earnings especially). These findings suggest that ensuring access to high-quality CTE is important, especially in areas where there is high demand for it among students. In Connecticut, we show that the benefits accrue to males only, but in Massachusetts, the impacts are broader-based and larger for students who come from less financially advantaged backgrounds.

What is also important about both of these contexts is that students are attending CTE-dedicated high schools, which is the least common form of CTE instructional delivery in the United States. Students explore multiple programs in 9th grade and then make an informed decision about what program of study to pursue for the rest of their high school experience. Within the program they choose, they then have a relatively stable set of peers and instructors, and there is alignment between content in the state graduation requirements, such as math and English Language Arts (ELA), and their technical area of study. Each of these dimensions of the experience tends not to exist in CTE delivered in comprehensive high schools or regional technical centers.

What findings would you highlight for state CTE leaders in particular?

In particular, I’d highlight the following:

Whole-school models of CTE may be more expensive but have been shown to have quite larger impacts on student success and so could be worth the long-term investment.
If whole-school models aren’t feasible, adapting some of the structures of how those schools operate in other settings could be worth pursuing.
Creating ways for students to explore multiple programs of study early in high school is likely to improve the experience of all students as they get more information about their interests and strengths.
Investing in skilled trade programs likely helps mitigate otherwise undesirable educational and social outcomes, especially for boys who struggled to be engaged in middle school. Avoiding tracking is critical, but also finding ways to engage students in ways that likely benefit them and their families and communities is critical.

You find male students enrolled in Connecticut’s Technical High School System (CTHSS) are more likely to graduate high school and experience a notable wage increase compared to males attending traditional high schools, yet this effect is not seen among female students. What do you make of this finding?

There are two things worth noting here:

Male labor force participation in the U.S. is generally lower than females, as are their likelihoods of completing high school or entering college. Thus, there is more opportunity to have a positive impact on males than females.
Males are more likely to enter traditionally male-dominated fields, such as manufacturing and construction, which are the two industries where we see the largest payoff for participating in these technical high schools. Thus, gendered patterns of program participation are related to similarly gendered patterns of employment, which are directly related to the earnings premiums we see for males.

Can you tell me about how you were able to leverage the state’s longitudinal data system, the value that this process brought to your work, and about any limitations you encountered??

We matched the CTHSS admissions records to the Connecticut State Department of Education’s (CSDE) longitudinal data system sequentially using the following criteria: SASID, exact match on first and last name plus birth year, first initial and exact match on last name plus birth year and month, and exact match on last name plus exact birth date. The reason for the sequential match process is reporting errors in the CTHSS application file on birth dates, spelling errors and uses of nicknames in the application file that parents and/or students fill out by hand. A failure to match after applying all of these criteria leads to the observation being omitted from the sample. Our resulting match rate was 95 percent yielding a final sample of 57,658 student applications.

From the CSDE longitudinal data system, we obtained information on each student’s race, gender, free or reduced-price lunch status, English learner and special education status (i.e. presence of an IEP).

The CSDE data also provided information on short- and medium-term educational outcomes including standardized test scores prior to and during high school, attendance, and high school graduation, as well as information on college attendance drawn from the National Student Clearinghouse.

Further, through the P20Win process, students in our sample were matched to Connecticut State Department of Labor (CSDOL) data. This CSDOL match is facilitated by Department of Motor Vehicle records that contain gender, birth date, and first and last name and is matched to the CSDOL data using social security numbers. CSDOL personnel then matched the resulting data to the CSDE data using an exact match on birth date and gender and a fuzzy match algorithm on the student’s name.

Failure to match applicants in the CSDOL data may have been driven by several factors including never having a driver’s license in the state of Connecticut, name changes due to marriage or other factors, moving out of state prior to or upon the completion of high school or failure to participate in the labor market after high school perhaps due to college attendance. Our labor market data ends in the 1st quarter of 2018. Therefore, we restrict this sample to the years 2006 to 2012 so that we have a potential for at least six quarters of data on each applicant.

Without the matching to the workforce outcomes, we would be left to wonder or speculate about what happened to most students, since there is no real evidence of impact on college-going (nor should we expect it early on for students who attended schools that emphasize manufacturing and skilled trades). Thus, the linkages were critical to being able to provide policymakers with relevant, detailed answers

The primary limitation is that we cannot see workers who moved out of state, who were employed by the federal government, who work for unreported wages, or who are independent contractors. However, our takeaway is that our findings are likely an underestimate of the benefits of CTE.

What new questions has this work raised for you that could be applied to future research?

The two biggest questions this work has raised for me so far are:

First, what financial benefits will persist through middle age and beyond?
Other work has shown that early-career benefits of CTE may decline or turn negative over a 30-year career if the jobs and skills become obsolete or workers are not able to maintain their labor force participation. The evidence that raises this concern is from much earlier periods, so following this over the long term is critical.
Second, are there conditions under which females see similar, or at least positive benefits? The gendered patterns of results are both informative and distressing. For instance, Sade Bonilla (2020) found larger high school graduation benefits from CTE in California where recent expansions of CTE had focused on health services pathways, which were both in demand and disproportionately female. Relatedly, it would be important to get other measures of benefits, such as job satisfaction, flexibility, or other indirect financial benefits. For example, females are more likely to go into education. For early childhood educators, there may be an indirect financial benefit in the form of free or reduced-cost childcare for one’s own children. Further, flexibility in work schedules around children’s school schedules may also carry weight in estimating benefits.

Visit the Learning that Works Resource Center for additional publications examining career-centered education models and Advance CTE’s 50-state report on equity in CTE early postsecondary opportunities (EPSOs) released earlier this year.

The work of the CTE Research Network Lead is supported by the Institute of Education Sciences at the U.S. Department of Education with funds provided under the Carl D. Perkins Career and Technical Education Act through Grant R305N180005 to the American Institutes for Research (AIR). The work of the Network member projects is supported by the Institute. The opinions expressed are those of the authors and do not represent the views of the Institute or the U.S. Department of Education.

Amy Hodge, Policy Associate

By Stacy Whitehouse in Research
Tags: Connecticut, Data and Accountability, Early Postsecondary Opportunities, Equity and Access, Massachusetts, State Longitudinal Data System, student outcomes

Resource Recap: 5 Steps to Get Started with the CTE Without Limits Roadmap Tool

Thursday, March 31st, 2022

March 2022 marks one year since the release of Without Limits: A Shared Vision for the Future of Career Technical Education (CTE Without Limits). CTE Without Limits positions CTE as the catalyst for achieving a cohesive ecosystem that is responsive to each learner’s needs for college and career success.

To celebrate this milestone, Advance CTE elevated the work of vision partners through a Twitter chat, announced three states participating in a state cohort for vision implementation, and held a vision-focused Ask an Expert session. The event delved into Advance CTE’s hallmark vision implementation resource released last fall, Pushing the Limits: A Roadmap for Advancing CTE Without Limits.

This resource recap post breaks down the roadmap resource and provides first steps for state CTE leaders to prepare for and use this comprehensive tool.

Resource Background

Achieving CTE Without Limits is only possible through shared commitment and action among all CTE stakeholders The Pushing the Limits roadmap serves as the primary evaluation and planning tool for state and local CTE leaders to conduct a collaborative process that: a) provides an initial assessment of state policy and practice; b) identifies top areas for action; and c) develops implementation strategies for one or more vision principles.

The document is provided in both a combined format as well as separate by each of the five vision principles. The three to four action steps recommended for each vision principle can be evaluated by state leaders through five activities:

Data Review: Collect and analyze data to identify gaps in data availability as well as equity and opportunity gaps for learners that will impact roadmap planning.
State Assessment: Provides self-assessment questions to help state leaders reflect on the current alignment of policies and practice, capacity for change, and potential impact of moving the needle for each action. The completer assigns a score to each section.
SWOT Analysis: Applies information from self-assessment questions to identify top-level Strengths, Weaknesses, Opportunities and Threats to advance each action step.
Models and Resources: Provides sample implementation steps, policies and resources to inspire action;
Heat Map: Scores reach assessment section in a heat map that identifies intersecting areas of high need and high capacity for change, to aid leaders in narrowing the focus of their work.
Action Planning: Offers an action planning tool to develop SMART (Specific, Measurable, Attainable, Relevant and Time-Bound) goals and action steps in real-time, six months and a year to drive systemic change.

Getting Started

Effective use of the roadmap requires intentional planning and collaboration. Here are five ways to get started:

Select your core state or local team that includes representatives with deep experience in K-12, postsecondary and workforce policy and practice through the lens of CTE.
Complete Advance CTE’s State Capacity Tool to determine which vision principle(s) to focus on.
Gather data and guiding documents to inform the self-assessment, including program participation and outcomes, statewide and regional agreements across system, statewide and department initiatives and goals, etc.
Complete the self-assessment as individual core team members and share answers prior to soliciting additional input.
Identify priority principles and action areas to address first based on conditions in your state.

Maximizing the Resource

CTE leaders can take several steps to maximize this resource to realize systems change at all levels, including:

Input: Receive input on the self-assessment questions beyond your core team, including learners, educators, administrators and support staff at the local, regional and state level. This can be accomplished by sharing the roadmap questions or creating a separate survey of selected questions from the roadmap in multiple choice format.
Capacity: Align action planning to meet the capacity of state team members, partners and institutions.
Aim Higher: At the same time, revisit your ‘dream list’ of initiatives, supports and goals and leverage the roadmap to push your work to the next level.
Utilize Existing Collaborative Channels: Don’t reinvent the wheel; utilize board and commission meetings, workgroups, conferences and other state and local collaborative events for information gathering and roadmap completion.

Advance CTE staff are available to support CTE leaders in this important work. Visit Advance CTE’s staff web page for contact information. Visit Advance CTE’s vision web page for additional vision education, assessment and implementation resources.

Stacy Whitehouse, Senior Associate Communications and State Engagement

By Stacy Whitehouse in CTE Without Limits, Uncategorized
Tags: CTE Resources, CTE Without Limits, Data and Accountability, systems alignment

Improving CTE Data Quality: Information is Used Effectively to Promote Quality and Equity in Career Pathways

Monday, May 17th, 2021

A high-quality career readiness data ecosystem should not only collect, analyze and report data but also use the information to promote quality and equity. State leaders should integrate career readiness data into policymaking and decision-making processes to further a statewide career preparation system that is high quality and equitable.

Ohio’s Strengthening Career and Technical Education for the 21st Century Act (Perkins V) Equity Labs guide key stakeholders through a process of identifying gaps in access to or success in Career Technical Education (CTE) programs, discussing root causes of those gaps, and creating actionable plans to address them. In 2019, for the first round of Equity Labs, each of Ohio’s 91 Career-Technical Planning Districts (CTPD) — local education agency configurations that consist of a number of school districts and deliver CTE programs — submitted their data to the Ohio Department of Education, which then developed a report with data in three main categories: meaningful access, engagement and enrollment, and student outcomes. The data in the report is disaggregated by race/ethnicity, gender, migrant status and special population status. Each CTPD was invited to bring five to 10 internal stakeholders to participate in the Equity Lab to analyze the data, identify the largest and most pressing gap, and conduct a root cause analysis on the identified gap. Lastly, participants developed a SMART (specific, measurable, attainable, relevant and time bound) goal to address the gap and a plan to evaluate movement toward improvement.

The Ohio Department of Education showed districts how the exercise of analyzing data, identifying root causes, and creating actionable plans to address the identified gaps can be applied to a district’s Comprehensive Local Needs Assessment (CLNA). The CLNA is required by Perkins V and mandates that local districts evaluate their progress toward equitable outcomes for all students. By connecting the Equity Labs to the CLNA, districts can better understand which groups of students are gaining access to CTE programs of study that lead to high-wage, high-skill or in-demand careers; where the gaps exist; and what systems-level changes should be made so every learner can benefit from meaningful, high-quality CTE.

Bringing together local leaders to analyze and reflect on career readiness data and acknowledge the areas that need improvement is critical to ensure that the statewide career preparation system is high quality and equitable for all learners. Read the Advance CTE case study, Ohio’s Perkins V Equity Labs, to learn more about how Ohio leverages data to improve the quality and equity of its CTE programs. For additional resources on improving the quality and use of career readiness data, check out the Career Readiness Data Quality microsite.

This is the sixth and final edition in a series of Advance CTE data quality blogs to accompany Advance CTE’s latest releases, Career Readiness Data Quality and Use Policy Benchmark Tool and Data Quality Case Studies. For more resources on data and accountability or access and equity in CTE, please visit the Learning that Works Resource Center.

Brian Robinson, Policy Associate

By admin in Uncategorized
Tags: career readiness, Data and Accountability, Ohio

Improving CTE Data Quality: Data Is Collected Consistently and Accurately

Wednesday, February 17th, 2021

In order for data to be trusted, policies and protocols must be in place to ensure consistent collection of reliable, valid and complete career readiness data. States can establish universal definitions and automated processes to collect and interpret data and work with practitioners and the public to foster an understanding of data elements to build trust in their data.

One example is Texas’ automated learner identification system and statewide programs of study. Prior to the establishment of this new system in 2015, each local school district could develop its own programs of study and course sequences. This led to inconsistent data collection, as districts could have different course requirements for the same program of study. Additionally, the state relied on districts to self-report the number of Career Technical Education (CTE) concentrators based on their locally-developed programs of study. Because districts decided which courses counted towards concentration or completion of a program of study, this meant that learners could complete a course in one district that counted towards one program of study, but that same course would not count in another district.

Texas addressed this massive data challenge by creating new statewide programs of study and a uniform framework for collecting data. To develop these programs of study, several state agencies including the Texas Education Agency, Texas Workforce Commission, Texas Workforce Investment Council, and the Texas Higher Education Coordinating Board worked together to leverage labor market information and set benchmarks for in-demand, high-wage, and high-skill occupations. Related occupations that satisfy the criteria were then grouped together and a set of course sequences with accompanying course codes were developed to form a program of study. Because of these changes, every school district now has the same grouping of available aligned courses for each program of study.

Read the Advance CTE Case Study Collecting Data Consistently and Accurately: Texas’ Automated Learner Identification System and Statewide Programs of Study to learn more about how Texas is improving CTE data quality. For additional resources on improving the quality and use of career readiness data, check out the Career Readiness Data Quality microsite

This is the first edition in a series of Advance CTE data quality blogs to accompany Advance CTE’s latest releases, Career Readiness Data Quality and Use Policy Benchmark Tool and Data Quality Case Studies. For more resources on data and accountability in CTE, please visit the Learning that Works Resource Center.

Brian Robinson

Policy Associate

By admin in Uncategorized
Tags: Data and Accountability, data quality

COVID-19 Will Disrupt Early Postsecondary CTE Credit Attainment: Don’t Make Learners Bear the Burden

Monday, June 15th, 2020

The last weeks of March were a test of adaptability for the nation’s education institutions. Over the span of weeks, school districts, community/technical colleges and area technical centers had to amend remote learning policies amid unprecedented school closures. The rapid pace at which systems and institutions have adapted to the COVID-19 (Coronavirus) pandemic has been likened to turning a battleship at the pace of a speedboat.

Early Postsecondary Opportunities (EPSO) – which include dual enrollment, dual credit, concurrent enrollment and other related opportunities – play an important role in facilitating successful transitions between secondary and postsecondary education. According to the National Center for Education Statistics, approximately one-third of high school graduates take courses for postsecondary credit at some point during their educational careers. The integration of EPSO courses in Career Technical Education (CTE) pathways is creating options-rich, intuitive linkages between high school, college and career. This approach yields benefits to students, their families and their communities: earning postsecondary CTE credit in high school can lead to higher rates of college enrollment, persistence and success. But as schools, districts and colleges adapt to a new Coronavirus reality and a pervasive shift to online learning, where does that leave learners who are enrolled in CTE EPSO courses?

Coronavirus-related shutdowns put pressure on CTE EPSO courses in a number of ways:

Credit attainment and transfer: Many institutions have transitioned to binary (pass/ fail or satisfactory/unsatisfactory) grading for the spring and summer 2020 terms in recognition of the myriad of equity and logistical challenges presented by an abrupt, unexpected move to online education. This approach attempts to hold the learner harmless in weathering this dramatic shift in their education. Binary grading may have a wide-ranging impact on the acceptance of transfer credit of which EPSO credit is a subset. Like transfer credit, EPSO credit enjoys wide acceptance, often guaranteed by state law or system governance, but those policies were not developed to account for widespread use of binary grading.
Applied curriculum: Hands-on learning is a key feature of CTE, and without access to school buildings, learners may face challenges in demonstrating all of the skills and competencies necessary to receive credit for coursework. Even as many postsecondary courses transition to online and remote delivery, many practical CTE courses have been suspended altogether or significantly scaled back. Similarly, programs that integrate work-based learning activities as an important component of the curriculum are struggling to gain access to clinical hours, internships and other high-value college and career crossover activities. There is an opportunity to look at thoughtful approaches that extend deadlines for course completion, leveraging competency-based approaches, badging, certifications, portfolios and challenge exams to document the attainment of learning outcomes by students.
Funding: Funding for EPSOs varies from state to state, but in most cases tuition is paid through a combination of state funding, support from the learners’ home districts and from the learners themselves. Funding to decrease the cost of participation is an important equity lever for states looking to ensure they have a strong, robust talent pipeline to address their current and future workforce needs. What happens if a class is suspended mid-semester of if a learner withdraws due to circumstances related to the Coronavirus? States and institutions will need to clarify how to address these cases.
Data and accountability: More than a dozen states are measuring postsecondary credit attainment as their CTE program quality indicator for the Strengthening Career and Technical Education for the 21^st Century Act (Perkins V) performance accountability system. Enrollment disruptions in the spring and summer 2020 terms will impact states’ performance benchmarks, and if closures continue into the fall, this will impact their first year of Perkins V reporting. The Coronavirus also threatens to affect the other secondary CTE program quality indicators, as many learners earn recognized postsecondary credentials or complete work-based learning opportunities through partnerships with postsecondary institutions. EPSO programs are built on partnership and require mutual support, understanding and investment by school districts and postsecondary institutions. These partnerships create a better opportunity pivot in approach and resource sharing.

The field has been quick to recognize these challenges, but given the decentralized nature of higher education – in most cases, articulation agreements for CTE credit are negotiated at the local level between individual districts and partner colleges – the response has been inconsistent and incomplete. This situation creates both challenges and opportunities.

So how can states respond to some of these challenges? Next week the National Alliance of Concurrent Enrollment Partnerships (NACEP) and Advance CTE will explore some strategies to minimize the burden on students and honor their commitment to learning.

This blog post is the first in a two-part series about the impacts of the Coronavirus on CTE dual enrollment. It was written by Amy Williams, Executive Director of NACEP, and Austin Estes, Manager of Data & Research for Advance CTE.

By admin in COVID-19 and CTE
Tags: Concurrent Enrollment, Coronavirus, COVID-19, Credit Transfer, Data and Accountability, dual credit, dual enrollment, Early Postsecondary Opportunities, postsecondary

How to Seek Funding to Support CTE Research Partnerships

Thursday, May 28th, 2020

Over the past six months, Advance CTE and the Institute of Education Sciences (IES) have worked together to highlight the power of Career Technical Education (CTE) research partnerships in improving quality and equity in CTE. In Michigan, years of close collaboration between the Department of Education and the University of Michigan has enabled state leaders to address critical policy questions like choosing a secondary CTE program quality performance indicator. South Dakota leveraged relationships in the research community to improve data quality and foster a data-driven culture at the state level. And in Massachusetts, state leaders are working alongside long-time research partners to identify critical access and opportunity gaps and build solutions that enable equitable access to high-quality CTE.

Partnerships like these provide measurable benefits by allowing state policymakers to make informed decisions that impact learner success and bolster state talent pipelines – but they do come at a cost. The partnerships highlighted in this series were supported via a combination of state, federal and foundation funds. Research grant funds are most often used to cover personnel time for work on the research project, both at the university or research organization and at the partner education agency. As many of the state agency interviewees mentioned, it is difficult to carve time out of their regular responsibilities to work on a research project. By securing dedicated funding to cover part or all of a person’s salary, a state agency can afford to spend time on a research project. In addition, research grant funds can be used to provide incentives for students, teachers and schools to participate in a research study, for the development and administration of surveys or classroom observation tools (to complement information available in administrative data systems), and for software and hardware to analyze and house the data.

With growing public support for CTE, fueled by urgent needs for skilled labor, CTE programs will be called upon to do even more. States should be prepared with a research and evaluation strategy to determine whether and which strategies are most effective (and cost-effective). So how should states go about establishing and funding new CTE research partnerships?

Options for Financing State CTE Research Partnerships

There are a number of avenues states can take to finance CTE research. Federal sources of funding for CTE-related research include the Department of Education, the Department of Labor and the National Science Foundation. Education research funding may also be available at other agencies, such as the Department of Agriculture. Private funding for CTE research projects is also available from foundations such as the ECMC Foundation[i], the Smith Richardson Foundation, and the Laura and John Arnold Foundation.

The National Center for Education Research (NCER) at IES launched a special CTE topic in its Education Research Grants program in 2017 to encourage researchers to study CTE. Funded grants under this topic have examined CTE-related issues such as industry certifications, applied-STEM (Science, Technology, Engineering and Mathematics) CTE pathways and work-based learning. IES also funds CTE research under other programs and maintains a CTE Statistics webpage. In 2018, in partnership with the Office of Career, Technical, and Adult Education (OCTAE), IES formed a CTE Research Network to increase the amount and quality of causal research in CTE. CTE Research Network members have been studying the impact of various CTE programs and delivery models on student high school, postsecondary and labor market outcomes. The National Center for Special Education Research (NCSER) began funding CTE research for students with disabilities in 2019.

New Opportunity to Apply for Federal Funding to Study CTE!

There is good news for state leaders and researchers interested in initiating CTE research partnerships. NCER has just released its Fiscal Year 2021 Request for Applications (RFA) for its Education Research Grants Program (CFDA 84.305A). This grants program, one of several in NCER, was established in 2002 to produce research that is scientifically rigorous and relevant to the needs of education practitioners and decisionmakers. NCER welcomes CTE-related research proposals under the CTE topic or under other topics (such as STEM, Improving Education Systems, and Postsecondary and Adult Education). NCSER has a separate RFA for its special education research grants program (CFDA 84.324A) and welcomes applications to study CTE for students with disabilities.

Research grant applications are due at midnight (Eastern time) on August 20, 2020. Letters of intent (not required but encouraged) are due on June 11, 2020. Each of the open RFAs, as well as archived webinars for applicants about the IES grant process, are available on the IES funding opportunities page.

Applicants should start early to make sure they have everything they need. In addition to viewing on-demand webinars, applicants should be sure to read the RFA closely and pay attention not only to the application requirements but also to the IES recommendations for a strong application. For example, applicants should describe their theory of change and any prior research on the issue; align their research methods to the research questions; describe measures and data source; and make sure the sample size offers adequate statistical power. This grants program is very competitive, and peer reviewers will be paying attention to whether applicants follow the recommendations. Everyone involved in the submission process should also familiarize themselves with the IES submission guide, which details the steps necessary to successfully submit an application online.

We are eager to hear any and all ideas! Corinne Alfeld (Corinne.alfeld@ed.gov) and Austin Estes (aestes@careertech.org) would be happy to discuss them, and Corinne can also provide technical assistance in writing research grant application to IES. She can be reached by email to set up a phone call to discuss project ideas.

This final blog post wraps up our series aimed at increasing state CTE research partnerships by highlighting ways to seek research funding. Corinne Alfeld, Research Analyst at IES and Austin Estes, Manager of Data & Research at Advance CTE, collaborated to create this blog series in the hopes that more state agencies would partner with researchers to examine research questions related to CTE using state data.

[i] The ECMC Foundation is a funder of Advance CTE’s work.

By admin in Uncategorized
Tags: CTE Research, Data and Accountability, funding, Institute of Education Sciences, National Center for Education Research

Learning from CTE Research Partnerships: Using Data to Address Access and Equity Barriers in Massachusetts

Tuesday, March 17th, 2020

As part of our ongoing blog series aimed at increasing state research on Career Technical Education (CTE), Corinne Alfeld, Research Analyst at the Institute of Education Sciences (IES) and Austin Estes, Manager of Data & Research at Advance CTE, are conducting interviews with individuals who are part of successful CTE State Director research partnerships. The third interview was with Cliff Chuang at the Massachusetts Department of Education and Shaun Dougherty of Vanderbilt University. [Note: this interview, from February 5, 2020, has been edited for length and clarity].

Could you start by talking about the projects that you’ve worked on, your research questions, and how you settled on those research questions?

Shaun – It grew out of my dissertation work that was using some of the school data and then some of the statewide data from Massachusetts. It started pretty narrowly but the director of research was happy enough with what I was able to do that she talked about whether we could address some additional questions, and more data was becoming available. That more or less triggered the expansion, and then with Cliff coming into the role it became a two-way conversation that was more explicitly about what’s of academic interest and what’s of interest or of need on the practice and policy side for CVTE [career/vocational technical education].

Cliff – I would say that the particular catalyst for our most recent partnership is our desire as an agency to understand the waitlist demand issues related to chapter 74 CVTE in Massachusetts. If I recall correctly, we put out an RFR (request for responses)* for a research partner to help us analyze different aspects of who is and is not getting access to CVTE programs in Massachusetts. And Shaun and his partner Isabel at Harvard, a grad student there, their bid was selected. From that project there have been a lot of offshoots through the CTEx exchange collaboration that Shaun and others have established. We’ve been engaged in a lot of informal research inquiry as well as additional formal research that uses that data.

Could you talk a little bit about what the findings were from that project and what have been implications in the academic space but also on the policy front, how are you using those findings to change policy in Massachusetts?

Shaun – The basic findings were that in fact there is much more interest in these high-quality CTE programs, these chapter 74-approved programs in these standalone technical high schools, than can be met by current supply. This was more confirmatory evidence with a little more granularity and maybe confidence in the figures than was possible previously.

Cliff – Shaun’s team also helped us look at just the straight enrollment data comparisons, which is still not as ideal as looking at applicant data. It was helpful to have a more rigorous definition of what data protocols are needed around application and admissions. We have now made the decision to collect waitlist data systematically at the state level to allow researchers like Shaun to more rigorously analyze across the board the attributes of who’s interested in voc tech, who’s getting in, who’s applying, etc.

I think it also stimulated a variety of program initiatives on the part of state government in Massachusetts to increase access to CVTE programs through collaborative partnerships like After Dark, which is an initiative that seeks to utilize shop space in our technical schools after the regular school day paired with academics provided by a partner academic school to get more kids the technical training that we are unable to do in the standard day program structures.

I would also add that Shaun is continuing other aspects of the research now that we’re very excited about, based in part on some of the research they did do to look at longer term trends of students and their outcomes post high school.

Shaun – The first order concern is that lots of people want [access to CVTE programs] and there’s a limited amount of it, so should we have more?

The second order concern – but certainly not secondary question – is one about equity and whether or not the students who were applying and the students who were getting access look like a representative cross-section of the community at large. We know that students who choose CTE or select a lot of it are maybe different than those who don’t, but we don’t know a ton about whether and how we expect students who are making those investments to look like the overall population or whether or not access concerns lead to equity concerns.

Cliff – We would like to look more closely at whether the gaps are simply due to application gaps – which is still an issue in terms of kids not applying – or whether there are actual gaps related to who is applying and getting in. That was the data gap that we haven’t quite been able to close yet. But Shaun was able to create some comparative data that is just based on enrollment that has allowed us to engage in these conversations. We’re having the conversation about trying to expand the number of seats available so there’s less of a waitlist, but also to ensure that access into the existing seats is equitable and doesn’t disadvantage certain subgroups over others.

Over the course of the partnership, what have been some of the major challenges and hurdles that you’ve faced? What are some of the speedbumps that you’ve hit getting things formalized up at the front?

Shaun – Fortunately, one thing that we didn’t face, although I know it’s an obstacle in many places, is processes related to how one gets permissions and access to the data. In fact, as the process has evolved, having those structures in place has made it really easy, so that if Cliff and I say “hey, we’d like to add this,” it’s a pretty easy amendment of the MOU [memorandum of understanding]. And then the people who deliver the data get approval and then they deliver it through a secure portal.

Cliff – I would also say that researchers left on their own probably would have had much less success in getting district participation in the survey study we did together. I, on the other hand, am someone with positional authority at the state level and established relationships that I can leverage to get that participation. And then I can pass it off to the research team that actually has the expertise and bandwidth to execute on the very labor-intensive data collection, both quantitatively and qualitatively.

It seems like you have a good partnership and a good synergy between the state office and the research team. If you were talking to CTE leaders and other researchers, what are some strategies and practices to make sure that partnership runs effectively and can be as impactful as possible?

Cliff – I think it’s important to have someone in the role of a researcher director type person whose job it is to facilitate these partnerships and to do some of the nitty gritty around data sharing, MOUs, etc. The other thing I would say is to have a commitment to an evidence base in terms of policymaking, and have people in the programmatic leadership who see the value of that and have enough knowledge of how research functions to parlay whatever policy or relational capital they have to support the research agenda.

Shaun – I think sometimes overcoming the incentives related to purely academic publishing restricts some of the willingness of some academic researchers to invest or to think about important questions in practice and policy. It’s being willing to realize that strong partnerships with local and state agencies means that more and better work can be done, and the work can have impact in real time. There is something very fulfilling and useful and practical about taking that approach from a research standpoint and then, if you come from practice like I did, then it helps ground the work.

Other blog posts in this series can be viewed here.

*Cliff explained that this is a formal process by which they solicited proposals for pay. “What’s been nice is that because it’s a partnership, Shaun has secured funding from other sources so there’s not an explicit contractual arrangement always. Aspects of the research that are ongoing are follow-ons from the original study. We have an interest in continuing to mine the data long-term to inform practice and policy.”

By admin in Research
Tags: Cliff Chuang, CTE Research, Data and Accountability, Equity and Access, Massachusetts, Shaun Dougherty

Posts Tagged ‘Data and Accountability’