Data and Accountability « Advance CTE Blog

COVID-19 Will Disrupt Early Postsecondary CTE Credit Attainment: Don’t Make Learners Bear the Burden

Monday, June 15th, 2020

The last weeks of March were a test of adaptability for the nation’s education institutions. Over the span of weeks, school districts, community/technical colleges and area technical centers had to amend remote learning policies amid unprecedented school closures. The rapid pace at which systems and institutions have adapted to the COVID-19 (Coronavirus) pandemic has been likened to turning a battleship at the pace of a speedboat.

Early Postsecondary Opportunities (EPSO) – which include dual enrollment, dual credit, concurrent enrollment and other related opportunities – play an important role in facilitating successful transitions between secondary and postsecondary education. According to the National Center for Education Statistics, approximately one-third of high school graduates take courses for postsecondary credit at some point during their educational careers. The integration of EPSO courses in Career Technical Education (CTE) pathways is creating options-rich, intuitive linkages between high school, college and career. This approach yields benefits to students, their families and their communities: earning postsecondary CTE credit in high school can lead to higher rates of college enrollment, persistence and success. But as schools, districts and colleges adapt to a new Coronavirus reality and a pervasive shift to online learning, where does that leave learners who are enrolled in CTE EPSO courses?

Coronavirus-related shutdowns put pressure on CTE EPSO courses in a number of ways:

Credit attainment and transfer: Many institutions have transitioned to binary (pass/ fail or satisfactory/unsatisfactory) grading for the spring and summer 2020 terms in recognition of the myriad of equity and logistical challenges presented by an abrupt, unexpected move to online education. This approach attempts to hold the learner harmless in weathering this dramatic shift in their education. Binary grading may have a wide-ranging impact on the acceptance of transfer credit of which EPSO credit is a subset. Like transfer credit, EPSO credit enjoys wide acceptance, often guaranteed by state law or system governance, but those policies were not developed to account for widespread use of binary grading.
Applied curriculum: Hands-on learning is a key feature of CTE, and without access to school buildings, learners may face challenges in demonstrating all of the skills and competencies necessary to receive credit for coursework. Even as many postsecondary courses transition to online and remote delivery, many practical CTE courses have been suspended altogether or significantly scaled back. Similarly, programs that integrate work-based learning activities as an important component of the curriculum are struggling to gain access to clinical hours, internships and other high-value college and career crossover activities. There is an opportunity to look at thoughtful approaches that extend deadlines for course completion, leveraging competency-based approaches, badging, certifications, portfolios and challenge exams to document the attainment of learning outcomes by students.
Funding: Funding for EPSOs varies from state to state, but in most cases tuition is paid through a combination of state funding, support from the learners’ home districts and from the learners themselves. Funding to decrease the cost of participation is an important equity lever for states looking to ensure they have a strong, robust talent pipeline to address their current and future workforce needs. What happens if a class is suspended mid-semester of if a learner withdraws due to circumstances related to the Coronavirus? States and institutions will need to clarify how to address these cases.
Data and accountability: More than a dozen states are measuring postsecondary credit attainment as their CTE program quality indicator for the Strengthening Career and Technical Education for the 21^st Century Act (Perkins V) performance accountability system. Enrollment disruptions in the spring and summer 2020 terms will impact states’ performance benchmarks, and if closures continue into the fall, this will impact their first year of Perkins V reporting. The Coronavirus also threatens to affect the other secondary CTE program quality indicators, as many learners earn recognized postsecondary credentials or complete work-based learning opportunities through partnerships with postsecondary institutions. EPSO programs are built on partnership and require mutual support, understanding and investment by school districts and postsecondary institutions. These partnerships create a better opportunity pivot in approach and resource sharing.

The field has been quick to recognize these challenges, but given the decentralized nature of higher education – in most cases, articulation agreements for CTE credit are negotiated at the local level between individual districts and partner colleges – the response has been inconsistent and incomplete. This situation creates both challenges and opportunities.

So how can states respond to some of these challenges? Next week the National Alliance of Concurrent Enrollment Partnerships (NACEP) and Advance CTE will explore some strategies to minimize the burden on students and honor their commitment to learning.

This blog post is the first in a two-part series about the impacts of the Coronavirus on CTE dual enrollment. It was written by Amy Williams, Executive Director of NACEP, and Austin Estes, Manager of Data & Research for Advance CTE.

By admin in COVID-19 and CTE
Tags: Concurrent Enrollment, Coronavirus, COVID-19, Credit Transfer, Data and Accountability, dual credit, dual enrollment, Early Postsecondary Opportunities, postsecondary

How to Seek Funding to Support CTE Research Partnerships

Thursday, May 28th, 2020

Over the past six months, Advance CTE and the Institute of Education Sciences (IES) have worked together to highlight the power of Career Technical Education (CTE) research partnerships in improving quality and equity in CTE. In Michigan, years of close collaboration between the Department of Education and the University of Michigan has enabled state leaders to address critical policy questions like choosing a secondary CTE program quality performance indicator. South Dakota leveraged relationships in the research community to improve data quality and foster a data-driven culture at the state level. And in Massachusetts, state leaders are working alongside long-time research partners to identify critical access and opportunity gaps and build solutions that enable equitable access to high-quality CTE.

Partnerships like these provide measurable benefits by allowing state policymakers to make informed decisions that impact learner success and bolster state talent pipelines – but they do come at a cost. The partnerships highlighted in this series were supported via a combination of state, federal and foundation funds. Research grant funds are most often used to cover personnel time for work on the research project, both at the university or research organization and at the partner education agency. As many of the state agency interviewees mentioned, it is difficult to carve time out of their regular responsibilities to work on a research project. By securing dedicated funding to cover part or all of a person’s salary, a state agency can afford to spend time on a research project. In addition, research grant funds can be used to provide incentives for students, teachers and schools to participate in a research study, for the development and administration of surveys or classroom observation tools (to complement information available in administrative data systems), and for software and hardware to analyze and house the data.

With growing public support for CTE, fueled by urgent needs for skilled labor, CTE programs will be called upon to do even more. States should be prepared with a research and evaluation strategy to determine whether and which strategies are most effective (and cost-effective). So how should states go about establishing and funding new CTE research partnerships?

Options for Financing State CTE Research Partnerships

There are a number of avenues states can take to finance CTE research. Federal sources of funding for CTE-related research include the Department of Education, the Department of Labor and the National Science Foundation. Education research funding may also be available at other agencies, such as the Department of Agriculture. Private funding for CTE research projects is also available from foundations such as the ECMC Foundation[i], the Smith Richardson Foundation, and the Laura and John Arnold Foundation.

The National Center for Education Research (NCER) at IES launched a special CTE topic in its Education Research Grants program in 2017 to encourage researchers to study CTE. Funded grants under this topic have examined CTE-related issues such as industry certifications, applied-STEM (Science, Technology, Engineering and Mathematics) CTE pathways and work-based learning. IES also funds CTE research under other programs and maintains a CTE Statistics webpage. In 2018, in partnership with the Office of Career, Technical, and Adult Education (OCTAE), IES formed a CTE Research Network to increase the amount and quality of causal research in CTE. CTE Research Network members have been studying the impact of various CTE programs and delivery models on student high school, postsecondary and labor market outcomes. The National Center for Special Education Research (NCSER) began funding CTE research for students with disabilities in 2019.

New Opportunity to Apply for Federal Funding to Study CTE!

There is good news for state leaders and researchers interested in initiating CTE research partnerships. NCER has just released its Fiscal Year 2021 Request for Applications (RFA) for its Education Research Grants Program (CFDA 84.305A). This grants program, one of several in NCER, was established in 2002 to produce research that is scientifically rigorous and relevant to the needs of education practitioners and decisionmakers. NCER welcomes CTE-related research proposals under the CTE topic or under other topics (such as STEM, Improving Education Systems, and Postsecondary and Adult Education). NCSER has a separate RFA for its special education research grants program (CFDA 84.324A) and welcomes applications to study CTE for students with disabilities.

Research grant applications are due at midnight (Eastern time) on August 20, 2020. Letters of intent (not required but encouraged) are due on June 11, 2020. Each of the open RFAs, as well as archived webinars for applicants about the IES grant process, are available on the IES funding opportunities page.

Applicants should start early to make sure they have everything they need. In addition to viewing on-demand webinars, applicants should be sure to read the RFA closely and pay attention not only to the application requirements but also to the IES recommendations for a strong application. For example, applicants should describe their theory of change and any prior research on the issue; align their research methods to the research questions; describe measures and data source; and make sure the sample size offers adequate statistical power. This grants program is very competitive, and peer reviewers will be paying attention to whether applicants follow the recommendations. Everyone involved in the submission process should also familiarize themselves with the IES submission guide, which details the steps necessary to successfully submit an application online.

We are eager to hear any and all ideas! Corinne Alfeld ([email protected]) and Austin Estes ([email protected]) would be happy to discuss them, and Corinne can also provide technical assistance in writing research grant application to IES. She can be reached by email to set up a phone call to discuss project ideas.

This final blog post wraps up our series aimed at increasing state CTE research partnerships by highlighting ways to seek research funding. Corinne Alfeld, Research Analyst at IES and Austin Estes, Manager of Data & Research at Advance CTE, collaborated to create this blog series in the hopes that more state agencies would partner with researchers to examine research questions related to CTE using state data.

[i] The ECMC Foundation is a funder of Advance CTE’s work.

By admin in Uncategorized
Tags: CTE Research, Data and Accountability, funding, Institute of Education Sciences, National Center for Education Research

Learning from CTE Research Partnerships: Using Data to Address Access and Equity Barriers in Massachusetts

Tuesday, March 17th, 2020

As part of our ongoing blog series aimed at increasing state research on Career Technical Education (CTE), Corinne Alfeld, Research Analyst at the Institute of Education Sciences (IES) and Austin Estes, Manager of Data & Research at Advance CTE, are conducting interviews with individuals who are part of successful CTE State Director research partnerships. The third interview was with Cliff Chuang at the Massachusetts Department of Education and Shaun Dougherty of Vanderbilt University. [Note: this interview, from February 5, 2020, has been edited for length and clarity].

Could you start by talking about the projects that you’ve worked on, your research questions, and how you settled on those research questions?

Shaun – It grew out of my dissertation work that was using some of the school data and then some of the statewide data from Massachusetts. It started pretty narrowly but the director of research was happy enough with what I was able to do that she talked about whether we could address some additional questions, and more data was becoming available. That more or less triggered the expansion, and then with Cliff coming into the role it became a two-way conversation that was more explicitly about what’s of academic interest and what’s of interest or of need on the practice and policy side for CVTE [career/vocational technical education].

Cliff – I would say that the particular catalyst for our most recent partnership is our desire as an agency to understand the waitlist demand issues related to chapter 74 CVTE in Massachusetts. If I recall correctly, we put out an RFR (request for responses)* for a research partner to help us analyze different aspects of who is and is not getting access to CVTE programs in Massachusetts. And Shaun and his partner Isabel at Harvard, a grad student there, their bid was selected. From that project there have been a lot of offshoots through the CTEx exchange collaboration that Shaun and others have established. We’ve been engaged in a lot of informal research inquiry as well as additional formal research that uses that data.

Could you talk a little bit about what the findings were from that project and what have been implications in the academic space but also on the policy front, how are you using those findings to change policy in Massachusetts?

Shaun – The basic findings were that in fact there is much more interest in these high-quality CTE programs, these chapter 74-approved programs in these standalone technical high schools, than can be met by current supply. This was more confirmatory evidence with a little more granularity and maybe confidence in the figures than was possible previously.

Cliff – Shaun’s team also helped us look at just the straight enrollment data comparisons, which is still not as ideal as looking at applicant data. It was helpful to have a more rigorous definition of what data protocols are needed around application and admissions. We have now made the decision to collect waitlist data systematically at the state level to allow researchers like Shaun to more rigorously analyze across the board the attributes of who’s interested in voc tech, who’s getting in, who’s applying, etc.

I think it also stimulated a variety of program initiatives on the part of state government in Massachusetts to increase access to CVTE programs through collaborative partnerships like After Dark, which is an initiative that seeks to utilize shop space in our technical schools after the regular school day paired with academics provided by a partner academic school to get more kids the technical training that we are unable to do in the standard day program structures.

I would also add that Shaun is continuing other aspects of the research now that we’re very excited about, based in part on some of the research they did do to look at longer term trends of students and their outcomes post high school.

Shaun – The first order concern is that lots of people want [access to CVTE programs] and there’s a limited amount of it, so should we have more?

The second order concern – but certainly not secondary question – is one about equity and whether or not the students who were applying and the students who were getting access look like a representative cross-section of the community at large. We know that students who choose CTE or select a lot of it are maybe different than those who don’t, but we don’t know a ton about whether and how we expect students who are making those investments to look like the overall population or whether or not access concerns lead to equity concerns.

Cliff – We would like to look more closely at whether the gaps are simply due to application gaps – which is still an issue in terms of kids not applying – or whether there are actual gaps related to who is applying and getting in. That was the data gap that we haven’t quite been able to close yet. But Shaun was able to create some comparative data that is just based on enrollment that has allowed us to engage in these conversations. We’re having the conversation about trying to expand the number of seats available so there’s less of a waitlist, but also to ensure that access into the existing seats is equitable and doesn’t disadvantage certain subgroups over others.

Over the course of the partnership, what have been some of the major challenges and hurdles that you’ve faced? What are some of the speedbumps that you’ve hit getting things formalized up at the front?

Shaun – Fortunately, one thing that we didn’t face, although I know it’s an obstacle in many places, is processes related to how one gets permissions and access to the data. In fact, as the process has evolved, having those structures in place has made it really easy, so that if Cliff and I say “hey, we’d like to add this,” it’s a pretty easy amendment of the MOU [memorandum of understanding]. And then the people who deliver the data get approval and then they deliver it through a secure portal.

Cliff – I would also say that researchers left on their own probably would have had much less success in getting district participation in the survey study we did together. I, on the other hand, am someone with positional authority at the state level and established relationships that I can leverage to get that participation. And then I can pass it off to the research team that actually has the expertise and bandwidth to execute on the very labor-intensive data collection, both quantitatively and qualitatively.

It seems like you have a good partnership and a good synergy between the state office and the research team. If you were talking to CTE leaders and other researchers, what are some strategies and practices to make sure that partnership runs effectively and can be as impactful as possible?

Cliff – I think it’s important to have someone in the role of a researcher director type person whose job it is to facilitate these partnerships and to do some of the nitty gritty around data sharing, MOUs, etc. The other thing I would say is to have a commitment to an evidence base in terms of policymaking, and have people in the programmatic leadership who see the value of that and have enough knowledge of how research functions to parlay whatever policy or relational capital they have to support the research agenda.

Shaun – I think sometimes overcoming the incentives related to purely academic publishing restricts some of the willingness of some academic researchers to invest or to think about important questions in practice and policy. It’s being willing to realize that strong partnerships with local and state agencies means that more and better work can be done, and the work can have impact in real time. There is something very fulfilling and useful and practical about taking that approach from a research standpoint and then, if you come from practice like I did, then it helps ground the work.

Other blog posts in this series can be viewed here.

*Cliff explained that this is a formal process by which they solicited proposals for pay. “What’s been nice is that because it’s a partnership, Shaun has secured funding from other sources so there’s not an explicit contractual arrangement always. Aspects of the research that are ongoing are follow-ons from the original study. We have an interest in continuing to mine the data long-term to inform practice and policy.”

By admin in Research
Tags: Cliff Chuang, CTE Research, Data and Accountability, Equity and Access, Massachusetts, Shaun Dougherty

Learning from CTE Research Partnerships: Building a Collaborative Data Culture in South Dakota

Tuesday, February 11th, 2020

As part of our ongoing blog series aimed at increasing state research on Career Technical Education (CTE), Austin Estes, Senior Policy Associate at Advance CTE, and Corinne Alfeld, Research Analyst at the Institute of Education Sciences (IES), are conducting interviews with individuals who are part of successful CTE State Director research partnerships. The second interview was with Laura Scheibe of the South Dakota Department of Education and Marc Brodersen of REL Central at Marzano Research. [Note: this interview has been edited for length; you can find the full interview transcript here].

Could you both talk about the project(s) that you have worked on and your research questions? How did the relationship start, and who approached whom?

Marc – When we were doing needs sensing with the states in our region, particularly with South Dakota, CTE emerged as a pretty high priority area. We needed to determine what the research questions were, what questions we could actually address, and what data were available that could be used in those research projects. So, this work started off as a technical assistance project where we were working with South Dakota pretty closely and getting all of the relevant players around a table and going through and mapping their data. And it was quite a long process.

Laura – There’s huge support in South Dakota behind CTE, but there wasn’t state-level evidence behind why CTE is such a good thing for students. So, the value that Marzano provided to the project in helping us walk through “this is the data that can help you, this is the process that we are going to go through to help you get to the answer” has been incredibly helpful and not something that we, as a pretty small department of education, could ever have undertaken on our own.

Can you talk about what research questions you ultimately came to and where you are in the process of answering those?

Marc –We have three main questions: 1) What is the impact of being a CTE concentrator on high school graduation, two- and five-year postsecondary enrollment, and completion status? 2) What is the impact of being a CTE concentrator on two-year and five-year employment and quarterly wage status? 3) How do the two-year and five-year outcomes vary by the various CTE Career Clusters^®?

Connecting education to workforce data is really difficult, and we’re talking about collecting data over a five to 10 year span for an individual student. Many state data systems don’t go back that far, or data systems have changed, so it’s difficult trying to identify one data system that has 10 or more years of data for an individual student. We’re making it work, but it takes some time and some finagling. We haven’t even begun to analyze the data so, unfortunately, we can’t talk about any preliminary findings.

What were some of the early roadblocks in building this relationship and starting to examine and compile some of the data?

Laura – One of the roadblocks was just getting everyone around the table and bought into the idea. We’re a fairly small state, so it wasn’t hard to reach out to my counterparts at the other agencies who would need to be involved, but this project was, and continues to be, something that is on top of the day to day work that we do. It’s not driven by any specific policy initiative but rather by everybody around the table acknowledging and recognizing that “yeah, this would be really useful for us.” But, in that sense, it’s hard to get everyone’s commitments to the time it has taken and takes to pull this off and making sure that we’ve got the right people around the room as well. We’ve involved not just the Board of Regents but the technical college system and the people with the workforce data.

Marc – Having somebody at the policy level, the data level and the leadership level in the room at the same time is almost essential, particularly when you’re at the brainstorming phase. You can have the leadership that’s going to say “yes, this is important, and I want you to devote time to this,” and then the data person is saying “well, that data just doesn’t exist,” and the policy person may not know about that piece. And having all three of those perspectives at the same time can save a lot of time and effort.

How do you plan to use this research project to further policy in South Dakota?

Laura – First and foremost, this particular project is demonstrating the value that CTE has to the secondary students. This project pre-dated Perkins V [the Strengthening Career and Technical Education for the 21st Century Act], but as we’re moving into implementation of Perkins V full force in the coming calendar year, with the new requirements that Perkins places on states — and therefore on schools — to be an approved program, we’re seeing school districts question if it’s really worth it. This project is really coming in at a good time where we will hopefully have some data where we can say, “yes, CTE is worth it.” Being able to message that is hugely valuable from the perspective of a CTE Director in a state where almost every single public school district runs an approved program. Now that we’ve got Perkins V and the [comprehensive local] needs assessment, it will be just one more bit of evidence for schools to be able to examine whether they’re providing the best opportunities for our kids.

What advice would you give to other researcher/ State Director partners for conducting CTE research or establishing similar partnerships?

Marc – From my perspective, as far as establishing a partnership, I think face-to-face interactions are invaluable. It takes a while to trust each other or establish a positive working relationship.

Laura – My advice to State Directors would be to really plan for it and make it a priority. And don’t make it something that isn’t part of the day-to-day because then I think the thread can get lost. I would also say getting that higher-level buy-in is really important. It’s important to make sure that you’ve got that policy-level partner to keep things moving along. The benefits will be there in the end, it just has to be woven into the day-to-day of what you’re doing in order to make it all come together.

Going through this process has helped me form partnerships with my colleagues in other agencies even more strongly than I had before. Just the exercise of having gone through all of that and understanding their work and their data and everything they do, having them understand my role and my constraints better, has just made us a more effective CTE/ workforce team in our state. As we move forward with Perkins V and WIOA [Workforce Innovation and Opportunity Act] state plans and all of this other stuff coming, it just benefits us and enables us to work more effectively and work faster now that we have those strong relationships. They were there before, but they’re definitely stronger now as a result of this project.

Marc – One of the things I thought was really neat was getting all these folks together and thinking deeply about data. It might not be the most exciting topic for a lot of folks, but going through the process gave everyone a better understanding of what they can do and how they might be able to work with others. And I think in the day-to-day, not everyone spends that much time thinking at the data and variable level. But doing that will increase everyone’s capacity to be able to do this kind of work moving forward.

One other thing to add just as a side note. Throughout this process, we also collaborated with Nancy Copa at the Common Education Data Standards (CEDS) when we were doing the data mapping piece. We did not officially map the South Dakota data to that, but we used the CEDS as kind of a template to provide us with a common dictionary to have these conversations across departments. And that was really useful. In fact, all of us – the different departments in South Dakota and the CEDS folks –co-presented at the last STATS-DC conference, which I thought was a very positive experience.

The full transcript of this interview can be accessed on Advance CTE’s website. Other blog posts in this series can be viewed here.

By admin in Uncategorized
Tags: CTE Research, Data and Accountability, Marzano Research, REL Central, South Dakota, systems alignment

Learning from CTE Research Partnerships: How Michigan Built Trust with Researchers to Better Understand State Data

Tuesday, January 21st, 2020

As part of our ongoing blog series aimed at increasing state research on career and technical education (CTE), Austin Estes, Senior Policy Associate at Advance CTE, and Corinne Alfeld, Research Analyst at the Institute of Education Sciences (IES), are conducting interviews with individuals who are part of successful CTE State Director research partnerships. The first interview was with Jill Kroll of the Michigan Department of Education and Dan Kreisman of Georgia State University (and Director of CTEx). [Note: this interview has been edited for length; you can find the full interview transcript here].

The first question we have is about the projects that you work on together: what were some of the research questions you came up with, and how did you come to settle on those research questions?

Jill – I first connected with Dan and with Brian Jacob at University of Michigan when I saw Brian present to our P-20 council about some research that he was doing connecting the wage record data for five community colleges. I was like “Gee, is there any way you can do something similar with the statewide secondary student data?” And he said it was possible. So I worked within our department procedures to find out how we could go about establishing a relationship that would allow this opportunity.

Dan – That led to a whole bunch of other discussions of things that we thought were interesting. So, to say that there is a set of research questions is not the way I view our relationship. We talk with folks in Jill’s office regularly to hear what questions are pressing for them, and then we try to help facilitate answering those and then see where those lead us. I think one of the important things is we try to think about where there are policy levers, so we want to say “If we answer this question, how can the state or the districts use that information to further their mission of providing CTE programming to students in Michigan?”

Jill – I’ve been really happy with the extent to which Dan and the research team have consistently focused on the “so what?” Rather than focusing on vague research questions of interest only to other researchers, they have emphasized their interest in doing research that has practical application, that can be used by educators in the field.

Could you share an example of how you’ve been able to use some of this evidence and research to change policy, or at least to shape your understanding on some decisions that you’re making at the state level?

Jill – When we were starting to work on our Perkins V [the Strengthening Career and Technical Education for the 21st Century Act] state plan, we had a short time to determine what we wanted to consider for our secondary indicator of program quality. Because Brian, Dan, and their students had been working with this data for so many years, they had the capacity to very quickly do the matching and come up with an approximation for us about what postsecondary credit attainment would look like, and what strengths and weaknesses they saw in the data. It would have been really difficult for our office, or even multiple state agencies, to have been able to work that quickly and give it the critical analysis that they did.

The other thing they did when we were making the decision for that indicator is look at the data that we had for work-based learning and tell us what could be done with it. What came out of that was that the data was not in any form that could be analyzed (text and PDFs). This was really revealing to our State Director Brian Pyles, and it led him to set a policy that we are going to build a consistent way of collecting data on work-based learning. So that is another piece where it influenced practice and policy. One of the most exciting and valuable things that I find about the partnership is that Dan and the other researchers have a lot more capacity to analyze the data in a way that we just don’t have the time to do. Sometimes we don’t have the expertise, and sometimes we just don’t look at the data in the same way.

Dan –And there’s a flip side that without their input, we often are looking at data and can’t make heads or tails of something. And we can get on the phone or write an email to someone over there and say “Hey we’re seeing this thing. Can you tell me what that means?” And they will come back with “Oh, the system changed” or “There was this one policy,” and “Here’s what you have to do to make it fit everything else.” And this happens all the time. We would be completely lost without this open channel that we have to their office.

I think it’s important not to dismiss the power of good descriptive work. Lots of times, the questions that states are grappling with can often be illuminated with some really careful and good descriptive work. You can say, “This is what we’re seeing, this is the big picture,” if you step back for a minute, and that information lots of times has been as valuable as the stuff we try to do that is more causally oriented in our research.

Jill – I agree, and I want to follow up on the whole issue of how important trust is. I cannot emphasize enough how important it is to me that Dan and the other researchers come to us with those questions, that they check in with us. That’s absolutely critical. Anyone who works with any kind of data knows that it’s just so complex. If you link tables wrong, or misunderstand a data field, you can come to a completely wrong decision. So that communication and that interaction and trust are key to accurate outcomes.

As you’re both looking ahead, what’s next on the agenda? What are some of the research questions and priorities you have for this partnership?

Dan – Number one is tracking students into the labor market. That’s our biggest and most outstanding question. And the degree to which CTE programs are preparing students for college and the labor market and careers. In terms of other projects, one of the things we’re interested in is technical assessments. We’re also part of a consortium of several states – that’s the CTEx group. We meet annually together, and that allows us to harmonize things across states to see how trends are similar, how enrollment rates work, all sorts of different questions across multiple states.

Jill – One of the things we’re talking about right now is that we don’t have, in an accessible form, data on access to a particular program. We know that career centers serve certain districts, but if someone asked, “If student A is going to Central High School, what programs do they have access to?” we don’t have a good way of answering that at the moment. We’ve had a couple of discussions about how we can work together to build basically a dataset that clarifies that. That would be mutually beneficial and would take resources from both in order to do something like that.

Thinking back on this partnership, is there any advice you would give to other State Directors or CTE researchers?

Dan – Building a strong relationship is the first thing you have to do. And part of that is spending time face to face talking about questions, moving around ideas, looking at data together. We had the benefit of a long windup period. We spent at least a year just talking about questions and putting together data before we even started doing any analyses. We also had buy-in from Jill’s office up and down the line from folks who were doing the research to people who were in policymaking roles. And without all of that, none of this would even have been possible.

And the second part is to not downplay the value of just providing good information. A lot of us on the research side don’t realize how little time folks in the state offices have to take a step back and say, “What’s going on with our data? Let’s look at the big picture.” And one of the things we can provide them is just giving them that big picture and handing it to them in a digestible way. And doing that is the first step, is a really good way to start building that trust. They really see the value of what you can do early on. And then you can start to get into more difficult or longer-term questions.

Jill – The first advice I would give is: Do it! Partner with researchers. I can’t say enough positive about it. The second is: Follow department procedures and be transparent with department leadership. You know that windup might be really, really slow while you jog through the channels that you need to in your department to do things by the book, but I think it pays off in the long run.

My third one is: Be transparent and open with school districts. Share what you’re doing and invite their input. Anybody who works with state data would probably know, you’re always a little hesitant about what the public would think about this use of data. The way that Dan and the postdocs and graduate students have openly shared the work that they’ve done with our CTE administrators has really helped, in that I have not gotten any doubt from districts.

The full transcript can be accessed on Advance CTE’s website. Other blog posts in this series can be viewed here.

By admin in Uncategorized
Tags: CTE Research, Data and Accountability, Michigan, systems alignment

Strengthening Career Readiness Systems through New Skills for Youth: A Look Back at States’ Impact

Wednesday, December 11th, 2019

Under Kentucky’s new program approval and review process, schools and districts can use state and federal funding to support career pathways only if their programs are aligned with priority industries or top occupations. This is just one of the strategies Kentucky used under the New Skills for Youth (NSFY) initiative to transform and phase out virtually every career pathway that was not well aligned with labor market demand.

From 2016 through 2019, Kentucky and nine other states in the NSFY initiative received $2 million and hands-on technical assistance and coaching to strengthen their career readiness systems. As part of the NSFY initiative, a $75 million national initiative developed by JPMorgan Chase & Co, the Council of Chief State School Officers, Advance CTE and Education Strategy Group worked with states to improve their career readiness systems.

Through NSFY, Delaware, Kentucky, Louisiana, Massachusetts, Nevada, Ohio, Oklahoma, Rhode Island, Tennessee and Wisconsin took action to:

Develop and scale high-quality career pathways: Massachusetts designed and launched a new initiative to expand access to high-quality college and career pathways (HQCCPs). HQCCPs include Innovation Pathways, which connect student learning to broadly defined, in-demand industry sectors, and Early College programs, which enable students to earn at least 12 college credits in high school.
Expand access to work-based learning opportunities: Louisiana piloted the Building Employment Skills for Tomorrow program in Bossier Parish in 2018 to connect learners with disabilities to work-based learning opportunities, equip them with real-world skills through training, and provide mentorship to program participants.
Strengthen data and accountability to incentivize career readiness: Ohio’s school report cards include a summative score for the Career & Postsecondary Readiness indicator and break down how learners achieved readiness by reporting the percentage of learners that earned industry-recognized credentials, completed dual enrollment, completed a pre-apprenticeship and more.
Lay the foundation for sustaining career readiness efforts: In 2017, Nevada, with support from the Nevada Department of Education, the governor and the Legislature, enacted six major policies to lay the foundation for a statewide career readiness system.

The impact of these states across the entire initiative is highlighted in the NSFY Impact Snapshots and NSFY Impact Summary, which examines the state role in catalyzing and transforming career readiness opportunities for youth.

Through NSFY, 10 states demonstrated the importance of strong state leadership to advance career readiness by setting a clear vision and agenda, catalyzing and scaling pathways and work-based learning, and ensuring access and equity in career readiness opportunities. As a result, the impact of the states was far-reaching. For instance, under NSFY Delaware was able to develop 19 career pathway programs in high-demand occupations and Tennessee was able to ensure that 100 percent of high school students have access to at least four early postsecondary courses.

To learn more about the work states completed under the NSFY initiative, register for Advance CTE’s A Look Back at States’ Impact through the New Skills for Youth Initiative webinar, which will take place on December 12 from 1-2 p.m. EST, and download the NSFY Impact Snapshots here.

Brianna McCain, Policy Associate

By admin in Publications
Tags: career pathways, Data and Accountability, Delaware, Kentucky, Louisiana, Massachusetts, Nevada, New Skills for Youth, NSFY, Ohio, Oklahoma, rhode island, sustainability, tennessee, wisconsin, work-based learning

New Report on The Role of Data and Accountability in Growing Youth Apprenticeship Programs

Tuesday, November 5th, 2019

Youth apprenticeship programs can give students access to valuable work-based learning experiences that provide insights into how their interest can connect to education and the workforce. Although these programs are often beneficial for participants, there is little data to show the programmatic landscape and impact. A lack of data inhibits the development and expansion of youth apprenticeship programs.

Advance CTE’s latest report, The Role of Data and Accountability in Growing Youth Apprenticeship Programs, explores how states are collecting data on youth apprenticeship programs, and what steps can be taken to collect high-quality enrollment and outcomes data.

There are many challenges that inhibit the ability of state and local communities to collect and use reliable data to support and improve youth apprenticeship programs. However, there are steps that state and local leaders can take to strengthen data collection and analysis, including:

Adopt a common, rigorous and measurable statewide definition of youth apprenticeship programs;
Develop a data definition and business rules that align with the statewide definition of youth apprenticeship to support consistent data collection and reporting;
Learn what data collection looks like at the local and state levels and where gaps exist; and
Leverage the opportunities provided by ESSA and Perkins V to implement data collection mechanisms.

There are also qualitative methods that can be used to support findings and build a comprehensive understanding of youth apprenticeship programs. This report explores how qualitative data supports findings, and supplements gaps in data.

This report was made possible by the Partnership to Advance Youth Apprenticeship (PAYA). PAYA was created by New America and includes Advance CTE, CareerWise Colorado, Charleston Regional Youth Apprenticeships at Trident Technical College, Education Strategy Group, JFF, The National Alliance for Partnerships in Equity, the National Fund for Workforce Solutions and National Governors Association. PAYA is appreciative of the support of the Annie E. Casey Foundation, the Ballmer Group, Bloomberg Philanthropies, the Joyce Foundation, JPMorgan Chase & Co. and the Siemens Foundation.

Meredith Hills, Policy Associate

By admin in Uncategorized
Tags: Data and Accountability, PAYA, youth apprenticeship

CTE’s Equity Challenge

Thursday, September 20th, 2018

Throughout history, and continuing today, learners of color, low-income learners, female learners and learners with disabilities have been historically tracked into terminal vocational programs leading to jobs with uncertain promise of economic growth and prosperity. While the quality of CTE programs has significantly improved since then, many of these same learners cannot access high-quality CTE programs of study that prepare them for success in postsecondary education and their future careers.

To help state leaders recognize these historical barriers and adopt promising solutions to close equity gaps in CTE, Advance CTE is launching a new series of policy briefs called Making Good on the Promise. The first two briefs are now available in the Learning that Works Resource Center.

The first brief explores CTE’s history, taking a close look at the practice of tracking learners into low-quality vocational programs and examining the different ways that certain learners have faced barriers to accessing high-quality CTE programs of study. The second brief highlights promising practices from states that are using data to identify and address access and achievement gaps by different learner populations.

Ultimately, each learner deserves to access a learning environment in which he or she is supported, feels welcome, and can acquire the knowledge, skills and abilities to achieve lifelong career success. But many of the structures and systems in place today enforce historical biases and discrimination that make it challenging for learners to access these opportunities. Reversing historical trends and expanding access and opportunity for each learner will require tough conversations, humility, and a commitment to both quality and equity.

In Delaware, for example, state leaders made a commitment to use state CTE data to expand equitable access to high-quality CTE programs. Through the regular CTE performance management process, the Delaware Department of Education (DDOE) compares the population of learners in CTE programs to learners in the larger student body to identify enrollment gaps.

If a certain learner population is either underrepresented or overrepresented in the program of study, it triggers a structured protocol. DDOE staff work in partnership with local leaders to conduct interviews with teachers, learners and parents and dig deeper into the root causes. DDOE and district staff debrief about the conversations and collectively develop a report summarizing the findings of the study. Although local sites are not required to act on DDOE’s recommendations, many recognize the need and seize the opportunity for additional state support.

Conversations about equity are often difficult, but they are necessary to secure access and opportunity for each learner. Collaborative, data-driven strategies like Delaware’s CTE performance management protocol allow state leaders to identify and address inequities in an impactful way.

In future briefs, we will explore how state leaders can work to rebuild trust among communities that have been historically under-served, expand opportunity for every learner, and put mechanisms in place to ensure learner success. The Making Good on the Promise series is made possible through the New Skills for Youth initiative, a partnership of the Council of Chief State School Officers, Advance CTE and Education Strategy Group, generously funded by JPMorgan Chase & Co. These briefs and all future resources can be accessed in the Learning that Works Resource Center at careertech.org/resource/series/making-good-promise.

Austin Estes, Senior Policy Associate

By admin in Advance CTE Resources, Publications, Resources
Tags: Data and Accountability, Delaware, Equity and Access, Tracking

DC, Texas Improve Data Systems; Colorado, Ohio’s Community Colleges Offer Bachelor’s Degrees

Tuesday, April 17th, 2018

As the legislative session moves forward, many states have passed bills that will help to improve data systems and expand opportunities for learners.

Data System Improvements

Recently, data system improvements have been a focus of policy efforts in order to better support and hold accountable districts, institutions and programs, as well as allow learners, employers and policymakers to stay informed.

In the District of Columbia, the Council of the District of Columbia passed the Workforce Development Systems Transparency Act, which requires the District’s Workforce Investment Council to detail the District’s spending on adult education programs and workforce development education programs, as well as the performance outcomes of those programs, in a public report. The performance outcomes information will include employment rates, median earnings, credential attainment, and completion rates. The first version of the report will include information about programs managed by seven DC entities, such as the Department of Employment Resources, and by 2020 programs administered by an additional 14 entities will be included in the report.

In Texas, the University of Texas System launched an updated version of the database Seek UT to include University of Texas graduates’ earnings in the hopes of showing the benefits of higher education. The database utilizes Census Bureau data and provides information on student’s median incomes for every program offered after one, five, and ten years after graduating, the percentage of students who went on to continue their education and the median loan debt for different programs. The database is viewed as a “work-around” of the current ban on a federal database that would link student-level education data to national employment data.

Community Colleges Offer Bachelor’s Degrees

Elsewhere, states are passing laws to expand community college offerings and to address the shortage of skilled employees.

In Colorado, a bill that allows Colorado’s community colleges to offer bachelor’s degrees in nursing recently became law. The bill was passed without the governor’s signature or veto. In a letter explaining this decision, Governor Hickenlooper cited concerns over limited stakeholder engagement by the bill’s proponents and potential conflicts between the various agencies that oversee higher education in the state.

In response to these concerns, the letter directs the Colorado Commission on Higher Education (CCHE) to convene stakeholders to determine how to best align programs with industry trends. This law was allowed to pass in response to a severe shortage of nurses in Colorado and after reports that more nursing disciplines require a masters or doctoral degree than in previous years.

Similarly, in Ohio, three community colleges received state approval to offer bachelor’s degrees in microelectronic manufacturing, aviation, unmanned aerial systems, land surveying and culinary and food science. These programs still need to receive accreditation from the Higher Learning Commission before the community colleges can offer the degrees.

Once accredited, these programs will help to achieve Ohio Governor Kasich’s goal to have 65 percent of the state’s workforce earn an industry recognized credential or degree by 2025. Governor Kasich has already showcased his support for community colleges to offer baccalaureates through the introduction and passage of legislation that supports this.

Brianna McCain, Policy Associate

By admin in Public Policy, Uncategorized
Tags: Colorado, community colleges, Data and Accountability, District of Columbia, Ohio, texas

States Can Strengthen Career Readiness Under ESSA; Will Round Two States Seize the Opportunity?

Wednesday, September 20th, 2017

Nearly two years of planning came to a head on Monday as states hit the second plan submission window under the Every Student Succeeds Act (ESSA). Under the law, states are required to engage in stakeholder consultation and develop a comprehensive, statewide strategy for spending and holding districts and schools accountable for billions of dollars in K-12 federal education funding.

Secretary Betsy DeVos’s Education Department is now responsible for reviewing and approving state implementation plans. However, review of the first round of state plans is still ongoing and four round two states have been granted an extension due to this year’s catastrophic hurricane season. Since the initial submission window in April of this year, Sec. DeVos has publicly approved 14 out of 17 plans, with Michigan, Massachusetts and Colorado still waiting for approval. The department is expected to approve those plans shortly. Among the states scheduled to submit plans this week, Texas, Florida, Alabama and South Carolina — each still reeling from the effects of Hurricanes Harvey and Irma — have been permitted to submit their pans later this fall.

Not all state plans have gone without controversy either. The extensive plan development process surfaced differences of opinion between various policymakers and stakeholders. In a few cases, state governors refused to sign their state plans before they were submitted to the U.S. Department of Education. While states are required to provide the governor 30 days to review and comment on the plan, ESSA does not explicitly require the governor’s approval.

Career Readiness in ESSA

As Advance CTE has discussed at length, there are several leverage points within ESSA — most of them new in this version of the law — that policymakers can use to drive career readiness in their states. The primary leverage points include:

Vision and Goal Setting: ESSA requires states to set out long-term goals for academic achievement, but states can go beyond the statute by setting ambitious targets for credential attainment, career readiness and successful postsecondary transitions. Articulating clear goals helps anchor the plan and ensure all strategies are aligned to serve the same end.
Accountability (Title I): Accountability is the most direct way states can incentivize career readiness through ESSA. States can use the “School Quality and Student Success” indicator — also known as the fifth indicator — to measure how well schools and local education agencies are preparing learners for successful transitions to college and careers.
Supporting Effective Instruction (Title II, Part A): Funds under Title II, Part A are often sub granted to local education agencies to provide professional development and support class size reduction strategies. However, ESSA allows states to set aside funds to train teachers and school leaders on strategies for integrating CTE and academic content as well as interpreting labor market information.
Student Support and Academic Enrichment Grants (Title IV, Part A): New under ESSA is the Title IV, Part A block grant, which consolidated categorically funded programs under the No Child Left Behind Act to give states and local school districts more agency to support student needs. Under the statute, grant funds may be used to support a well-rounded education, which now includes CTE.
21st Century Community Learning Centers (Title IV, Part B): 21st Century Community Learning Centers are designed to supplement the K-12 experience through non-school hours. They are required to support programs and activities related to a well-rounded education, including CTE. In the first round, a number of states chose to prioritize programs providing work-based learning, CTE and other career preparation activities.

While 11 of the first 17 submitted state plans included (or plan to include) a career-focused measure in their high school accountability rating systems, states overall missed the opportunity to fully leverage ESSA to maximize career readiness. Only five states describes specific state-level activities to support career readiness, STEM (Science, Technology, Engineering & Mathematics) and/or dual enrollment under Title IV, Part A. And only two states identified opportunities to support blended academic and technical professional development under Title II, Part B.

Nevertheless, there is still flexibility for state and local leaders to go beyond what is specifically laid out in their ESSA plans to adopt career readiness strategies and prepare learners for post-high school transitions. Local education agencies must develop their own strategic plans for using ESSA funds. Depending on their local context, school districts may elect to prioritize work-based learning, credential attainment and CTE programs of study.

In the coming months, Advance CTE will review the 34 remaining ESSA plans to determine where and how states are making connections between ESSA and career readiness. This analysis, to be released later this year, will update this summer’s “Mapping Career Readiness in State ESSA Plans” report.

Austin Estes, Policy Associate

By admin in Public Policy
Tags: Data and Accountability, ESSA, Every Student Succeeds Act

Posts Tagged ‘Data and Accountability’