P3 Report to Congress 2017

The Agencies have considered public input critical to both the development and the effective implementation of the P3 authority. Anticipating P3 authority, in June 2012, ED published a Request for Information on Disconnected Youth in the Federal Register (77 FR 32959) (RFI) seeking public input regarding the implementation of the potential P3 statutory authority, which was first proposed by in the FY 2013 budget request. The 171 comments ED received in response to the RFI emphasized the need for greater flexibility, highlighted promising initiatives, and offered recommendations for effectively serving disconnected youth and administering P3. The responses informed the Agencies’ thinking about how best to implement the P3 authority when it was enacted into law in FY 2014.

On April 28, 2014, the Agencies issued a consultation paper, Changing the Odds for Disconnected Youth: Initial Design Considerations for Performance Partnership Pilots (PDF, 19 pages), that provided background information about the pilot authority and described the Agencies’ initial thinking about the implementation of the authority.¹ The paper also encouraged stakeholders to respond by email to key questions about implementing P3 pilots. The Agencies used their research and analysis for the paper to guide two joint national webinars hosted by HHS on April 21 and 30, 2014, that provided information to the field and solicited feedback about the implementation of the authority.

In addition, on July 31, 2014, ED published a notice in the Federal Register that invited public comment on the application process for P3 (79 FR 44436). ED sought comments on the following three questions:

• What information, in addition to the information required by the Act, should entities be required to submit in their applications?
• What criteria should the Agencies use to evaluate applications?
• What technical assistance would be helpful to entities in preparing their applications?

ED received eight responses to the notice that made various recommendations with respect to the pilot application requirements and process. These recommendations informed the Agencies’ development of the application process. For example, one respondent recommended that applicants provide a logic model, while another recommended that the Agencies give priority to projects that are likely to achieve significant improvements in the outcomes of disconnected youth and projects that will replicate evidence-based approaches.

The Office of Management and Budget (OMB) designated ED as the lead Federal agency responsible in FY 2014 for soliciting applications and managing the P3 initiative on behalf of the Agencies.

ED published a notice inviting applications in the Federal Register on November 24, 2014 (79 FR 70033) (Round 1 notice) that described how pilots would be selected for the first P3 competition. The notice included three absolute priorities, which, for the purposes of the first round P3 competition, created separate categories for scoring and considering applications. The Agencies included these absolute priorities to test the theory of action underlying the P3 authority in a variety of settings. These absolute priorities were:

• Absolute Priority 1—Improving Outcomes for Disconnected Youth. Under this priority, we considered applications that proposed pilots that were designed to improve outcomes for disconnected youth who resided in communities that were not included in Absolute Priorities 2 and 3.
• Absolute Priority 2—Improving Outcomes for Disconnected Youth in Rural Communities. Under this priority, we considered applications that proposed to improve outcomes for disconnected youth in one or more rural communities only.
• Absolute Priority 3—Improving Outcomes for Disconnected Youth in Tribal Communities. Under this priority, we considered applications from a partnership that included one or more Indian tribes that proposed to improve outcomes for disconnected youth in one or more Indian tribes.

The Round 1 notice also included three competitive preference priorities that awarded additional points to applications that met certain requirements. The first two priorities awarded additional points to applicants that proposed to implement an independent evaluation of their projects, while the third awarded additional points linked to select local communities. Under Competitive Preference Priority 1, up to 5 additional points were available for applicants that proposed to carry out an independent evaluation that used a quasi-experimental design, while 10 additional points were available under Competitive Preference Priority 2 to applicants that proposed to carry out an independent evaluation that was a randomized controlled trial. Applicants’ evaluation proposals were scored based on the clarity and feasibility of the proposed evaluation design and the applicants’ demonstrated expertise in planning and conducting a study that used a quasi-experimental design or that was a randomized controlled trial. The third competitive preference priority awarded two points to projects that were designed to serve and coordinate with a Federally designated Promise Zone.

Additionally, the Agencies made available to the FY 2014 P3 pilots start-up grants of up to $700,000 each to support ongoing planning, streamlined governance, strengthened data infrastructure, improved coordination, and related activities to help pilots improve outcomes for disconnected youth.

Applicants were given 100 days to submit applications. To promote the opportunity and provide technical assistance to prospective applicants, the Agencies held two webinars that were hosted by DOL. The first webinar focused on the notice inviting applications, and the second provided technical assistance to prospective applicants that were considering proposing independent evaluations. These webinars were viewed collectively by 777 individuals. The Agencies also published answers to frequently asked questions about the notice and the pilot authority at www.youth.gov/P3 and updated the guidance as needed throughout the application period.

ED received 27 eligible applications in response to the Round 1 notice. Using the selection criteria published in the notice, Federal staff from the Agencies, as well as from the U.S. Department of Agriculture and the U.S. Department of Housing and Urban Development, with expertise in areas relevant to P3, such as dropout prevention, intervention, and re-engagement and improving employment outcomes for vulnerable youth and young adult populations, reviewed and scored these 27 applications. The selection criteria addressed: (1) the need for the project (up to 5 points); (2) the need for requested flexibility, including blending of funds and other waivers (up to 10 points); (3) project design (up to 25 points); (4) work plan and project management (up to 10 points); (5) partnership capacity (up to 15 points); (6) data capacity (up to 30 points); and (7) budget and budget narrative (up to 5 points). In addition, reviewers who had expertise in research and evaluation scored Competitive Preference Priorities 1 and 2, the two priorities for independent evaluations.

Following this review, the Agencies reviewed the top-ranked applications and the waivers they sought to determine whether they could be granted under the P3 authority or existing waiver authority, whether the inclusion of a program or particular waiver would be consistent with other significant legal or policy considerations, and whether the waiver was needed. To clarify their waiver requests, the top-ranked applicants were asked to respond in writing to Agency questions and to participate in one or more telephone conference calls. The Agencies selected nine applicants to be finalists for Round 1 pilots. On September 30, 2015, ED awarded $6,154,998 in start-up funds to these nine applicants, but conditioned their access to these funds on their signing a final performance agreement with the Agencies that administered programs included in the project. Abstracts of the nine projects appear in Appendix A.

The Agencies then proceeded to negotiate performance agreements with the nine entities. Section 526(c)(2) of Division H of the 2014 Act specifies the contents of these agreements. The 2014 Act indicated that each agreement must include:

• The length of the agreement, which could extend to September 30, 2018;
• The Federal programs and Federally funded services that are involved in the pilot;
• The Federal discretionary funds that are being used in the pilot and the period of availability for the obligation of such funds;
• The non‐Federal funds that are involved in the pilot, by source (which may include private funds as well as governmental funds) and by amount;
• The State, local, or tribal programs that are involved in the pilot;
• The populations to be served by the pilot;
• The cost‐effective Federal oversight procedures that will be used for the purpose of maintaining the necessary level of accountability for the use of the Federal discretionary funds;
• The cost‐effective State, local, or tribal oversight procedures that will be used for the purpose of maintaining the necessary level of accountability for the use of the Federal discretionary funds;
• The outcome (or outcomes) that the pilot is designed to achieve;
• The appropriate, reliable, and objective outcome‐measurement methodology that will be used to determine whether the pilot is achieving, and has achieved, specified outcomes;
• The statutory, regulatory, or administrative requirements related to Federal mandatory programs that are barriers to achieving improved outcomes of the pilot; and
• Criteria for determining when a pilot is not achieving the specified outcomes that it is designed to achieve and subsequent steps, including: (1) the consequences that will result; and (2) the corrective actions that will be taken in order to increase the likelihood that the pilot will achieve such specified outcomes.

The Consolidated and Further Continuing Appropriations Act, 2015 (Public Law 113-235) (the 2015 Act) extended the P3 authority to provide for up to 10 additional pilots and allow for the inclusion of certain funds from the Department of Justice. To implement this authority and solicit public comment, on October 22, 2015, ED published on behalf of the Agencies a notice of proposed priorities, requirements, definitions, and selection criteria (NPP) in the Federal Register (80 FR 63975). The priorities, requirements, definitions, and selection criteria proposed in this notice were based largely on the priorities, requirements, definitions, and selection criteria that were used in the November 2014 notice inviting applications for Round 1. For example, as in the November 2014 notice, the NPP proposed priorities for projects that serve disconnected youth in rural, tribal, and other communities. One of the lessons learned in Round 1 was that the use of absolute priorities promoted greater diversity in pilot settings than might otherwise have been achieved. In Round 1, the Agencies selected one project that proposed to serve a rural community (Eastern Kentucky Concentrated Employment Programs) and one project submitted by a partnership led by an Indian tribe that proposed to serve disconnected youth in an Indian tribe (Ysleta del Sur Pueblo). However, the Round 1 notice and the NPP documents differed in several other important respects:

• Additional Priorities for High-Need Subpopulations. The NPP also proposed additional priorities that focused on high-need subpopulations of disconnected youth, including priorities for disconnected youth who are unemployed and not enrolled in education, English learners, individuals with disabilities, homeless, in foster care, involved with the justice system, and refugees or other immigrants. In Round 1, the Agencies learned that State, local, and tribal governments may need additional incentives to focus their P3 projects on a variety of high-need subpopulations of youth. Most of the 27 applications that were submitted in Round 1 focused on disconnected youth who were at risk of dropping out of an educational institution. As a result, the Agencies proposed these additional priorities to create a means of providing incentives to applicants to serve the most vulnerable populations of disconnected youth.
• Priority for Work-Based Learning. In addition, the NPP proposed a priority for projects that provide paid work-based learning opportunities, including opportunities that are offered during the summer months, which are integrated with academic and technical instruction. As indicated in the NPP, the Agencies proposed this priority because the employment rate among youth has declined precipitously over the last decade, and addressing the employment needs of disconnected youth is critical to improving their well-being and preparation for lives as productive adults.
• A Single Priority for Evaluation. The NPP proposed to establish a single priority for projects that would support evaluations that use either a randomized controlled trial or a quasi-experimental design, rather than two separate priorities for these two kinds of evaluations as the notice inviting applications for the first Round of P3 had done. Applicants’ proposed evaluation plans would be evaluated based on the quality of the proposed evaluation’s design, appropriateness of the design to best capture key pilot outcomes, the scale of the contribution the evaluation will make to the evidence base, and the applicant’s expertise in planning and conducting comparable studies.
• Streamlined Application Requirements. To reduce burden on applicants and facilitate the review process, the NPP proposed to eliminate or streamline several of the application requirements. During their review of the applications in Round 1, the Agencies determined that some of the information applicants were required to submit could be streamlined or simplified to reduce burden on applicants, as well as to facilitate the timely review of applications by reviewers. Additionally, the NPP proposed to collect some of this information in table form to make clearer to applicants all of the data they must provide in their applications and to simplify how applicants provide these data.
• Simplified Selection Criteria. The NPP also proposed streamlined and simplified selection criteria to reduce burden on applicants, as well as to improve their focus on the factors that the Agencies consider to be most critical to the successful implementation of pilots.

Eleven parties submitted comments in response to the NPP. A key theme of the comments was concern that the FY 2014 selection process for pilots was too complicated. Generally, respondents encouraged the Agencies to streamline further the application requirements and simplify the review process. Some respondents encouraged the Agencies to direct or encourage applicants to focus on youth with the greatest needs and recommended additional priorities (e.g., applicants proposing to serve disconnected youth who are parents, applicants proposing to serve urban communities that have experienced violent protests) and supported additional priorities that were proposed (e.g., immigrant or refugees, English learners, homeless youth).

Other commenters were concerned that introducing new priorities in the competition under the FY 2015 authority would further complicate the process.

On behalf of the Agencies, ED published a notice of final priorities, requirements, definitions, and selection criteria (NFP) in the Federal Register on April 28, 2016 (81 FR 25339). In response to public comment, the NFP reduced burden on applicants by removing several application requirements that had been proposed in the NPP. The NFP also revised the priority for disconnected youth who are unemployed and out-of-school to limit it to those youth who face significant barriers to accessing education and employment. Additionally, the NFP revised the priorities for projects designed to improve outcomes for subpopulations of high-need disconnected youth (i.e., youth who are unemployed and out of school, youth who are English Learners (ELs), youth with a disability, homeless youth, youth in foster care, youth involved in the justice system, and youth who are immigrants or refugees) to specify that, in order to meet the priority, a project must serve the particular subpopulation identified in the priority and also be likely to result in significantly better educational or employment outcomes for the subpopulation. Finally, in response to a public comment, the NFP established an additional priority for projects that serve youth who are pregnant or parenting and that are likely to result in significantly better educational or employment outcomes for such youth.

On behalf of the Agencies, ED published a notice inviting applications (NIA) in the Federal Register on April 26, 2016, (81 FR 24573) for FY 2015 (Round 2 notice).
The Round 2 notice included four absolute priorities, which, for the purposes of the second round of P3 competition, created separate categories for scoring and considering applications:

• Absolute Priority 1—Improving Outcomes for Disconnected Youth. Under this priority, the Agencies considered applications that were designed to improve the outcomes of disconnected youth who reside in communities that are not included in Absolute Priorities 2, 3, or 4.
• Absolute Priority 2—Improving Outcomes for Disconnected Youth in Rural Communities. Under this priority, the Agencies considered applications that were designed to improve the outcomes of disconnected youth in one or more rural communities only.
• Absolute Priority 3—Improving Outcomes for Disconnected Youth in Tribal Communities. Under this priority, the Agencies considered applications from a partnership that included one or more Indian tribes that were designed to improve the outcomes of disconnected youth in one or more Indian tribes.
• Absolute Priority 4—Improving Outcomes for Disconnected Youth in Communities that Have Recently Experienced Civil Unrest. Consistent with language that Congress added in the Consolidated Appropriations Act, 2016 (Public Law 114-113) (the 2016 Act), under this priority, the Agencies considered applications that were designed to improve outcomes for disconnected youth in one or more communities that have recently experienced civil unrest. The notice indicated that, though the economy has recovered strongly in many places, many communities continue to struggle with high youth unemployment, low graduation rates, and crime. The notice stated that these and other continuing challenges can manifest in different instances of civil unrest, such as large protests or instances of civil disobedience, increases in self-directed or interpersonal violence in concentrated areas, or civic disorder prompted by a public health emergency.

The Round 2 notice also included four competitive preference priorities that allowed applicants to receive extra points for satisfying certain criteria:

• Competitive Preference Priority 1 awarded up to five additional points for projects that proposed to serve those disconnected youth who are neither employed nor enrolled in education and who also face significant barriers to accessing education and employment and that are likely to result in significantly better educational or employment outcomes for such youth.
• Competitive Preference Priority 2 awarded three additional points to projects that proposed to provide all disconnected youth it serves with paid work-based learning opportunities.
• Competitive Preference Priority 3 awarded two additional points to projects that were designed to serve and coordinate with a Federally designated Promise Zone.
• Competitive Preference Priority 4 awarded up to an additional 10 points to applications that proposed to conduct an independent evaluation of the impacts on disconnected youth of its overall program or specific components of its program that is a randomized controlled trial or a quasi-experimental design study.

Additionally, the Agencies announced that they would make available to the FY 2015 P3 pilots start-up grants of up to $350,000 to support ongoing planning, streamlined governance, strengthened data infrastructure, improved coordination, and related activities to help pilots improve outcomes for disconnected youth. The amount of the start-up grants was reduced based on competing priorities for agency funds and on the fact that one of the purposes of P3 is to focus on flexibility rather than new funding for the pilot sites. However, maintaining start-up grants was important based on feedback from the field in responses to the NPP that indicated that the start-up funding plays a critical role in launching and evaluating pilots.
Applicants were given 60 days to submit applications so that pilot finalists could be selected by the end of the fiscal year. To promote the opportunity and provide technical assistance to prospective applicants, the Agencies held a webinar on May 9, 2016, that was hosted by HHS. The webinar was viewed over 400 times during the first broadcast, and was available online for later viewing. The Agencies also published answers to frequently asked questions about the notice and the pilot authority at youth.gov/P3.

A number of applications received in response to the round 2 notice were not eligible because they did not meet all of the requirements in the notice, including the deadline for the submission of applications. To allow applicants additional time to prepare and submit their applications, ED reopened the competition on July 12, 2016, for one week to invite and accept additional applications. If applicants had submitted ineligible applications, they had time to address the issues that made their application ineligible and to submit a revised application.
In response to the notice reopening the competition, ED received three potentially eligible applications. Using the selection criteria published in the first notice, individuals with expertise in areas relevant to P3, such as dropout prevention, intervention, and re-engagement and improving employment outcomes for vulnerable youth and young adult populations, reviewed and scored these three applications. In addition, reviewers who had expertise in research and evaluation scored Competitive Preference Priority 4, the priority for independent evaluations.

The Agencies selected one applicant to be a finalist for designation as a Round 2 pilot, Maricopa County Education Service Agency in Phoenix, Arizona. On September 30, 2016, ED awarded $350,000 in start-up funds to this applicant but conditioned its access to these funds on its meeting certain conditions that would allow the agencies to grant the necessary waivers and its signing a final performance agreement with ED and DOL, the two agencies that administer programs that would be included in the project. An abstract that describes the proposed pilot appears in Appendix A.

The 2016 Act reauthorized P3, allowing for the creation of up to 10 additional pilots, as well as allowing for the inclusion of FY 2016 Homeless Assistance Grants administered by the Department of Housing and Urban Development (HUD). On behalf of the Agencies, ED published an NIA in the Federal Register on August 15, 2016, (81 FR 54056) for FY 2016 (Round 3 notice). This notice differed from the Round 2 notice in three respects:

• Homeless Assistance Programs. Consistent with section 242 of Division L of the 2016 Act, the Round 3 notice permitted applicants to include FY 2016 Homeless Assistance Grant programs administered by HUD. These are the Continuum of Care Program and the Emergency Solutions Grants Program.
• Deferred Application Requirements. The Round 2 notice required all applicants to include with their applications a memorandum of understanding or letter of commitment from each partner that described the terms and conditions of its participation in the pilot. All applicants for Round 2 also were required to include in their initial application an assurance that the State, local, or tribal government(s) with authority to grant any needed non-Federal flexibility, including waivers, has approved or will approve such flexibility within 60 days of an applicant’s designation as a pilot finalist; or include an assurance that non-Federal flexibility, including waivers, is not needed in order to successfully implement the pilot. To reduce burden on applicants, the Round 3 notice did not require applicants to include these documents in their applications. Instead, these documents were requested only from top-scoring applicants following the peer review of applications. Top-scoring applicants had 21 days after their notification by ED to submit these documents.
• Longer Application Period. The application period for Round 3 was 75 days, 15 days longer than the application period in the Round 2 notice. The deadline for applications was October 31, 2016.

As in Round 2, HHS hosted a webinar for prospective applicants on August 25, 2016. DOL, ED, and HHS staff participated in delivering the briefing. Eighty people viewed the live webinar, and a recording was viewed almost 600 times.

In response to the third notice inviting applications, ED received 15 potentially eligible applications. (An application is potentially eligible when the applicant meets the basic eligibility criteria but the application has not yet been reviewed for meeting all application requirements.) Using the selection criteria published in the first notice, individuals with expertise in areas relevant to P3, such as dropout prevention, intervention, and re-engagement and improving employment outcomes for vulnerable youth and young adult populations, reviewed and scored these applications. In addition, reviewers who had expertise in research and evaluation scored Competitive Preference Priority 4, the priority for independent evaluations.