Round 3 NIA Frequently Asked Questions

The Round 2 and Round 3 notices differ in three important respects:

• In Round 3, applicants may include FY 2016 Homeless Assistance Grant programs administered by the United States Department of Housing and Urban Development (HUD). This was not authorized under the Round 2 authority.
• The Round 2 notice required all applicants to include with their applications a memorandum of understanding or letter of commitment that is described in Application Requirement (e)(2). All applicants also were required to include in their initial application an assurance that the State, local, or tribal government(s) with authority to grant any needed non-Federal flexibility, including waivers, has approved or will approve such flexibility within 60 days of an applicant’s designation as a pilot finalist or that non-Federal flexibility, including waivers, is not needed in order to successfully implement the pilot. (Application Requirement (c)(2)). To reduce burden on applicants, the Round 3 notice does not require applicants to include these documents in their applications. Instead, these documents will be requested only from top-scoring applicants following the peer review of applications. Top-scoring applicants will have 21 days after their notification by the U.S. Department of Education (ED) to submit these documents to ED.
• The Round 3 notice includes an additional invitational priority, Improving Outcomes for Youth in Foster Care. This priority invites applicants to propose a pilot that — (1) will serve disconnected youth who are or have ever been in foster care; and (2) is likely to result in significantly better educational or employment outcomes for such youth. In this competition, invitational priorities express the Agencies’ interest in applications that will improve outcomes for particular subpopulations of disconnected youth. They do not provide a competitive or absolute preference over other applications. See FAQ M-1.

To be eligible to apply for P3, the lead applicant organization must be a recognized entity of either a State (as defined at 2 CFR 200.90), local government (as defined at 2 CFR 200.64), or tribal government (2 CFR 200.54 provides a definition of “federally recognized Indian tribe,” and, as stated in FAQ B-3, State-recognized tribes are eligible to apply for P3 as well), represented by a chief executive, such as a governor, mayor, or other elected leader, or the head of a State, local, or tribal agency. The lead applicant must be represented by an individual who is the authorized representative of the entity and able to bind the entity to legal commitments.

Lead applicants that are departments, divisions, or other agencies — must be recognized as government entities by the State, locality, or tribe that they represent. Any department or division of a State, local, or tribal government that is so recognized is an eligible lead applicant. For example, a city, county, municipality, town, township, parish, or governor’s office could be a lead applicant. In addition, a State, local, or tribal education, health, human services, housing, or labor agency; division of such agency; or State or local workforce investment board could be a lead applicant. Because recognition as a government entity may differ across jurisdictions, the specific State, local, or tribal statute or charter that authorizes an entity may need to be consulted in order to determine whether the entity is so recognized.

In some cases, the lead applicant for P3 will represent a partnership that includes multiple organizations or entities. As a result, it is important to understand that a nongovernmental organization may still participate as a partner in a pilot even if that organization is not eligible to serve as the lead applicant. Nongovernmental entities or other partner organizations can serve as critical partners in developing and designing a pilot application and also executing a pilot, such as by coordinating across partners, delivering services, or helping to manage data. The P3 partnership may involve any public and private organizations, including nonprofit, for-profit business, industry, and labor organizations, as well as other State, local, or tribal government entities that are not the lead applicant.

No. The lead applicant must be a government entity but is not required to already be receiving Federal funds to be eligible to apply (see FAQ C-8 for further information). In order to qualify for a pilot, the proposal must include at least: (a) two Federal programs administered by a Federal P3 agency ¹ that have policy goals related to P3; and (b) one of which is administered (in whole or in part) by a State, local, or tribal government.

Federal funds proposed for inclusion in a pilot may be contributed by organizations and agencies, other than the lead applicant, that would also be partners under the pilot. Partnerships are critical to pilots’ ability to provide innovative and effective service-delivery and systems-change strategies that meet the education, employment, and other needs of disconnected youth. We encourage applicants to build on strong, existing partnerships that have experience in working together to improve outcomes for disconnected youth. Partnerships will vary depending on the nature and focus of individual projects, but may cut across: State, local, and tribal levels of government; education, employment, and other agencies or programs operating within the same level of government; and governmental, non-profit, and other private-sector organizations.

Information on programs that may be eligible for inclusion in a P3 pilot can be found in the package of application instructions.

No. State-recognized Indian tribes, as well as tribes recognized by the Federal government, are eligible to apply.

Yes, if the applicant, whether directly or through one of its agencies or entities: (1) is wholly or partly administering a Federal program; (2) is classified as a State or local government for purposes of that Federal program; and (3) proposes to include that Federal program in the pilot.

A nonprofit organization may not serve as the pilot applicant or the fiscal agent for pilot implementation, but it still may play a significant role in the design and governance of a performance partnership pilot. For example, a nonprofit may:

• Facilitate the development of the pilot and prepare the application;
• Deliver services and coordinate service delivery under the pilot;
• Oversee broad implementation of the pilot, including providing progress updates and recommended course corrections to activities administered by government partners;
• Represent the State, local, or tribal partnership in meetings, communications, and negotiations with the Federal government on matters when all the partners, including the Federal government, agree that this is an appropriate role for the nonprofit; and
• Secure commitments from philanthropy, other nonprofit organizations, academic and research organizations, employers, or other private sector organizations.

When a performance partnership proposal envisions a role for nonprofits in the pilot, the applicant should clearly explain the proposed responsibilities of the nonprofit organizations, their role(s) in the governance structure, and their prior experience in successful collaboration with the participating State, local, and/or tribal governments. Nonprofits may be signatories to a performance agreement along with — but not instead of — participating State, local, and/or tribal government representatives. In these cases, the State, local, and/or tribal governments will continue to be the parties primarily responsible for meeting the terms of the partnership agreement. More information about the circumstances under which participating nonprofits may be signatories will be made available in the solicitation and during the application review process and may depend in part on the specifics of individual pilot proposals.

There are several information resources available that may be helpful to potential P3 applicants to identify potential partners including:

• Map My Community: An interactive mapping tool designed to locate federally supported youth programs in a community. This tool is available at youth.gov: http://youth.gov/map-my-community.

• America’s Service Locator: A search tool designed to locate Workforce Investment Boards, libraries, community colleges, or local employers. This tool is available at Careeronestop.org: http://www.servicelocator.org/contactspartners.asp.

• Map of Administration Community-Based Initiatives: This map shows some of the existing federal initiatives underway across the country and may be useful in identifying potential partnerships. This map is available at: https://www.whitehouse.gov/omb/place.

Applicants are not prohibited from submitting multiple proposals, nor are partner organizations prohibited from participating in more than one proposal. However, in order to be considered separate applications, each proposal must be distinct. We encourage applicants to indicate as clearly as possible in each application (e.g., in the title and abstract of their proposal) that they are submitting multiple applications, and that these applications are different from one another. Additionally, applicants and partner organizations must be capable of fully implementing each proposal if selected. Applicants should ensure that multiple distinct proposals do not propose to blend the same funds, including funds received by any of the partner organizations included in the proposal.

The Acts provide broad waiver authority for P3 projects. The Acts allow the heads of affected Federal agencies to waive statutory, regulatory, and other requirements that they are otherwise authorized to waive, as well as those that they might not otherwise be authorized to waive. In the Round 3 pilots (FY 2016) the affected agencies are the Departments of Education (ED), HUD,² Justice (DOJ),³ Labor (DOL), and Health and Human Services (HHS), along with the Corporation for National and Community Service (CNCS) and the Institute of Museum and Library Services (IMLS) (collectively, the Agencies). With respect to requirements that the Agencies might not otherwise be authorized to waive, the Acts include important safeguards that applicants and the Agencies must meet (see sections 526(d) and (f) of the 2014 Appropriations Act). Specifically, those waiver requests must be: consistent with the statutory purposes of that program; necessary to achieve the pilot’s outcomes and no broader in scope than necessary; and able to result in efficiencies or increased ability of individuals to obtain access to services provided by those Federal program funds. Requirements related to nondiscrimination, wage and labor standards, and allocations of funds to State and sub-State levels cannot be waived. In addition, the heads of the Agencies must determine that their agency’s participation and the use of proposed program funds: (1) will not result in denying or restricting individual eligibility for services funded by those programs, and (2) will not adversely affect vulnerable populations that are the recipients of those services (see FAQs C-2 and C-3 for more information).

The Agencies will consider waiver requests on a case-by-case basis in the context of the applicant’s full pilot proposal and these statutory protections. During the technical review, applicants will be scored, in part, based on the extent to which they: (1) demonstrate that the requirements for which they are seeking waivers are hindering successful achievement of outcomes for the target population of disconnected youth who are identified for the proposed pilot; and (2) provide a justification of how the waivers, individually or together, will reduce barriers, increase efficiency, support implementation of the pilot, and produce significantly better outcomes for the target population. Applicants should focus waiver requests on changes to major program requirements that would otherwise inhibit implementation. Examples of waivers include changes to eligibility requirements, allowable uses of funds, or performance reporting.

Following the technical review, the top-scoring applications will undergo a flexibility review of the applicant’s proposed waivers by interagency teams. Representatives of the Agency from which program flexibility is sought will evaluate whether the waivers requested by top-scoring applicants, in addition to the proposed blending of program funds, meet the statutory requirements for P3, and are otherwise appropriate. For example, if an applicant is seeking flexibility under programs administered by HHS and DOL, its requests for flexibility will be reviewed by HHS and DOL officials, and these DOL and HHS officials will determine the appropriateness of the flexibility request. During the flexibility review process, applicants may also be asked to participate in an interview in order to clarify requests for waivers and other flexibility, and potentially other aspects of their proposals.

Under standard practice in many Federal programs and as permissible under the authorizing statute, the entire eligible population does not actually receive services each year or grant cycle. Eligibility requirements set an outer boundary for the target population, but program design, funding, or other limitations may result in services being delivered to only a subset of the eligible population. If allowed under the relevant statute or regulations, grantees may focus activities on a limited subset of the eligible population. This means that, even in programs in which funds are allocated to grantees based on the identification of specific types of individuals, there is no guarantee that each identified individual will receive services.

Similarly under P3, applicants may propose to focus their activities either on a limited subset of the eligible population of a particular program or on a broader eligible population without “denying or restricting the eligibility” of the individuals to receive services. An applicant may propose to blend a portion of funding from a program to serve a target population that differs from the program’s exact statutory eligibility requirements. An applicant may do so by either proposing to waive eligibility requirements to broaden the target population, or proposing to work with a targeted subpopulation of a particular program.

One important factor for applicants to consider is how much a proposal will affect the proportion of eligible individuals who actually receive services under a particular program (including existing program-funded services as well as any new or comparable services provided under the pilot). For example, an applicant could violate this provision if it proposes to blend all or most of the funds of a particular program, but would serve only a very limited subset of that program’s eligible population, as defined in the program’s authorizing statute, through its P3 activities. Such a proposal could deny or restrict the eligibility of individuals for service of a program because its implementation could directly result in the vast majority of a program’s eligible population not receiving services.

The Agencies have determined that there are at least two situations in which a proposed blending of funds would result in an adverse effect on the recipients of services under a particular program. The first situation involves a program that creates a universal entitlement (i.e., “entitlement” or mandatory programs) that enables all eligible individuals to receive services or benefits. Funding from such programs may not be blended under a pilot because the pilot would serve only a subset of the eligible participants, thereby adversely affecting the remaining participants. The second situation involves programs that provide individuals with direct benefits (such as vouchers, credits, scholarships, or other payments). Funding from these programs may not be blended under a pilot under any circumstances because such a pilot would adversely affect the recipients of the direct benefit.

For all programs for which a pilot applicant proposes to blend funds or seek waivers, the applicant must describe how it will ensure in its pilot proposal that the recipients of services under the original program will receive a level of services or maintain a level of outcomes comparable to what would occur in the absence of the P3 activities. In considering whether blending funds would adversely affect the recipients of services funded by the original program, the applicant should also consider whether there are other non-mandatory Federal funds or non-Federal funds that will be used to continue to serve the recipients.

An applicant must provide strong justification that the new approach that would result from any waivers or other flexibility is necessary to achieve the outcomes of the pilot, is no broader in scope than is necessary to achieve those outcomes, and will result in either (1) realizing efficiencies by simplifying reporting burdens or reducing administrative barriers with respect to such discretionary funds; or (2) increasing the ability of individuals to obtain access to services that are provided by such discretionary funds. (See Section 526(f)(2)(B) of the 2014 Appropriations Act.) Applicants are asked to provide this description in response to Selection Criterion (B) in the Round 3 notice inviting applications.

P3 pilots may represent valuable flexibilities for agencies and communities seeking to improve outcomes for disadvantaged youth including agencies that are already participating in a title IV- E demonstration project authorized by section 1130 of the Social Security Act. However, the flexibility authorized for P3 under the Acts cannot be applied to allow blending of costs used to draw title IV-E foster care matching funds because these funds are considered mandatory funds, regardless of whether a State has a title IV-E waiver. This means that these funds cannot be blended with other Federal funds or be subject to the additional waivers available under P3.

Nonetheless, in order to improve youth outcomes, applicants could still propose to coordinate a P3 pilot with a title IV-E demonstration project, including by braiding together funding streams so that IV-E funds retain their original identity and requirements. In using title IV-E funds, agencies must ensure that they continue to follow all applicable requirements of the title IV-E program and their waiver terms and conditions.

Some of the Federal funds that could potentially be blended in a P3 project have restrictions related to the use of funds for construction. For example, most funds awarded by the Department of Education cannot be used for construction. See 34 CFR 75.533. Consistent with P3 Application Requirement (c)(1) and Selection Criterion (b), if an applicant would like to use some of the P3 funds which normally cannot be used for construction for construction purposes, the applicant must include a waiver request in its application that identifies the programs and provisions for which it is requesting a waiver and an explanation for why the relevant agencies should waive provisions prohibiting use of funds for construction.

We note that construction does not include minor remodeling, which means minor alterations in a previously completed building. Minor remodeling also includes the extension of utility lines, such as water and electricity, from points beyond the confines of the space in which the minor remodeling is undertaken but within the confines of the previously completed building.

Yes. Applicants may propose to blend and braid funds that they or any of the partner organizations under the proposed pilot receive and administer. The lead applicant itself need not be a direct grantee or sub-grantee for these funds.

In order to be eligible for selection as a FY 2016 P3 pilot, applicants must propose to include at least some FY 2016 funds.

Funds eligible for inclusion (either through blending, braiding, or requesting associated waivers of program requirements) are certain discretionary funds appropriated in FY 2016 to ED, HHS, DOL, CNCS, and IMLS, as well as FY 2016 funds from DOJ’s Office of Justice Programs and HUD’s Homeless Assistance Grants. Applicants should specify in their application the funds appropriated by Congress for FY 2016 that they would incorporate into their pilots. It is important to note that the fiscal year when funds were appropriated by Congress may not be the same as the year when State, local, or tribal governments receive those funds. If you have a question about the fiscal year in which the Federal funds you would like to use were appropriated, please contact the State or Federal agency from which you received those funds.

The statutory requirements for programs eligible to be included in a pilot remain the same as in the first and second rounds of P3: they must target disconnected youth, or be designed to prevent youth from disconnecting from school or work, and provide education, training, employment, and other related social services. Similarly, for a program to be blended as part of a pilot, under the statute, the Federal agency must determine that doing so will not: (1) deny or restrict an individual’s eligibility to receive services; or (2) adversely affect vulnerable populations that receive services from that program. As in the first and second rounds, the statute does not permit pilots to blend mandatory funds, meaning funds from entitlement programs such as Temporary Assistance for Needy Families (TANF).

Application Requirement (g) requires applicants to provide: (1) for each Federal program, the grantee, the amount of funds to be blended or braided, and the percentage of total program funding received by the grantee that the amount to be blended or braided represents; and (2) the total amount of funds from all Federal programs that would be blended or braided under the pilot.

No, mandatory funds such as TANF, Medicaid, Social Security, Vocational Rehabilitation Services, and most Foster Care IV-E programs (see FAQ C-5) cannot be blended in a pilot. Mandatory funds can support a pilot as a braided resource; however, there is no authority to offer waivers or flexibilities for these programs under P3.

The Agencies will consider the inclusion of competitive grant funds that have already been awarded on a case-by-case basis. The Agencies will determine if the scope, objectives, and target population(s) of the grant appropriately and sufficiently align with the scope objectives, and target populations of the proposed pilot. Situations in which it may be appropriate to include an already-awarded competitive grant or grants in a pilot include cases in which there are similarities between the competitive grant and the proposed pilot, such as the project plan, performance goals and metrics, proposed participants, leveraging of diverse funding, and partnership approaches, and an increased potential to amplify an existing program model and improve outcomes for disconnected youth.

Situations in which it may not be appropriate to include already-awarded competitive grant funds in a P3 pilot include cases in which, for example, the competitive grant is undergoing a rigorous evaluation that could be negatively affected or interrupted by the inclusion in the P3 pilot. Additional situations in which it might not be appropriate include if the competitive grant had been awarded based on a proposal to serve a specific population that would not align with the pilot’s proposed target population, or if the proposed pilot approach could, in any way, adversely affect that targeted population or the overall goals of the competitive grant.

The Agencies will consider the strength of the applicant’s justification for including the already-awarded competitive grant funds in its proposed P3 pilot. This requires the applicant to clearly demonstrate that the scope, objectives, and target population(s) of the competitive grant appropriately and sufficiently align with the proposed pilot’s scope, objectives, and target population(s). The applicant must also justify any potential changes in terms and conditions of the existing competitive grant that may be required for the purposes of the pilot (such as allowable costs and activities).

In general, a comprehensive needs assessment is a systematic process to develop an informed understanding of the gaps or needs that exist, as well as the factors or root causes that contribute to the existence of those needs. A needs assessment first defines the scope of the assessment and may outline key questions to be answered by the assessment. Next, it gathers data to analyze and document findings, which may include strengths, gaps, opportunities, and challenges. Using this information, and other applicable evidence-based research, the assessment then establishes priorities and strategies for addressing the identified issues.

The Agencies acknowledge the diversity in definitions and processes for conducting a needs assessment. One example of a specific type and process for conducting a needs assessment is the community needs assessments described by HHS, Administration for Children & Families (ACF), at http://www.acf.hhs.gov/programs/ocs/resource/conducting-a-community-assessment-1.

While a specific model or process is not required, the P3 review process includes consideration of the extent to which the applicant used a comprehensive needs assessment that was conducted or updated (either by the applicant or by other partners or organizations) within the past three years (Selection Criterion (a)). The needs assessment should use representative data on disconnected youth in the jurisdiction(s) to be served by the pilot that are disaggregated according to relevant demographic factors (such as race, ethnicity, gender, age, and disability status) to: (a) show disparities in outcomes among key sub-populations; and (b) identify an appropriate target population of disconnected youth with a high level of need. For example, a comprehensive needs assessment that an applicant conducts or uses for purposes of this application may analyze workforce, education, and well-being data for disconnected youth in defined areas of service and identify a target population with significant outcome disparities in comparison to other peer groups. Applicants, especially those that are conducting a needs assessment for purposes of the P3 application, are encouraged to align priorities and next steps identified through the needs assessment to the pilot logic model in order to inform the overall project design (See FAQ F-1, Logic Models).

A logic model (also referred to as theory of action) is a well-specified conceptual framework that identifies key components of the proposed process, product, strategy, or practice (i.e., the “active ingredients’’ that are hypothesized to be critical to achieving the relevant outcomes) and describes the relationships among the key components and outcomes, theoretically and operationally (34 CFR 77.1). In other words, a logic model clarifies what the applicant is seeking to change or produce through the pilot — the expected results, and the intermediate and long-term outcomes — and identifies how the project’s activities will contribute to achieving that result.

As described in the Notice Inviting Applications (NIA), P3 applicants are required to submit a graphic (no longer than one page) that depicts the pilot’s logic model and illustrates the underlying theory of how the pilot’s strategy will produce intended outcomes. The first step in developing a theory of change, after identifying the issue(s) to be addressed, is to identify the theoretical solution(s) based on available data. The next step is to describe the desired outcomes and impacts in addressing the issue and develop a plan for attaining those goals. Using this information, a logic model communicates how the program would operate when implemented. A variety of frameworks are used to describe the parts of a logic model, and P3 does not require a specific model. However, applicants are encouraged to include the following elements in their logic model.

1. Inputs include the resources that are needed to carry out the program plans. Examples of inputs are personnel, facilities, funding streams, supplies, and equipment.
2. Activities are the services and interventions that are proposed as part of the program design. It is helpful to consult evidence from the field regarding the effectiveness of the activities in achieving the desired outcomes and goals. It should be clear from your logic model how the key components are related to, or expected to produce, the outputs that ultimately lead to the intervention’s intermediate and longer-term outcomes.
3. Outputs are the immediate results or products of the project activities, which are often (but not always) described in numerical terms. For example, outputs might include the number of youth who complete a certification program.
4. Interim indicators are goals that the intervention is expected to help achieve that lead to achievement of long-term outcomes. It may be helpful to include indicators that encompass different levels (such as participant-level, organizational-level, or system-level outcomes) and across time (such as short-term and long-term).
5. Long-term outcomes are the expected changes in behavior, attitudes, aptitude/skill, knowledge, etc. for the target population. In particular, because these pilots are intended to improve outcomes for disconnected youth, long-term outcomes are related to reconnection of youth or successful prevention of disconnection, including by ensuring youth are enrolled in school or gainfully employed.

Logic models may also show assumptions made by the applicant, as well as any external factors that may bear on the intermediate and long-term outcomes. These elements provide context for the proposed interventions.

For additional information on how to develop a logic model, see DOL’s WorkforceGPS Web site: https://innovation.workforcegps.org/ or “Evaluation Toolkit for Prospective WIF Grantees,” starting on page 10: https://www.workforcegps.org/sitecore/content/sites/innovation/resources/2014/06/11/16/22/designing-your-wif-evaluation-the-evaluation-toolkit-for-prospective-wif-gr. The Regional Educational Laboratory Pacific, one of the 10 Regional Educational Laboratories established and funded by ED’s Institute of Education Sciences, also has produced an Education Logic Model Application that can be used to build logic models. The Education Logic Model Application is available at: http://relpacific.mcrel.org/resources/elm-app/. Finally, CNCS has produced a resource on “How to Develop a Program Logic Model” at: http://www.nationalservice.gov/sites/default/files/upload/How%20to%20Develop%20a%20Logic%20Model%20508.pdf (PDF, 36 pages).

G. Outcomes and Interim Indicators

Applicants may choose from the menu of outcome measures and interim indicators listed in the NIA and that appear below or may propose alternative indicators and outcome measures if they describe why their alternatives are more appropriate for their proposed projects.

The specific outcome measures and interim indicators the applicant uses should be grounded in its logic model, and informed by applicable program results or research, as appropriate. Applicants must also indicate the source of the data, the proposed frequency of collection, and the methodology used to collect the data.

NOTE: The Workforce Innovation and Opportunity Act of 2014 (WIOA) (Public Law 113-128) establishes performance indicators for certain recipients of WIOA funds that P3 applicants may reference as useful examples of outcomes that applicants could establish for their P3 projects.

Following are additional examples of non-education/employment measures that may be appropriate for use in a pilot Use of these additional non-education/employment measures is not required.

Housing Stability and Other Well-Being Outcomes

Outcomes related to well-being of disconnected youth include stable housing as well as those related to personal, cognitive, and developmental status, such as self-regulation, coping skills, conflict resolution skills, personal efficacy, ability to plan, and prosocial behavior.⁴

Housing Stability and Homeless Reduction

The United States Interagency Council on Homelessness published a report, “Framework to End Youth Homelessness,” in February 2013 that identified four core outcomes for youth who are experiencing homelessness: stable housing, permanent connections, education or employment, and social-emotional well-being. The report did not provide interim indicators for these outcomes or the pathways to improving services, but it did include a logic model.⁵

Additionally, HUD has developed a series of system performance measures for Continuums of Care, to better gauge the performance of a community-wide effort to end homelessness instead of focusing only on isolated program or project performance. These system measures look at criteria such as length of time households remain homeless, the extent to which households that have exited to permanent destinations return to homelessness, the total number of households experiencing homelessness in a community, successful housing placement, and employment and income growth. ⁶ These system performance measures may be useful to P3 communities who are considering how to measure housing stability and reductions in homelessness.

Personal and Developmental Well-Being

Research indicates that major predictors of future youth disconnection include poor grades, poor health (including mental health and severe disability), problem peers, and early parenthood. ⁷ Tracking interim outcomes related to these risk factors, like pregnancy prevention and improved mental health, could improve an applicant’s ability to serve this population. For information on the role of risk assessment in service-planning and achieving short-term and long-term outcomes, see http://www.acf.hhs.gov/programs/opre/resource/framework-for-advancing-the-well-being-and-self-sufficiency-of-at-risk-youth.

Delinquency Prevention

Sample outcomes for programs serving justice-involved youth, in addition to the education and employment outcomes listed above, include the number and percent of youth who offend or re-offend while in the program.

While applicants are not required to use studies from Federal evaluation clearinghouses, applicants are encouraged to use (and cite) research, such as studies that appear in various Federal evaluation clearinghouses, to inform their pilot design, as relevant. While each Federal Clearinghouse on evidence-based interventions uses somewhat different procedures and criteria in its work, they are similar in that the research studies that they include are summarized, and the strength and rigor of their findings are assessed according to specific guidelines.

Clearinghouses with evidence that is related to potential P3 pilots include:

• ED’s What Works Clearinghouse (WWC): Evidence on programs, products, practices, and policies in education (http://ies.ed.gov/ncee/wwc/).
• DOL’s Clearinghouse for Labor Evaluation and Research (CLEAR): Evidence on labor-related issues (http://clear.dol.gov/).
• HHS’ Teen Pregnancy Prevention Evidence Review: Evidence on programs with impacts on teen pregnancies or births, sexually transmitted infections (STIs), or sexual activity (http://www.hhs.gov/ash/oah/oah-initiatives/teen_pregnancy/db/tpp-searchable.html).
• SAMHSA's National Registry of Evidenced-Based Programs and Practices (NREPP): Evidence on mental health and substance abuse interventions (http://nrepp.samhsa.gov/).
• HHS’ Home Visiting Evidence of Effectiveness: Evidence on home visiting program models that target families with pregnant women and children from birth to age five (http://homvee.acf.hhs.gov/).
• DOJ’s CrimeSolutions: Evidence on criminal justice, juvenile justice, and crime victim services, programs, and practices (http://www.crimesolutions.gov).
• Child Welfare Information Gateway: Information on child welfare programs and services (https://www.childwelfare.gov/).

Other useful Federal Clearinghouses that include literature summaries, program resources, and promising practices, although they do not rate the quality of the analysis or findings, include:

• Self-Sufficiency Research Clearinghouse: Research on low-income and TANF families (http://www.opressrc.org).
• Workforce Strategies Solutions: Research on education and training; employment, retention, and advancement; and management and operations (http://strategies.workforce3one.org).
• National Clearinghouse on Families and Youth: Resources to end youth homelessness, teen pregnancy, and family violence (http://ncfy.acf.hhs.gov/).

I. Evaluation

Although there are many different types of evaluation, only an impact evaluation can establish whether a program or intervention caused an observed outcome. A randomized controlled trial (RCT), when appropriate, can provide the most rigorous causal evidence.

An RCT research design measures the “impacts” of the intervention or program on individuals or systems. An impact is an estimate of the direction (positive or negative) and magnitude (by how much) of the change in outcomes that can be directly attributed to the intervention.

The key to this design is random assignment. Eligible applicants are randomly assigned, such as by lottery, to the treatment group that receives the services provided by the intervention or to a control group that does not. This approach ensures that the two groups are identical in all respects except that one will participate in the intervention (program services) and the other will not. Therefore, any differences in outcomes between these groups (such as different rates of employment) can be attributed to the intervention.

RCTs are considered the “gold standard” (i.e., the most reliable form) of evaluation because they allow programs to assert, with a certain degree of confidence, that participants have improved their outcomes because of that program. Although RCT studies can require more effort to design and implement, if random assignment is conducted correctly, the results provide clear, rigorous evidence of program effectiveness. Additionally, the results from an RCT evaluation will provide important contributions to the evidence base for the intervention. Results from this evaluation approach are also valuable to stakeholders and scholars in determining whether the expected impacts were realized, and in developing approaches that build on this evidence to refine and expand programs.

Quasi-experimental design (QED) studies are similar in most respects to RCTs except, importantly, they determine the members of a treatment and control group by methods other than random assignment. Typical methods including “matching” a treatment group of service participants to a group of similar individuals who did not participate, using characteristics of the individuals prior to their choice to participate. When RCTs cannot be used, quasi-experimental design (QED) studies can sometimes provide good estimates of impact, though they cannot fully distinguish the effects of the intervention (program services) from differences between the two groups that existed before the intervention began and that: (1) cannot be easily measured or used in matching (e.g., persistence, motivation, grit), and (2) could be related to important outcomes. For this reason, impacts estimated from QED studies have to be treated with some caution.

Implementation studies are important components of rigorous evaluations, no matter the design used. An implementation study illuminates and explains “what is happening and why” in the design, implementation, administration, operation, services, and outcomes of social programs. This type of study can provide context and information that makes evaluation results more useful for improving program implementation. In addition, findings from implementation research can be used to inform future program development or replication.

For further details and guidance on the key components of evaluation, please review “The Evaluation Toolkit for Prospective Workforce Innovation Fund Grantees” available at https://www.workforcegps.org/sitecore/content/sites/innovation/resources/2014/06/11/16/22/designing-your-wif-evaluation-the-evaluation-toolkit-for-prospective-wif-gr.

Applicants’ budgets must include funds for their participation in two meetings during the project period. The meetings will take place in Washington, DC, and are expected to last for three days. Applicants should plan to bring at least two project staff and may send up to four. An applicant that proposes to conduct an independent evaluation of its pilot may bring an additional fifth person from the evaluation team. All participant travel, accommodations, and meals should be budgeted using start-up funds.

Pilots may propose to spend start-up funds at any point over the course of the project period (which may not exceed September 30, 2020). Start-up funds do not need to be expended in the first year of the program. Applicants should request a specific start-up grant amount that is between $200,000 and $250,000 and describe in their Budget Narrative how the pilot will use these start-up funds to support effective implementation, such as planning, governance, technical assistance, site-specific evaluation, capacity-building, and coordination activities. Examples of other uses include supporting the measurement of pilot performance and results, such as modifications to information systems.

Based on input from the field about the importance of TA, the Agencies are helping sites meet their needs in a number of ways:

• Start-up grant funding. Applicants may propose to use the start-up grant funding for a variety of purposes, including TA that is specific to the needs of the proposed pilot.
• Other Federal funds blended under P3. Pilot sites may also secure TA using a portion of other Federal funds, beyond the start-up grants, that are blended for P3 purposes.
• Community of practice. All P3 grantees must commit to participating in a community of practice that includes an annual meeting of pilot sites (paid with grant funding that must be reflected in the pilot budget that is submitted as part of the application) and peer-to-peer learning activities. A community of practice is a group of grantees that agrees to interact regularly to solve a persistent problem or improve practice in an area that is important to them and the success of their projects. Establishment of communities of practice under P3 will enable grantees to meet, discuss, and collaborate with each other regarding grantee projects.
• Evaluation. Federal agencies are working to identify resources to support P3 pilots (and their independent evaluators) in conducting rigorous impact evaluations appropriate for their circumstances. Such support could include workshops on common approaches to designing an evaluation common problems encountered, tradeoffs of different evaluation designs, and data collection topics, as well as site-specific assistance on specific issues that arise.
• The Agencies will support the community of practice to organize and disseminate TA tools and resources that have been created and/or identified by the Agencies that would have broad applicability across the P3 pilots. These resources might include links to grantee webinars or transcripts and recordings from calls with project directors, written guidance to assist pilots in understanding program requirements and relevant laws and regulations, and program announcements and other news. The Agencies also intend to provide guidance on Federal information and privacy laws (such as the Paperwork Reduction Act, the Family Educational Rights and Privacy Act, and the Health Insurance Portability and Accountability Act) and assist grantees in developing model consent forms.

Pay for Success, also referred to as Social Impact Bonds, is an innovative financing model that leverages philanthropic and private dollars to fund services up front, with the Government reimbursing investors after the services generate results. Both P3 and PFS are funding models that focus jurisdictions on defining specific outcome goals for a well-defined target population, using reliable data to measure progress, and generating evidence about cost-effective interventions.

Both P3 and PFS are complex, emerging models that the Federal government is testing, and each one requires some experience and expertise to implement successfully. As noted in the NIA, P3 applications will be scored, in part, based on partnership capacity. This capacity takes into account the extent to which partners have the necessary authority, resources, expertise, and incentives to achieve the pilot’s goals, resolve unforeseen issues, and sustain efforts to the extent possible after the project period ends, including by demonstrating the extent to which, and how, participating partners have successfully collaborated to improve outcomes for disconnected youth in the past. PFS brings together new partners, such as outside investors, to implement a new model of service delivery that, in many cases, the partners will be implementing for the first time. As a result, applicants may be challenged to demonstrate partnership capacity, including how partners have successfully collaborated to improve outcomes for disconnected youth, if they propose a PFS model that partners have not previously implemented.

Although applicants may be challenged to demonstrate partnership capacity if they propose blending PFS funds or supporting a PFS model with P3 funds, there are ways in which a P3 project may be used separately from a PFS project that would still complement or prepare for a PFS project. For example, a jurisdiction that is considering pursuing PFS but lacks needed information about current outcomes for youth before structuring a project might first propose a P3 project to understand these outcomes through the careful tracking required under P3. Or a jurisdiction seeking to sustain a successful intervention first tested through PFS might propose to continue supporting the intervention through a P3 project.

Other Federal resources that are helping jurisdictions to build capacity to implement Pay for Success include grants recently awarded by the Social Innovation Fund. More information on the Pay for Success model is available at http://payforsuccess.org/.

No. There is no minimum or maximum number of youth to be served by the pilots. However, applicants must define the target population to be served by the pilot, based on data and analysis demonstrating the need for services within the relevant geographic area. Please see Application Requirement (b), Target Population.

M-1. There are several different kinds of priorities in this competition. How do these different priorities work?

The Round 3 P3 competition includes three different types of priorities. Here is what each of these types of priority involves:

Absolute priorities: For the purposes of this competition, absolute priorities create separate categories for scoring and considering applications. An applicant must meet an absolute priority to be considered for funding. These absolute priorities are helpful because a diverse group of communities could benefit from P3. We are aware that strategies to improve outcomes for disconnected youth may differ across environments and wish to test the authority in a variety of settings. This Round 3 competition includes four different absolute priorities:

• Absolute Priority 1: Applications that are designed to improve outcomes of disconnected youth. Note that any applicant that does not qualify for absolute priorities 4, 3, or 2 may be considered under Absolute Priority 1.
• Absolute Priority 2: Applications that are designed to improve outcomes for disconnected youth in one or more rural communities (as defined in the Round 3 notice) only;
• Absolute Priority 3: Applications that: (1) propose a pilot that is designed to improve outcomes for disconnected youth who are members of one or more State- or federally-recognized Indian tribal communities; and (2) represent a partnership that includes one or more State- or federally-recognized Indian tribes; and
• Absolute Priority 4: Applications that are designed to improve outcomes for disconnected youth in one or more communities that have recently experienced civil unrest.

Note that any applicant that does not qualify for Absolute Priorities 2, 3, or 4 may be considered under Absolute Priority 1.

Competitive preference priorities: Competitive preference priorities allow applicants to receive extra (bonus) points for meeting the priorities. An applicant can qualify for these points regardless of the absolute priority for which they apply. This Round 3 competition includes four competitive preference priorities:

• Competitive Preference Priority 1: Projects that: (1) will serve disconnected youth who are neither employed nor enrolled in education and who face significant barriers to accessing education and employment; and (2) are likely to result in significantly better educational or employment outcomes for such youth (up to 5 points);
• Competitive Preference Priority 2: Projects that provide all disconnected youth served by the project with paid work-based learning opportunities (3 points); and
• Competitive Preference Priority 3: Projects that are designed to serve and coordinate with a federally designated Promise Zone (2 points); and
• Competitive Preference Priority 4: Applicants that plan to conduct independent impact evaluations of at least one service-delivery or operational component of their pilots (site-specific evaluation), in addition to participating in any national P3 evaluation (up to 10 points).

Invitational priorities: Invitational priorities express the Agencies’ interest in applications that will improve outcomes for particular subpopulations of disconnected youth. They do not provide a competitive or absolute preference over other applications. This Round 3 competition includes three invitational priorities:

• Invitational Priority 1: Applications that propose a pilot that: (1) will serve disconnected youth who are homeless youth (as defined in the Round 3 notice); and (2) is likely to result in significantly better educational or employment outcomes for such youth.
• Invitational Priority 2: Applications that propose a pilot that: (1) will serve disconnected youth who are involved in the justice system; and (2) is likely to result in significantly better educational or employment outcomes for such youth.
• Invitational Priority 3: Applications that propose a pilot that: (1) will serve disconnected youth who are or have ever been in foster care; and (2) is likely to result in significantly better educational or employment outcomes for such youth.

For the FY 2016 P3 competition, applicants may determine whether a particular LEA is eligible for the SRSA or RLIS programs by referring to information on the following Department Web site: http://www2.ed.gov/programs/reapsrsa/eligible15/index.html.

The first tab in the spreadsheets available at this site lists LEAs that are eligible for SRSA; the second tab lists LEAs that are eligible for RLIS. Applicants may determine school locale codes by referring to the following Department Web site: http://nces.ed.gov/ccd/schoolsearch/.

Involvement in a pilot by an LEA or school is not a requirement to participate in P3.

An applicant may demonstrate that its application meets Absolute Priority 2 by including in its application a description that includes: (1) a list of the communities it proposes to serve; and (2) a list and the NCES identification codes of (a) the LEA or LEAs that serve each of the communities it proposes to serve if the applicant qualifies for this priority through the criterion using the SRSA program or the RLIS program, or (b) the school or schools that serve each of the communities it proposes to serve if the applicant qualifies for this priority through the criterion using school-level NCES locale codes.

No, the NIA does not include a definition of the term “civil unrest.” However, the notice identifies several examples of what might be considered “civil unrest,” such as large protests or instances of civil disobedience, increases in self-directed or interpersonal violence in concentrated areas, or civic disorder prompted by a public health emergency. An applicant that applies under the absolute priority for communities that have experienced recent civil unrest should describe the instance(s) of civil unrest, including (1) a description of the civil unrest that occurred in the community or communities it intends to serve; and (2) the date or dates the civil unrest occurred.

Anapplicant seeking the additional points available under the competitive preference priority for Promise Zones must provide a HUD Form 50153 (Certification of Consistency with Promise Zone Goals and Implementation) that has been signed by an authorized Promise Zone official.

This form may be found at the link below:
http://portal.hud.gov/hudportal/documents/huddoc?id=HUD_Form_50153.pdf (PDF, 1 page).