1 Wilder Research Center Program theories and logic models 2 Wilder Research Center What are program theories? What is a program theory? Programs whose objectives are to cause changes in participants are inherently theory-based. According to Len Bickman (1987), a program theory can be defined as “a plausible and sensible model of how a program is supposed to work.” A good program theory logically and reasonably links program activities to one or more outcomes for participants. Program theories can often be captured in a series of “if-then” statements – IF something is done to, with, or for program participants, THEN theoretically something will change. For example, an aggression reduction program based on social learning theory may have the following underlying theory: “IF facilitators model nonviolent conflict management strategies and provide opportunities for participants to practice these strategies, THEN participants will successfully develop nonviolent conflict management skills.” Theories can be developed based on: • Established theories or models of professional practice. • Previous research findings (from your own program or published literature) • Expertise and experience of program staff • Feedback and input from key program stakeholders One potentially helpful model for considering the program theory is the theory of action (Bennett, 1979). This model outlines the basic theory underlying many intervention programs. According to this model: • With available resources (e.g., staff, equipment, materials), activities or services are provided. • Program participants engage in these activities and react to what they experience. • As a result of their involvement in these activities, program participants experience changes in their knowledge, attitudes, and skills. • These knowledge and attitude changes promote changes in participants’ behavior and practice. • As a result of these behavior and practice changes, the program has an overall impact on the individuals served or on the broader community. For example, a child abuse intervention program may have the following theory of action. First, qualified counselors will provide education and counseling services to parents who have a history of abusive behavior towards their children. Second, parents will participate in these education and counseling services and will react to these experiences. Third, parents will develop an increased understanding of the impact of abuse on their children and increased knowledge of effective and non-violent anger management and parenting strategies. Fourth, as a result of this increased understanding and knowledge, parents will successfully implement these anger management and parenting strategies at home. Fifth, as a result of these improved anger management and parenting practices, child abuse will be reduced and family systems will be strengthened. 3 Wilder Research Center How do I develop a program theory? The purpose of this step is to understand and describe how the activities that are provided should promote changes in participants. It is very important to go deeper than superficial assumptions about how activities link to outcomes. Instead, one should consider the available theories and research evidence that support these connections. In this way, we can be more confident in the underlying strength of our service delivery model. Use the list of activities that you developed prior to this workshop. Select one of your activities and answer the following three questions. Repeat for the remaining program activities. ACTIVITY: ______________________________________________________ IF the activity is provided, THEN what should be the result for participants? WHY do you believe the activity will lead to this result? Is there an underlying formal theory or set of assumptions guiding this activity? What evidence do you have that this activity will lead to this result (e.g., data from your own or other programs, published literature, etc.)? 4 Wilder Research Center What are logic models? What is a logic model? A logic model is one tool for illustrating an underlying program theory. A logic model illustrates the linkages between program components and outcomes. It can be portrayed in various ways (text, table, diagrams, etc.). Once developed, a logic model can be used in multiple ways, including evaluation planning, program design, goal setting, communication with stakeholders, and program improvement. A logic model usually involves the following elements: • Inputs: any resources or materials used by the program to provide its activities (e.g., money, staff, volunteers, facilities, equipment, supplies) • Activities: any services or treatments provided by the program • Outputs: any quantifiable products of a program (e.g., number of classes taught, number of people served, amount of educational materials distributed, number of hours of service delivered) • Outcomes: any characteristics of the participants that, according to the theory and goals of the services, can be reasonably expected to change as a result of the participant’s receiving services A sample logic model is attached. The model is read as follows: • The first column summarizes the key inputs or resources needed to provide services. • The second column lists the actual activities/services provided by the program. • The third column lists the quantifiable products or outputs of the services provided. Beginning with the fourth column, the model illustrates the outcomes of the program on participants. The number of outcome columns vary depending on the underlying logic. One frequent approach is to have the following three columns, but there may be more or less. • The fourth column usually describes the immediate impacts or results of these activities. They should be read as “if the activities are provided, then these outcomes should result.” Immediate impacts typically refer to changes in knowledge or awareness, as these types of changes typically precede changes in behaviors or practices. • The fifth column typically describes intermediate outcomes. They should be read as “if the immediate outcomes occur, then the intermediate outcomes should result.” Intermediate outcomes usually refer to behavioral changes that follow the immediate knowledge and awareness changes. As we move from immediate to intermediate outcomes, the direct impact of the activity and accountability of the program decrease. • The sixth column describes the long-term outcomes. They should be read as “if the intermediate outcomes occur, then the long-term outcomes should result.” Long-term outcomes usually refer to more global changes, such as community impacts. Again, at this level the direct impact of the activity and accountability of the program decrease. • This model includes a seventh column, which describes the intended impact of the program on the larger community. If you are hoping to achieve community-level impacts, it is important to think through the steps that must happen first before these changes can be seen. 5 Wilder Research Center LOGIC MODEL: Program providing community leadership training and service learning opportunities. Provide training and leadership opportunities Provide ongoing support/ mentoring • Participants acquire problem solving, leadership, and social skills • Participants have increased awareness of cultural and individual differences • Participants effectively communicate in diverse teams • Participants demonstrate improved social skills • Participants demonstrate increased public leadership skills • Participants are comfortable and effective working with diverse groups of people • Participants are active in their community Provide community service/ service learning opportunities Participants have increased knowledge of community issues and increased commitment to community work • Communities are strong and vital (higher employment, less crime, less poverty) • Communities have increased volunteerism and community ownership • # of people served • # of hours of service provided • # of people served • # of hours of service provided • # of people served • # of hours of service provided Community Impact Long Term OutcomesIntermediate OutcomesImmediate OutcomesOutputsActivitiesInputs • Trained staff • Volunteer mentors • Community partners • Training materials • Funding • Office space • Supplies 6 Wilder Research Center How do I develop a logic model? Once you have developed a program theory, you can use that as the basis for constructing a logic model. Go through the following steps to create the logic model. First, clarify the linkages between activities and outcomes Fill in the following table for each activity. Use the program theory to illustrate the linkages between each activity and its outcomes. Consider the order in which outcomes should appear. What should be the first changes experienced by participants? How should these initial changes promote other, more long-term changes? It is generally unlikely that behavior changes will be the first changes experienced by participants. According to the theory of action, behavior and practice changes are usually preceded by changes in knowledge, attitudes, or skills. Community impacts usually cannot occur unless individuals in that community have changed their behaviors or practice. ACTIVITY: Immediate outcomes (changes in knowledge, attitudes, skills) Intermediate outcomes (changes in behaviors or practices) Long term outcome/ Overall Impact 1. 2. 3. 4. 5. 7 Wilder Research Center Second, add inputs and outputs to the model for each activity The next step in developing a logic model is to consider the program’s inputs and outputs. Inputs are the resources and raw materials that go into a human service program. Anything a program uses to accomplish its purposes is an input. What are your major program inputs? Outputs are quantifiable products — anything a program produces. Outputs are different from outcomes. While outcomes describe the actual impact of a program on participants, outputs simply describe the amount of service that was provided to participants. Outputs are most often expressed in terms of number or amounts, such as the number of people who participated in an activity or the amount of service that was received. For each of your activities, what are the outputs or products that should emerge as a result? Activity Outputs 1. 2. 3. 4. 5. 8 Wilder Research Center Third, construct a draft model Third, create a draft of your logic model. First, fill in the boxes to indicate the inputs, activities, outputs, and outcomes. Depending on the complexity of your model, you may not use all boxes or you may need to add more boxes. Second, add arrows to illustrate the connections between your inputs and your activities, between your activities and outputs, and between your outputs and each sequence of outcomes. Short-Term Intermediate Long-term INPUTS ACTIVITIES OUTPUTS OUTCOMES OUTCOMES OUTCOMES 9 Wilder Research Center Fourth, review the logic model and revise if needed The final step is to review your logic model. Answer the following questions. If the answer to any question is “unsure” or “no,” go back to the model. Consider revising the model. Please note that often it takes multiple revisions of the logic model before it reaches its final form. YES UNSURE NO Do the outcomes represent meaningful benefits or changes for participants? Will the outcomes help you communicate the benefits of your program? Are your outcome goals clear and understandable? Are your outcome goals realistic? Are the outcomes participant-focused, rather than program-focused? Does your model include the outcomes of greatest importance to your key stakeholders? Is it reasonable, based on research, theory, or commonsense, that the program can influence outcomes in a substantial way? Does the model include all important program activities that participants receive? Does the model make appropriate connections between inputs, activities, outputs, and outcomes?