Skip navigation

Tag Archives: program management

Introduction

This post is about uses of logic models in evaluation. It is one in a series about logic models and competitive grant seeking. Its context is the United States of America. Other posts discuss the uses of logic models throughout proposal planning and project implementation, types of logic models, typical elements in logic models, samples of logic models, and other topics.

 

Outcomes and Outputs

Logic models are versatile tools. Not only are they useful in planning and implementing a proposal, they are just as useful in creating an evaluation design.

 

An outcome is not the same as an output. An output is a product or an event and is reported as a number. An outcome is a logical result of an output. It is reported both as a number (after collecting the data) and a ratio. It may help to think of an output as a means (to an end), and an outcome as an end.

 

Outputs Outcomes
Installed 50 corner streetlights. Reduced intersection traffic accidents by 53%.
Held 6 interdiction workshops. Increased border drug seizures by 71%.
Created 10 classroom blog websites. Increased writing scores by 9%.
Developed 10 science lab lessons. Increased science lab scores by 12%.
Trained 700 program volunteers. Reduced afterschool adult-child ratio by 50%.

 

A logic model for an Evaluation Plan has seven basic elements:

  • Outcome: What do you to happen because of your project?
  • Indicator: What are the observable and measurable behaviors and conditions?
  • Target Audience: What is the specific population to be measured?
  • Data Source: What are the source(s) of information about the behaviors and conditions to be measured?
  • Data Interval: When are data to measure the indicator to be collected?
  • Target: What is the amount of change you desire to occur?
  • Results: What was the actual amount of change as measured using the data collected?

 

The seven elements can be organized in a table:

Desired Outcome Indicator Target Audience Data Source Data Interval Target Results
             
             

 

Examples of a Logic Model for Evaluation

These examples illustrate how to use a logic model in designing an Evaluation Plan.

 

Example 1: Science Education

  • Outcome: Participants will be more proficient in Science.
  • Indicator: Ratio of tested 6th graders who score proficient or higher
  • Target Audience: All 6th graders who participate regularly in the project.
  • Data Source: State-mandated 6thgrade assessments.
  • Data Interval: After test administration in April 2019.
  • Target: 75% or more of 500 participating and tested 6th graders
  • Results: 425 or 86.7% of 490 6th graders scored proficient or higher.

 

Example 2: Violence Prevention

Outcome: Fewer participants will be suspended for fighting in school.

  • Indicator: Ratio of HS students suspended for fighting.
  • Target Audience: All HS students who participate regularly in the project.
  • Data Source: District Title IV suspensions and expulsions reports.
  • Data Interval: After end of each academic ranking period and end of school year.
  • Target: 5% or fewer of 1200 participating high school students.
  • Results: 48 or 4.06% of 1180 high school students were suspended for fighting in school during the school year.

 

Example 3: Literacy Development

  • Outcome: More 6th, 7th, and 8th graders will read for pleasure.
  • Indicator: Ratio of students who read for pleasure during the school year.
  • Target Audience: All 6th, 7th, and 8th graders who participate regularly in the project.
  • Data Source: Surveys of participating students and their parents/guardians.
  • Data Interval: After end of each academic ranking period and end of school year.
  • Target: 65% or more of 900 participating and surveyed middle school students
  • Results: 620 or 71.75% of 864 respondents reported reading for pleasure during the school year.

 

 

 

 

 

Advertisements

Introduction

This post presents an alternative structure for logic models. It is one in a series about logic models and competitive grant seeking. Its context is the United States of America. Other posts discuss the uses of logic models throughout proposal planning and project implementation, types of logic models, typical elements in logic models, samples of logic models, and other topics.

 

Alternative Logic Model Structure

Although their configurations and their labels do vary widely, most logic models have six basic elements. What follows is a second (alternative) structure for the basic elements of a logic model. In it, the first two elements focus on what you plan to do. The others focus on what is to happen both while and after it is done.

 

The National Science Foundation (NSF) presents an instructive alternative structure for analyzing the elements of a basic logic model:

 

Inputs: What resources will be used to support the project?

 

Activities: What are the primary things the project will do or provide?

 

Outputs: How many and what sorts of observable and/or tangible results will be achieved?

 

Short-Term (or Immediate) Outcomes: What will occur as a direct result of the inputs and activities (typically in terms of changes in knowledge, skills, and attitudes)?

 

Mid-Term (or Intermediate) Outcomes: What results should follow from the initial outcomes (typically in terms of changes in behavior, policies, practice?)

 

Long-Term (or Ultimate) Outcomes or Impacts: What results should follow from the initial outcomes (typically stated in terms of changes in broader conditions?)

 

The time flow in a basic logic model reads from left to right [See: W.K. Kellogg Foundation, Logic Model Development Guide (2006), p. 1. These elements can be organized in a six-column table:

 

Inputs Activities Outputs Short-Term Outcomes Mid-Term Outcomes Long-Term Outcomes
           
           

 

Research Performance Model

Not every logic model is strictly linear. Scientific research is an iterative process. A research project has feedback loops between its input and activities, its research design and implementation, and its results and its outputs. In addition, the outputs of one research activity often serve as the inputs for a subsequent activity. Such circuits can repeat until a research project yields reliable results.

 

For more discussion of logic models, readers may want to visit the National Science Foundation’s Advanced Technological Education (ATE), the Southwest Educational Development Laboratory (SEDL), and the W.K. Kellogg Foundation’s Logic Model Development Guide (2006).

 

Introduction

This post discusses some of the benefits of using logic models throughout proposal planning and project implementation. It is one in a series about logic models and competitive grant seeking. Its context is the United States of America. Other posts discuss the uses of logic models throughout proposal planning and project implementation, types of logic models, typical elements in logic models, samples of logic models, and other topics.

 

Benefits

The varied uses of logic models promise many benefits to grant seekers and proposal planners, as well as to grant makers. Appropriately constructed, a logic model can guide a project or initiative’s lifespan from cradle to grave. As a trans-temporal tool, it can be used to forecast what will be done, monitor what is being done, and then evaluate what has been done.

 

Among the many benefits of using logic models in seeking grants are to:

  • Generate an inventory of what is at hand and what is still needed to carry out a project or initiative
  • Assist in planning a proposal and in monitoring, adjusting, and evaluating subsequent implementation
  • Relate contemplated or anticipated activities to projected outputs and outcomes
  • Clarify how project activities will contribute or are contributing to accomplishing specific objectives
  • Enhance an applicant’s focus on obtaining results through its planned project activities
  • More efficiently and effectively communicate to target audiences a project’s goals, activities, strategies, and intended outcomes
  • Provide project implementers and participants with a clear roadmap for implementation, monitoring, and evaluation
  • Identify sources and uses of data for tracking progress toward target outcomes
  • Provide a single synoptic snapshot of a project’s scope of work and potential significance
  • Facilitate coordination of resources, selection of strategies, and realistic formulation of desired outcomes
  • Create among all stakeholders a shared understanding of and focus on program goals and strategies
  • Build a strong case for how and why a project or initiative is worthy of a funder’s investment in it
  • Communicate key project features to external audiences such as funding agencies, the general public, and legislators
  • Enhance the role and usefulness of monitoring and evaluation as management and learning tools

 

Planning Tool

One useful place to explore the many uses and benefits of logic models is W.K. Kellogg Foundation’s Logic Model Development Guide (2006). As you create your own logic model, always be mindful that other grant makers may define and understand some aspects of logic models quite differently than what you find here. Always defer to the specific funder’s instructions and guidance about using logic models in proposal development and project management.

 

 

Introduction

This post looks at types of logic models. It is one in a series about logic models and competitive grant seeking. Its context is the United States of America. Other posts discuss the uses of logic models throughout proposal planning and project implementation, benefits of logic models, definitions of logic models, typical structures and elements for logic models, samples of logic models, and other topics.

 

Types

Logic models are not all the same. There are at least three basic types, each with its own focus. Logic models can be created as single-focus constructs, or as constructs that combine two or more focuses. The best ones fit and reflect the needs of both the grant seeker and the specific funding opportunity.

 

Theory Models focus on the theory of change that underlies the design and work plan for a project. They are useful for identifying assumptions in selecting strategies and for linking them to activities. They are also useful in linking a research-based rationale to specific action steps.

 

Activities Models tend to focus on the specifics of implementation and the sequencing and coordination of planned activities. They are particularly useful for monitoring implementation and for project management.

 

Outcomes Models try to connect inputs and/or activities with desired results. They also sort outcomes and impacts over time (e.g., short-term, mid-term, and long-term). They are useful for developing plans for evaluation and other purposes related to accountability.

 

The table below sums up each type of model:

Types Key Questions Time Focuses Uses
Theory Why?

How?

Future Selecting strategies

and forecasting

Activities What?

How?

Who?

Present Monitoring

and coordinating

Outcomes So What?

How Well?

Past Evaluation

and reporting

 

For a more complete understanding of logic models, be sure to look for coming posts – particularly those about the structure of logic models and about samples of logic models.

 

 

 

Introduction

This post is about typical elements used in developing logic models. It is one in a series about logic models and competitive grant seeking. Its context is the United States of America. Other posts discuss the uses of logic models throughout proposal planning and project implementation, types of logic models, samples of logic models, and other topics.

 

Basic Elements

Although there is considerable variation in the available literature, most logic models appear to have six basic elements. Two or three of these focus on what an applicant plans to do. The others focus on what is to happen both while and after it is done. What follows is one of many schemas for the basic elements of a logic model.

 

Some of the available literature identifies six elements in a basic logic model (which can be sequenced left to right in a six-column table, as below):

 

Purpose and Context: What is the project’s scope of work? What is its overarching goal? Which specific audiences and systems will be expected to benefit? What problem or need or priority will it address?

 

Inputs or Resources: What resources are already available to do the work (finances, labor, facilities, other assets)? What further resources are needed?

 

Activities: With the available and requested inputs, what project activities will you implement? Over what span of time will you undertake them? In what sequence will they occur?

 

Outputs or Results: What events, products, or services will you deliver to target audiences through the project activities?

 

Participant Outcomes: How will participants’ awareness, knowledge, behavior, skills, or level of functioning change measurably subsequent to the activities and outputs?

 

Contextual Outcomes: How will organizations, operating environments, communities, policies, or other larger contexts change measurably subsequent to the activities and outputs?

 

Depending upon the type of model used (e.g., Theory, Outcomes, Activities) there are many variations in these elements. To get started, readers may want to explore the Southwest Educational Development Laboratory (SEDL) discussion of logic models.

 

Purpose and Context Inputs or Resources Activities Outputs or Results Participant Outcomes Contextual Outcomes

 

For more discussion of logic models, readers may want to visit the Southwest Educational Development Laboratory (SEDL), and the W.K. Kellogg Foundation’s Logic Model Development Guide (2006).

 

 

Introduction

This post presents several definitions of logic models. It is one in a series about logic models and competitive grant seeking. Its context is the United States of America. Other posts discuss the uses of logic models throughout proposal planning and project implementation, types of logic models, typical elements in logic models, samples of logic models, and other topics.

 

Purposes

Applications to competitive grant programs commonly require a logic model. As visual and synoptic depictions, they are useful ways to represent how the parts relate to the whole.

 

There are nearly as many definitions of logic models as there are discrete logic models themselves. Among these definitions’ common aspects are that a logic model is a versatile planning tool, that it is a product of collective inputs, and that it is a simplified graphical representation of more detailed plans described in a proposal narrative.

 

Among the primary purposes of using logic models in proposals are to improve foresight in project planning, to facilitate effective implementation, to focus implementers’ efforts on attaining desired outcomes, and to ensure 360-degree accountability.

 

Definitions

Any online search of logic models soon turns up many explanations of what they are, what they do, and how they look. Among the many discoverable explanations are:

 

“A logic model is normally presented as a one page visual diagram…. It is an important tool that facilitates planning, implementation and evaluation of a project intervention…. There are a variety of formats by which a logic model may be presented…. (Public Safety Canada).”

 

“Typical logic models use table and flow chart formats… to catalogue program factors, activities, and results and to illustrate a program’s dimensions…. (W.K. Kellogg Foundation)“

 

“A logic model is a highly visual method of demonstrating relationships between project resources, activities, outputs, and outcomes (SEDL).”

 

“A key element of the logic model diagram is showing the linkages (via lines joining the various boxes) between the activities and the eventual outcomes/impacts (Public Safety Canada).”

 

IF-THEN Logic

Thinking through a logic model uses an IF-THEN sequential logic of relationships. If certain inputs are available, then certain activities can occur; if the activities occur, then certain outputs can be expected; if certain outputs occur, then certain outcomes are likely over time; and if certain outcomes occur, then the identified needs or problems should change.

 

“Logic models can be useful tools to demonstrate integrated, systemic planning in relation to the achievement of goals and expected outcomes (SEDL).”

 

“Logic models should be a dynamic tool that assists staff in planning, implementation, and assessment efforts (SEDL).”

 

“There is no one ‘best’ logic model ((W.K. Kellogg Foundation)” – except, perhaps, the one that a particular grant maker may require.

 

Introduction

Applications to competitive grant programs commonly require a logic model. Development and use of logic models are common practices in planning projects before seeking a grant. They are also common practices in implementing and evaluating projects after winning a grant.

 

This post looks at typical elements of logic models. It is one in a series about logic models and competitive grant seeking. Its context is the United States of America. Other posts discuss the uses of logic models throughout proposal planning and project implementation, types of logic models, definitions of logic models, samples of logic models, and other topics.

 

Elements

In a compact form, a logic model represents key elements in a proposed Program Design (also called a Work Plan or Plan of Action), which is commonly submitted as part of a competitive grant proposal. Depending upon the specific funding opportunity, a logic model may identify most of these elements, if not all of them:

  1. A program design component
  2. A goal
  3. An objective (usually two or more per goal)
  4. A set of activities and/or strategies (usually two or more per objective)
  5. A timeline (with or without milestones)
  6. A plan for evaluation
  7. An assignment of persons responsible
  8. A budget allocation (representing one or more budget line items)

 

Developing a Logic Model

In creating a logic model for a grant proposal, several elements are typically necessary:

 

Component: Captures the programmatic focus of one or more goal.

Example: Leadership Development

 

Goal: States the broad intention of one or more objectives.

Example: To increase school principals’ skills in leading community engagement.

 

Objective: States the specific focus of effort defined in terms of measurable results.

Example: To increase 80% or more of 80 participating principals’ leadership skills through 10 hours of training in how to implement best practices in community engagement during the 2018-19 school year.

 

Activities: Indicates what the applicant’s staff (and/or its partners) will do to accomplish an objective.

Examples: (1) Organize and conduct a two-day (12-hour) Leadership Retreat. (2) Hold two three-hour follow-up sessions during the project period.

 

Timeline: Specifies when staff will complete the activities for an objective.

Example: Retreat: 08/2018. Follow-up: 12/2018 and 04/2019.

 

Evaluation: Determines whether, and to what degree, an objective is met.

Examples: Attendance, pre-post surveys, principals’ records, and community feedback surveys.

 

Staff Responsible: Identifies who will do the activities, by position title.

Example: Consultants, Professional Development Director, and Internal Evaluator.

 

Budget: Allocates funding to support accomplishing an objective.

Example: Specific amounts allocated to various appropriate cost categories and line items.

 

 

 

For decades now, outcome evaluation has been a key aspect of planning proposals that win grants. It requires a systematic analysis of projects or initiatives as they unfold over time, from inputs and activities onward through outputs and outcomes. The net result is a logic model useful for virtually every element of an evaluation plan.

 

Input:

It’s helpful to think of an input as any resource necessary to do the work of a project. As such, an input can be human labor (paid or volunteer personnel), materials (equipment and supplies), finances (existing and future funding), and facilities (locations where project activities will occur). Most inputs must be in place and available for use before a project starts.

 

Activity:

An activity is a work task associated with a project. It may occur before a project starts (e.g., planning), during a project (e.g., implementing and monitoring), or after a project ends (e.g., continuing and close-out). Some activities are singular or discrete events (e.g., yearly conferences), others are continuous processes (e.g., classroom instruction), and yet others may be both (e.g., training). Rationales for selecting specific activities should reflect documented needs and proposed objectives; often they also must reflect research into best practices.

 

Output:

An output is a unit of production or a unit of service, and stated as a number. It is often the focus of a process objective. As units of production, outputs include numbers of newsletters published, numbers of blog articles posted, numbers of curricular units developed, numbers of workshops held, numbers of library books purchased, and so on. As units of service, they include numbers of students taught, numbers of staff trained, numbers of parents contacted, numbers of patients treated, and so on. Outputs do not indicate what measurable changes occurred in the users of the products or in the recipients of the services.

 

Outcome:

An outcome is an observable and measurable change that occurs in a pre-defined population of intended beneficiaries either during or after a project. Often it is expressed in terms of a change in knowledge or skill (short-term outcomes), or a change in behavior (mid-term outcomes), or a change in affect, condition, or status (long-term outcomes). An outcome objective focuses on what is expected to happen as a consequence of staff undertaking a set of activities and of intended beneficiaries participating in them.

 

Thus, increased knowledge in teaching engineering is an expected outcome of taking a course in it; being a high school teacher who completed such a course is an output. Again, a reduced annual rate of middle school bullying is an expected outcome of implementing a school-wide model anti-bullying program; holding 12 hour-long sessions for all school staff on applying the model is an output. And creating a positive school climate is an expected outcome of a comprehensive school reform initiative; installing 20 posters about civic virtues throughout every school is an output.

 

Performance Target:

As the subject of a well-formulated project objective, each desired outcome is a performance target, which typically may be stated as a number or a ratio or both. The best target is both feasible and ambitious. Grant recipients use outcome indicators to observe, measure, monitor, and evaluate their progress toward attaining each performance target.

Once an organization has won a multi-year grant, evaluation is essential to getting it renewed year to year. One way to share evaluation findings is the Annual Performance Report (APR).

 

Although the specific contents of an APR vary from funder to funder, they also tend to have similar structures from report to report. What follows is one typical structure:

 

Face Page or Title Page: Should identify the grant recipient, the grant maker, and the grant program. Also may need to provide unique numerical identifiers: submission date, grant award number, employer identification number, grantee DUNS number, and others.

 

Table of Contents: Should always include whatever major topics, in whatever predetermined sequence that a specific funder may require.

 

Executive Summary or Abstract: Should offer an overview of findings and recommendations and be no longer than one page.

 

Overall Purpose of Evaluation: Should state: why the evaluation was done; what kinds of evaluation were performed; who performed them; what kinds of decisions the evaluation was intended to inform or support; and who has made, is making, or is going to make such decisions.

 

Background or Context: Should briefly describe the organization and its history. Should describe the goals and nature of the product or program or service being evaluated. Should state the problem or need that the product or program or service is addressing. Should specify the performance indicators and desired outcomes. Should describe how the product or program or service is developed and/or delivered. Also should characterize who is developing or delivering the product or program or service.

 

Evaluation Methods: Should state the questions the evaluation is intended to answer. Also should indicate the types of data collected, what instruments were used to collect the data, and how the data were analyzed.

 

Evaluation Outcomes: Should discuss how the findings and conclusions based on the data are to be used, and any qualifying remarks about any limits in using the findings and conclusions.

 

Interpretations and Conclusions: Should flow from analysis of the evaluation data. Should be responsive to the funder’s evaluation priorities (e.g., measuring GPRA or GPRMA performance indicators in Federal grants).

 

Recommendations: Should flow from the findings and conclusions. Also should address any necessary adjustments in the product or program or service and other decisions that need to be made in order to achieve desired outcomes and accomplish goals.

 

Appendices or Attachments: Should reflect the funder’s requirements and the purposes of the specific evaluation. Appendices may include, for example: the logic model governing the project; plans for management and evaluation included in the original proposal; detailed tables of evaluation data; samples of instruments used to collect data and descriptions of the technical merits of these instruments; case studies of, or sample statements by, users of the product or program or service.

 

What follows is a sample Position Description for a Program Developer. It is one of a series of posts of samples describing positions in a fictional STEM Partnership Project.

 

Position/Time Commitment: Project-paid Program Developer (100% FTE)

 

Name: Maria CD Garcia-Lorca, M Ed

 

Nature of Position:

Raises funds through partner contributions and through corporate, foundation, and individual donations; recommends, plans, develops, solicits, and submits capacity-building proposals for funding from non-federal sources.

 

Accountability:

This position is directly accountable to the Project Director.

 

Duties and Responsibilities:

  1. Identify, select, and recommend non-federal sources of capacity-building funding
  2. Develop and submit proposals for funding to non-federal sources of capacity-building support
  3. Propose, develop, and implement strategies to raise funds through partners’ contributions
  4. Prepare and make presentations to potential individual and corporate donors
  5. Provide timely needs assessment and evaluation data to the external evaluator
  6. Assist in preparing project continuation and renewal proposals
  7. Conduct and participate in training activities related to fundraising, grant writing, and proposal development
  8. Collect and internally disseminate information on fundraising and grant writing activities, opportunities, and outcomes
  9. Assist school administrators, teachers, and IHE project partners in securing non-federal funding for school-based STEM improvement activities
  10. Meet regularly with the project staff, present and new project partners, the school district and its committees, and the project’s Partnership Advisory Team (PAT)
  11. Attend workshops and conferences on grant writing and fundraising
  12. Assist the Project Director and Project Coordinator in partnership-building activities and work closely with project partners at all levels

 

Qualifications:

  1. Master’s degree
  2. At least five years of experience in fundraising and program development
  3. Knowledge of the philosophy and goals of educational partnerships and capacity building
  4. Demonstrated success in writing funded grant applications to sources at the local, state, regional, and national levels
  5. Familiarity with the district’s STEM improvement initiatives, and with basic principles of publishing and graphic design
  6. Ability to develop promotional material
  7. Excellent writing and public speaking skills
  8. Ability to relate effectively to administrators, teachers, students, parents, public and private sector partners, potential donors, and officials of funding programs
%d bloggers like this: