PD staff and implementing partners should complete the monitoring process for a subset of PD section activities (see What should I monitor).  The PD monitoring process includes three parts:

1. Logic model 
2. Monitoring plan 
3. After-action review 

Logic model

What is a logic model?

Logic models are visual guides that outline how section activities will lead to expected changes in priority audience groups, ultimately achieving a defined objective or goal.  Similar to a math equation or recipe, a logic model ensures balance between what goes in on one side and what comes out on the other.  In the context of public diplomacy, the logic model combines inputs (resources), audiences, and actions to produce outputs and short-term and long-term outcomes.  The logic model serves as a template for "if-then" thinking, helping to articulate the relationship between actions and desired results.  This relationship is typically presented in a table format, like the one below:

Why complete a logic model?

Logic models are useful tools for several reasons:

• Logic models establish a common reference point for PD practitioners on how section activities will work, including important “if-then” relationships between section activity elements.  They are a useful, high-level guide to ensure everyone on a team understands the big picture and keeps objectives front and center during the project’s roll-out.
• Logic models underpin key elements of monitoring plans.  The content outlined in logic models informs what monitoring data to collect about section activities.  In situations where an evaluation is appropriate, a logic model’s articulation of an initiative/section activity’s “how and why” helps evaluation teams to identify areas to investigate.  
• Logic models help ensure the alignment of resources, actions and results and add clarity to planning by asking teams to refine their objectives, hypotheses, and assumptions.  
• Logic models promote accountability by providing a checklist during implementation to ensure major parts of a section activity are being implemented as intended.  
• Logic models enable clear communication with stakeholders by allowing you to tell the story of how a section activity will advance policy goals.
• Logic models can guide assessments and encourage learning.  Outlining the major components of an intervention makes it easier to ask and answer questions like “Did we achieve the outcomes we anticipated?” “Did we reach our priority audience?” and “Were our planning assumptions correct?” 

When to complete a logic model? 

Completing a logic model is part of the detailed planning process.  

The logic model is a useful tool to facilitate the detailed planning process, described in greater detail in PD in Practice.  PD sections should complete a logic model for each new or ongoing activity or initiative the section decides to monitor.  To begin this process, PD sections already should have  conducted situation analysis, which includes desk research on the relevant topic and local conditions and identifying a priority audience.  Additionally, sections already should have created a problem statement that articulates the gap between or challenges faced in moving from the current (observed) environment and the future (desired) environment.  With these elements in place, sections are ready to begin detailed planning and complete a logic model.

Who should complete a logic model?  

The team directly responsible for implementing the section activity should complete the logic model, whether that is an implementing partner or a project lead within the PD section.  

The section activity lead should consult with a diverse group of stakeholders to draft the logic model to: 

(1) establish strong logical relationships (i.e., between inputs, audiences, activities and outputs, and between outputs and outcomes), and

(2) identify an initiative/activity’s relevant assumptions and external factors.  

How to complete a logic model

When creating a logic model, start by discussing and agreeing upon the expected outcomes, so you can plan for the other components to lead to the desired long-term results.  

Then, use the logic model template to record your team’s thinking in a systematic way.  

Step 1:  Develop short-term and long-term outcomes

Regardless of who directly implements the section activity, PD section leads should identify the expected outcomes and how they align with foreign policy priorities during the detailed planning process.

• Outcomes:  Outcomes are the effects your efforts should have on your priority audience and align with the initiative or section activity goals.  
  1. Short -term outcomes:  Short- term outcomes are those that happen immediately after activities or actions.  

For example:  A short-term outcome of an entrepreneurship workshop could be that participants develop a marketing strategy.  A short-term outcome of a social media campaign may be increased awareness of post-implemented events and workshops among the social media audience.

2. Long-term outcomes:  Long-term outcomes are also effects or results, but they happen in sequence after and often build upon short-term outcomes.  A section activity may build awareness in the short term, and this awareness could lead to a long-term outcome of an attitude change.  This shift could take three weeks, or it could take three months, or even three years; the sequence is more important than the timeline.

For example:  A long-term outcome of the entrepreneurship workshop could be an increase in clients as a result of implementing a marketing strategy.  A long-term outcome of the social media campaign could be that followers take action, such as participating in post events and workshops.  

Additional guidance on developing outcomes can be found in Appendix E: Developing Outcomes.  

Step 2:  Develop outputs

• Outputs:  Outputs are the direct products or actions that result from the section activity, such as the number of workshops or participants.  They are typically tangible and countable.  Some common PD outputs are number of participants, number of likes on a social media post, or number of workshops held.

Step 3:  Brainstorm activities

• Activities:  Activities are the tangible actions you will take (e.g., design and release a social media campaign raising awareness on a potential trade agreement with the United States) or things you will do to reach your priority audiences and achieve your desired outputs and outcomes.  List the activities you are planning to do.  If you are writing a logic model for a section activity, you may break the section activity into discrete tasks or steps (e.g., create video for social media, create survey, book venue, book speaker, register participants).  If you are writing a logic model for an initiative, you may list the various section activities you will be completing (e.g., social media campaign, curriculum development,  workshop series, and a speaker series).

Step 4:  Identify your audiences

• Audiences:  The section activity audience is the group(s) of people you want to reach and affect through your policy-centered efforts.  You should prioritize and segment your audiences before planning specific efforts, so you should have a clear idea of your audience before developing a logic model, and then record that audience in the model.  While many sectors and industries use logic models, the  “Audience” column is unique to the PD logic model, and it highlights  the fact that audiences are central to PD work.

Step 5:  Develop a list of inputs

• Inputs:  Write the resources invested to start and maintain the section activity or initiative.  This column’s entries typically include staff time, funding, and facilities.  Other inputs may include venue space, curriculum materials, literature, organizational partnerships, equipment, computer software, expertise, a survey, images, graphics, etc.

Step 6:  Document assumptions

• Assumptions:  Assumptions are things you believe to be true that inform why you believe a section activity will work.  Planning assumptions are things that you assume will hold true for your plans to stay on track.  For example:  if you plan to do a radio campaign, you assume that your target audience owns a radio and listens to the radio.  Or you could plan for a speakers’ series in February, based on the speaker’s availability and the country’s major holiday, but plan to shift the event to May if funding is not available in time.

Step 7:  Review and refine your logic model

• Review:  Finally, look over your entire logic model, and work to refine it to ensure that the connections from one component to the next are simple and clear.  Section activities are built on underlying assumptions of cause and effect.  Use the logic model to check these assumptions, by focusing on the “if-then” relationships within the logic model.  Are there any gaps?  Are there any areas where you have to rely on a lot of assumptions to proceed from the “if” to the “then”? 

Resources for completing logic models

For more information on completing a logic model, please consult the following:

• Appendix C:  Logic model template
• Appendix D:  Completed logic model example
• Appendix E:  Developing outcomes

Monitoring Plan

What is a monitoring plan?

A monitoring plan is a document that outlines how you plan to measure a section activity’s outputs and outcomes.  It defines what progress and success look like by identifying indicators and tools for data collection.  The monitoring plan includes a list of output and outcome indicators, the tools used to collect indicator data, performance targets for each indicator, and actual values for each indicator, obtained through data collection.

A monitoring plan is typically represented in a table like the one below:

What is the difference between a section activity or initiative monitoring plan and a grantee monitoring plan?

The monitoring plan included as part of the PD monitoring process should not be confused with a grantee monitoring plan.  As described in the Federal Assistance Directive (FAD), a grantee monitoring plan asks PD sections to draft a plan that describes the proposed monitoring mechanisms (e.g., regular phone calls, site visits, expense sampling, reviewing reports, ) that they will take to ensure the grant is implemented as agreed. 

Why complete a monitoring plan?

A monitoring plan is a document that outlines how you plan to measure a section activity’s outputs and outcomes.  It defines what progress and success look like by identifying indicators and tools for data collection.  The monitoring plan includes a list of output and outcome indicators, the tools used to collect indicator data, performance targets for each indicator, and actual values for each indicator, obtained through data collection.

A monitoring plan is typically represented in a table like the one below:

Step 1:  Transfer Monitoring plan.

1. Copy and paste the activity from the logic model to the first column.  
2. Write whether the category is an output, short- term outcome, or long- term outcome in the second column.
3. Copy and paste the text of the output or outcome into the third column.  

Step 2:  Draft Indicators.  

What are indicators?

We cannot measure performance without indicators.  Indicators measure the progress or change of an output or outcome.  Indicators describe what we can count or observe to represent the outputs and outcomes from your logic model, measuring actual results compared to the expected outcomes.  Indicators are checkpoints or signals that tell us if we're on the right track and getting closer to (or further away from) our goals.

Indicators are measurements that help us understand how well a section activity's expected output or outcome is progressing.  For example, if a school wants to improve student capability in math, then one indicator might be student test scores on a standardized test.  Another indicator could be the number of students who request additional tutoring services.  

Indicators describe both outputs and outcomes.  

Output indicators measure the direct, concrete results of activities.  Output indicators measure the outputs from the logic model and show how much and what kind of things are being produced or completed in an activity.  Often these indicators are simple accountings of what has happened.  

Output indicators are commonly written as:

• # of [audience] who [participated in/completed activity] within [time]; OR
• # of [activities] held/completed/organized within [time].

Examples of output indicators include:

• Number of participants completing a workshop by the end of the period of performance.
• Number of alumni network events held in one year.
• Number of likes or shares on a social media post within one month of posting.

Outcome indicators show change or progress towards outcomes from your logic model.  These indicators typically include percentage change (either increase or decrease) in awareness, understanding, knowledge, skill, behavior, or attitude.

Outcome indicators are commonly written as:

• # or % of [audience] who [how output/outcome will be measured]

Examples of outcome indicators include:

• Quantitative:  The percent of participants who report an increased awareness of sources of foreign malign influence in their country by the end of the workshop.  
• Quantitative:  The percent of participants who report that they plan on applying what they learned in the workshop when they write essays for study abroad in the United States.  
• Qualitative:  The type of professional resilience tactics participants learned by the end of the program.  

Because of the long-term nature of outcomes, posts can choose from multiple types of outcome indicators.  

• Short-term outcome indicators capture and measure changes that occur immediately following an activity, often focusing on changes in the knowledge, awareness, or understanding of the activity’s participants.  
  • Examples of short-term outcome indicators include:
    • percent of training participants who report increased awareness of credible, fact-based news sources.
    • percent of network members who report they built new connections at a conference.
• Long-term outcome indicators describe the ultimate results of the activity you hope to achieve, often focusing on changes in skills, behaviors, or attitudes.  They can take weeks, months, or even years to realize and may be observed outside the time frame of your program, but the sequence is most important.  Long-term outcomes simply happen after short-term outcomes.
  • Examples of long-term outcome indicators include:
    • percent of training participants who increase consumption of credible, fact-based news sources.
    • percent of network members who collaborate on a science or entrepreneurship project with fellow STEM network members.

Second, differentiate between quantitative and qualitative indicators, acknowledging the importance of both numerical data and narrative perspectives.

• Quantitative indicators help us understand something using numbers and numerical values to describe things.  For example, instead of saying, “Some people liked the event.” (which is qualitative), a quantitative indicator might tell us that “50% of the people who attended the event signed up to receive more information about the topic,”  or “The program resulted in a 15% decrease in attendees who believed U.S.  sanctions caused food shortages from the baseline survey.” Quantitative indicators might include things like the number of attendees, percent change (positive or negative), amount spent or received, or the frequency of an event or occurrence.
• Qualitative indicators help us understand something without using numbers.  These focus on the qualities or characteristics of a situation, not the counting.  For example, instead of saying "there are 10 people," or “positive perceptions increased by 15%” (which are quantitative), a qualitative indicator might describe what attendees said they learned from an event.  Qualitative indicators might include narratives or stories, perceptions, descriptions, or ratings such as great, good, fair, poor.

How do I create indicators? 

To create indicators, follow the these steps:

1. Select an output or outcome from the logic model and brainstorm what you want to measure.  
2. Determine what you will count or observe and draft the indicator to describe that.  
3. Use the SMART criteria to refine the indicators.
4. Enter the indicators into the rows of the monitoring plan template in the “Indicator” column.  

Step A:  Select an output or outcome from the logic model and brainstorm what you want to measure.  Choose the output or outcome you want to measure.  Brainstorm a list of things you could observe or count that would show that your selected outcome or output  is (or is not) happening.  If it is an output, what kinds of measurement or observations would capture a change in the audience? 

Step B:  Determine what you will count or observe, and draft the indicator to describe that.  Draft an indicator that describes what you will count or observe.  

Step C:  Use the SMART criteria to refine the indicators.  SMART is a useful acronym that can help you double check that your indicators are a good fit for your activity and can actually be collected.  

The letters in SMART stand for the following words and concepts:

• Specific.  The target should cover only one concept; don’t use indicators that have two purposes or two components because doing so will make it impossible to disentangle the two.  
• Measurable.  You must have the ability to collect useful data in an effort to monitor the indicator.  
• Achievable.  Targets should be achievable, but ambitious.  Finding the balance is something of an art.  Targets that are both ambitious and achievable elicit excellent performance.  
• Relevant.  The indicator should be related to the concrete change in conditions you hope to achieve.  
• Timebound.  There should be a due date.  A deadline ensures everyone is aware of what is required and we are able to hold key stakeholders accountable.

Review the following not-so-SMART and SMART-er indicator examples:

Step D:  Enter the indicators into the monitoring plan template rows in the “Indicator” column.  Once you have double-checked that the indicators will accurately help your team  learn about whether the outputs and outcomes have been achieved, add it in the “Indicator” column in the monitoring plan template.  

Step 3:  Select a data collection tool.  

Each indicator requires a data collection tool to track progress.  Consider where and how you will collect data for each indicator.  Potential data collection tools include surveys, interviews, focus groups, direct observation, and document review.  

What are surveys?

Surveys help you learn about the knowledge, attitudes, beliefs, behaviors, or attributes of a (sometimes large) group of people.

Why use a survey?

• You want to know who your audience is (sex, age, etc.)
• You want to understand the knowledge, attitudes, beliefs, and behaviors of your audience.  
• You want to track how these things change over time.  
• You want qualitative feedback on a program.  

Resources: 

• The Basics of Consuming and Using Survey Data
• Survey Design Best Practices

What are interviews?

Interviews are one-on-one conversations between an interviewer and a respondent that primarily ask about firsthand knowledge, helping you learn more about an individual’s motivations, context, or perceptions, or language use without influence from others.  Interviews give time for an individual to discuss topics, attitudes, perspectives, and experiences in depth.  “Expert” or “key informant” interviews are those conducted with people who have specialized knowledge or subject matter expertise about a topic.

Why use an interview?

• You want to obtain information on a topic directly from people with relevant knowledge, expertise, or experience.
• You want to gain depth of understanding beyond what is possible in a survey.
• You want to understand motivations or reasons for attitudes and behaviors.
• You want to collect recommendations about program implementation.

Interview best practices

• Create an interview guide with a set of questions that matches research needs and the interview setting.
• Take advantage of moderator’s role to:
  • Probe in depth on individual questions;
  • Create a structured and planned out conversation.
• Take thorough notes, identify key moments or statements, review transcripts for further analysis.

What is a focus group? 

Focus groups are semi-structured discussions with a small group of participants (usually 4-10), and led by a moderator to explore participants’ feelings, experiences, and preferences about a topic or issue.  Semi-structured discussions have pre-planned questions and themes to broadly structure the conversation, but allow for unplanned questions based on what is heard.  Focus groups differ from individual interviews because they encourage participants in the group to react/respond to each other.

Why use a focus group?

• You want to gain a deeper understanding of motivations, context, perceptions, or language from a small group of participants (usually 4-10).
• You want to understand how individuals discuss a topic with other(s).
• You want to gather a large amount of information in a short period of time.
• You want to hear about shared and differing perceptions and attitudes about a topic, initiative, program, or activity.
• You want to explore language use and how participants talk about a topic.  

Focus group discussion best practices

• Create a discussion guide or protocol with:
  • open-ended questions on key themes;
  • probing questions and follow-ups.
• Take advantage of the moderator's role to encourage interaction.
• Take thorough notes, identify key moments or statements including points of agreement or disagreement, review transcripts for further analysis.

Resource:  The Basics of Qualitative Research

What is observation? 

Observations are a “watch and listen” exercise where observers record processes, events, and actions that they see and hear.  There are different types of observation, including:

• Structured observation, which records information about a predetermined checklist;
• Participant observation, which entails engaging in project activities and recording what occurred, impressions, and reflections.

Why use observation?

• You want to study processes, actions, and events in their natural setting.
• You want to uncover patterns, conditions, or problems that may not be revealed through other methods of obtaining information from individuals.
• You want to understand the results of performance monitoring and why.  

Observation Best Practices

• Identify in advance what you need to observe in order to obtain the information that answers the questions you have about your section activity:  
  • Consider what people do and say, what they know, and what they make and use; and
  • How will you observe and record those things?
• Take detailed notes during and immediately after the observation; these “field notes” are raw data for analysis.

What is document review? 

Document review involves collecting and reviewing program documents and administrative information to gain insights into your activity.  Examples of documents you may review include: program applications, past evaluation reports, grant applications, alumni records, and documents produced by participants.  For example, you might ask participants in an entrepreneurship program to complete business plans as part of their participation.  Reviewing those plans may help you gain insights into how many people actually completed them or the quality and utility of those plans.  

Why use a document review?

• You want to gather background information.
• You want to gain additional insight into a section activity or process.
• You want to uncover additional insights that may be valuable.
• You want to compare two or more sources of data in the same category to determine their similarities and differences.

Document review best practices

• Define the reason for the document review and outline your goals in advance.  
• Establishing clear criteria for the review, including the review’s purpose, target audience, and any specific guidelines or standards that must be followed.
• Establish a list of relevant documents to review.
• Extract key information and organize it in notes.

Consider what tools for data collection would be most feasible to use to collect information from the participant or beneficiary, as well as who will analyze the data and how it will be analyzed and presented.  Once you identify a data collection tool, add it in the “Data Collection Tool” column in the monitoring plan template.

Step 4:  Identify a target.  

Each indicator should have a target, which describes the expected value either at regular intervals during section activity implementation (quarterly targets for example) or at the end of the activity.  After you decide how you will collect data about your indicator, you will set a target or goal for that indicator.  How well do you think the measure or observation will do by the end of the activity?  The target sets the level of performance you expect for a specific measure during a certain period of time.  When the activity is complete, or at important milestones along the way, you will compare the targets to the actual results to see how well the activity is doing.  Targets should be doable but still challenging to achieve.  To set targets, it helps to look at the starting point, called baseline data.  For activities that have been going on for a while, the baseline could be formed from past activity performance.  How many people did you train or how much did it cost last time?   For larger projects, you might collect new data to determine a baseline measurement.  You might, for example, send a survey to participants before the program begins to measure their awareness on a topic.  For new activities, the baseline for number of participants or activities is usually zero since they haven’t happened yet.

• Once you identify a target, add it to the “Target” column in the monitoring plan template.

Resources for completing monitoring plans

For more information on completing a monitoring plan, please consult the following resources:

• Appendix F:  Monitoring plan template
• Appendix G:  Completed monitoring plan example

After-action review 

What is an after-action review?

An after-action review (AAR) is a structured, critical review of a section activity (or part of it).  It uses a simple, structured meeting and/or note-taking template designed to help a team pause and reflect, prioritize specific management actions and make an action plan for any required next steps.  An after-action review provides space for the team and stakeholders to reflect on a section activity’s planning, implementation, and monitoring data, and it prompts them to consider how those reflections can and should inform future decisions.  

Why complete an after-action review?

Decision makers at any level can use after-action reviews to build a culture of intentional, collaborative learning for adaptive management.  When they guide decision making, AARs make things such as social media campaigns, events, or training sessions better when we do them again.  Sometimes, people do collect feedback from participants and colleagues, but it may not be consolidated, or connected to decision making.  Action plans set deadlines and help teams focus on the most important tasks from the AAR.  PD sections can also incorporate items from the AAR into their regular reporting, including in PD Tools, front- channel cables, information memos, and other communications within the mission.  

How to complete an after-action review

The after-action review provides a simple and straightforward structure for teams to debrief, document what they learned, and articulate recommendations for the future.  Most after-action reviews are held within a week or two of an initiative or activity’s completion.  In a large project, you may have an AAR after a major milestone.  AAR meetings can last for varied lengths of time, from one hour to several hours, depending on an activity’s complexity.  For very complex projects, you may choose to hold more than one AAR, broken up by project phase or by some other aspect.  

Planning and facilitating an after-action review

• Identify the ideal time to set up a review.  Ensure people have enough time after the end of an activity to collect their breath and their thoughts, but also that not so much time passes that they forget what happened.  Doing an AAR while memories are still fresh and relevant is important.  Meetings can always be rescheduled if needed.  If time is limited, consider adding more time to a recurring team meeting.  Send calendar invitations with all necessary meeting information, including virtual options if available.  
• Set expectations in advance.  Set up the meeting time and purpose well in advance, so the team is on the same page about when the team will debrief and discuss.  Send an agenda with tentative timing in advance of the meeting, so everyone understands the meeting goals.  Share all the monitoring data you collected on outcomes with activity planners and stakeholders for the discussion.  
• Consider offering an avenue for written feedback or input.  Consider offering a way for participants to share written input before or during the meeting.  There are several ways of doing this: for example, creating a collaborative document before the meeting for participants to share ideas; using a form (such as Microsoft Forms or Google Forms to to collect input to questions in advance; using an anonymous tool such as Slido or Mentimeter to solicit responses during the meeting; or asking participants to create their own debrief/AAR documents that can be shared or kept private as they participate in the meeting.  These written documents can help more introverted participants have a voice in the meeting, lessen the burden or fear of sharing an unpopular view, enable more people to participate in a large group, and broaden the perspectives of the group as a whole if some people are absent.  
• Assign a facilitator.  Consider assigning  an outside facilitator, so the whole team can participate fully in the AAR itself.  The facilitator should keep an eye on the time and agenda to ensure all topics are covered adequately, but not be an active participant.  They also should create space and opportunities for all participants to speak and participate; this role is especially important to cultivate an environment of inclusion and accessibility.  The facilitator also should work to keep the team’s discussion focused on achievements or challenges that are within the team’s ability to change.  
• Take and share notes.  Assign a notetaker, separate from the meeting facilitator, to record and distribute notes and manage the list of action items.  

For additional tips on running AAR meetings (and other types of meetings), refer to Plan a Great Meeting on the PD Foundations Learning Hub.  

Holding the AAR

When it is time to hold the AAR meeting,  structure the meeting around answering four questions, and then creating an action plan.  The four questions are as follows:

1. What did we expect to happen?
2. What actually happened?
3. What caused the difference between 1 and 2?
4. Based on our analysis in question #3, what will we do the same or differently next time? 

The first two questions are based on observations and descriptions, rather than judgments.  The third question is based on analysis, and the fourth question asks participants to apply judgment to their analysis.  This deliberative process is designed to help teams think systematically through events and processes, and to separate events from their follow-on personal effects, both of which are important to consider in an AAR.  

After answering those questions, the team moves on to developing its action plan, which provides time for the team to prioritize and operationalize next steps by agreeing on the next major decision point, including tasks to be completed before that decision point, along with corresponding points of contact and deadlines.  

During the AAR, the notetaker should consolidate answers to the four questions, note any questions for follow- up, and record the team’s action plan.  A fillable template is provided in Appendix H, to facilitate notetaking and consistency in your monitoring AARs.

Use the AAR to identify areas for sustaining excellence, opportunities for improvement, and ideas for adjustment.  If the team seems to be getting stuck, you may need to decide to move a conversation to a different method or setting, to find additional information or invite new perspectives, to allow the AAR meeting to continue to move forward.  In an AAR meeting, it is essential to allow participants to express multiple perspectives and viewpoints.  Finally, AAR meetings can be intense, so be sure to allow space for people to process the things that come up during the meeting.

Using the results of the after-action review

A successful AAR will result in reflection, learning, and recommendations for improvement that can be applied to current and future activities.  It will help you capture good practices and identify how to improve or end bad practices, promoting continuous team learning and improvement.  

• Share findings with stakeholders:  Make sure participants and stakeholders receive the notes and action plan emerging from the after-action review.  
• Prioritize knowledge management.  Save AAR notes in a location all staff can access so that everyone, especially new team members, can understand any decision making and apply the recommendations to future work.
• Incorporate in existing team processes.  Ensure that action items are included in team planning workflows.  Where possible, add tasks and deadlines to team calendars to ensure action items are delegated and completed by the proposed deadline.
• Consider sharing with the larger PD community.