U9D1-28 – Exploring Outcome-Based Evaluation Options – Please follow all instructions outlined below. Do not skip anything. Readings and Forms are attached.

U9D1-28 – Exploring Outcome-Based Evaluation Options

Using the two peer-reviewed articles on ethnographic and OBE approaches you identified in the this unit’s study, provide a summary of your reflective action learning assessment on the validity and accuracy of the evaluation methods
used in terms of the progress, results-based accountability, and performance reporting.

Be sure to also consider some of the ups and downs, uncertainties, challenges, efficiency, or effectiveness issues with service, program or project use patterns, ethnographic implications, as well as the intended OBE methods to assess
your public needs assessment and planning project’s progress.

Finally, because the scientific merit action research template (SMART) form is the research plan that serves as the skeleton for your dissertation proposal in the DPA program, perhaps, it may be good to get introduced to it.  Being
introduced early to the SMART form will not only help you better acclimate yourself on what will be expected of you at the dissertation phase, but also you could perhaps start reflecting on the relevant questions noted on some of the
sections as lenses to help shape your public needs assessment and planning project.

For this, also provide a summary of how the action learning activities and knowledge base on public needs assessment and planning, can be helpful when it is time to complete sections of the SMART form for your dissertation proposal in
the DPA program. 

For this discussion, you are not required to complete the SMART form itself now. However, to acclimate yourself on what will be expected of you at the dissertation phase, just indicate how knowledge developed in this course can be translated to help address specific sections of the form.

Article Search
In preparation for the discussion in this unit, locate two peer-reviewed articles that used ethnographic and OBE approaches and can support your public needs assessment project. You can use the Databases A–Z library guide for searching the articles.

You will use this Scientific Merit Action Research Template (SMART) Form in a reflective action learning exercise in the
discussion later in this unit.

Transcripts Reading
Ethnography is a method of research design that is intended to address a problem in depth and detail. It forms the basis for many of our public needs assessment projects.

Complete the following transcripts:
Read What is Ethnography? to understand more about this important research methodology and project process.

Read Applying What You Have Learned about how you can leverage the work you have done throughout this course and ways by which the knowledge base acquired and competency developed can enhance your future action research dissertation project.

Readings
Note: Be certain to read the unit introduction, as it may contain important information and references pertaining to this unit’s content and activities.

Use your Designing and Managing Programs text to complete the following:

Read Chapter 9, “Designing Effectiveness-Based Information Systems,” pages 161–188.

Read Chapter 11, “Impact Program Evaluation and Hypothesis Testing,” pages 203–212.

Use your Conducting Needs Assessments text to read Chapter 10, “Reporting the Findings,” pages 171–179.

Read Coryn, Noakes, Westine, and Schröter’s 2011 article, “A Systematic Review of Theory-Driven Evaluation Practice From 1990 to 2009,” from American Journal of Evaluation, volume 32, issue 2, pages 199–226.

INTRODUCTION – Formulating Outcome-Based Evaluation of Needs Assessment Report for Stakeholders

At the outset of this course, you were introduced to the action learning knowledge base on the public needs assessment and planning processes representing the contextual gap between the present conditions or situation occurring within the community of interest and what ought to have been occurring to ensure the best possible services, programs, or projects for those whose needs-based issues are at stake. Also, you learned that the needs assessment planning process is that measurable instrument used in determining the measurable action that can foster desirable systems changes to bridge or
fill that gap. However, without a results-based accountability instrument and performance reporting framework, there can be no way of determining whether that contextual gap has been bridged and its measurable outcomes been attained.

Moreover, as an undergirding centerpiece, you have also learned that in conceptualizing the measurable action, the needs
assessment and planning processes provide a comprehensive framework for determining action intervention and service or
program use patterns in facilitation of an action-oriented research project. However, crucial to the facilitation of the public
needs assessment and planning process that can impact the action-oriented research project is the delineation of the
positionality of research facilitator. Also, because the central focus of this course is on the public aspect of needs
assessment planning projects, it is important to understand its ethnographic implications.

Moreover, your future participatory action research dissertation study includes you as a member of a team that works to
find a solution to a public problem or gaps in service, program, or project use patterns within the public domain.
Participatory action research builds on ethnography as one component of the research model. And that is another reason
to understand ethnography as a qualitative research methodology. Ethnography has its roots in a number of disciplines,
and each adds depth and richness to the research design.

Thus, throughout this course, you have been exploring the various components of public needs assessment and planning
processes and have learned a great deal about how gaps in service, program, or project use patterns can be bridged to
effect positive systems changes. In bridging those gaps, the components of public needs assessment and planning
processes may require any of the following plans: proximate or short-term, ultimate or long-term, and proximate-ultimate
or multileveled strategic planning. The outcome-based evaluation (OBE) is the accountability instrument that can provide
the framework for determining the impacts of such plans on program or project use patterns.

Even in the previous unit, you also explored how KTA intervention framework or other suitable frameworks can be used in
facilitation of your needs-based action learning of program or project planning, implementation, monitoring, and
performance evaluation processes. The OBE serves as the program or project inquiry blueprint in ascertaining to what
extent the program’s implementation has facilitated a successful outcome. In other words, without the OBE, how do you
really know the extent to which the public needs assessment and planning processes or perhaps the KTA interventions
have been effective or efficient in bridging the gaps in service, program, or project use patterns for the positive systems
changes? And without the OBE, how sure are you that all of the relevant needs-based data have even been well collected
to give accurate data representation of the SAF?

Also, to what degree of certainty have you really determined this SAF as constituting the needs-based issues? How valid
and reliable have the various analytic tools been efficient and effective in helping gather the relevant data in delineating
the SAF? Under which condition of certainty is the etiological analysis of the needs assessment planning processes
conducted presents a clear delineation of the assumptions that undergird the needs-based SAF? And, how certain is such
etiological analysis of the needs assessment planning processes embedded in theory-driven outcomes or theoretical
foundations of program use patterns? How have such theory-driven outcomes informed the conceptualization of
measurable action plan in shaping the overall goal, outcome objectives, process objectives, program or project hypotheses,
and activities? All of the above action learning questions thus echo the uncertainty that may surround the public needs
assessment and planning processes.

That is to say that, no public needs assessment and planning processes may run a completely smooth course. In fact, there
may be ups and downs, uncertainties, challenges, efficiency or effectiveness issues with service, program or project use
patterns, and perhaps successes, or it could be that things just did not go as well planned. The important points are that
progress, results-based accountability, and performance reporting, however slow, should continue to be made as the
undergirding principles. Sometimes, the progress, results-based accountability, and performance reporting are just not as
obvious, which is why methods of evaluating different stages and activities in ways pertinent to the service, program or
project use patterns are important. For example, in some situations, it is subjective and not so much measured as could
have been observed.

In other cases, we can quantify our planned action goals or create milestones to measure progress. For example, if our
planned action goal is to introduce fresh healthy foods to all 78 markets in an area of a city, an interim or short-term goal
might be 25 stores in the first year. We can measure our progress against that. An action plan might lay out steps to reduce
community crime through better public awareness. A survey taken at certain points in the project timeline could tell us how
well our awareness program is being understood by the public. Again, we can take a snapshot in time of our progress. In
this unit, we will consider some of the ups and downs, uncertainties, challenges, efficiency or effectiveness issues with
service, program or project use patterns, and the ethnographic implications, as well as the OBE methods we can use to
assess our public needs assessment and planning project’s progress.

Finally, remember that all of the action learning issues covered in this course are intended to introduce you to the process
of exploring the needs-based patterns of service, program, or project use gap that may serve as the backbone for your
future action research dissertation project in the DPA program. For this, the public needs assessment and planning course
is a starting point of exploring possible needs-based patterns of service, program, or project use gap in preparation for
your ultimate action research dissertation project.

Currently, because the scientific merit action research template (SMART) form is the research plan that serves as the
skeleton for your dissertation proposal in the DPA program, it may be good to get introduced to it. Being introduced early
to the SMART form will not only help you better acclimate yourself on what will be expected of you at the dissertation
phase, but also you could perhaps start reflecting on the relevant questions noted on some of the sections as lenses to help
shape your public needs assessment and planning project.

OBJECTIVES
To successfully complete this learning unit, you will be expected to:

1. Explore results-based accountability instrument and performance reporting framework used in determining the measurable outcome of systems changes.

2. Assess methods of performance evaluation and outcomes measurement for public needs assessments.

3. Compare methods used to evaluate uncertainties, challenges, and efficiency or effectiveness issues with service, program, or project use patterns, success, and progress in a public needs assessment project.

4. Explore outcome-based evaluation applications and implications on public needs assessment and planning processes, theory-driven outcomes, and action-oriented research projects.