+353-1-416-8900REST OF WORLD
1-800-526-8630U.S. (TOLL FREE)

Assessing Learning Outcomes

  • ID: 40846
  • Report
  • January 2002
  • 81 Pages
  • American Productivity & Quality Center, APQC
1 of 4
If a key element of your organization’s mission is to teach, there is no outcome more important than elicited learning. Whether a college or a corporate training division, learning is the intended product of instruction. Most teaching institutions today have accessible data on the number of hours spent in teaching the credit hours and degrees generated the proportion of instructors with doctorates the grade point averages of the students and even the percentages of entrants who obtain the targeted certificates, licenses to work, and degrees. But none of these outputs is a valid or reliable measure of what was learned.Those who need to know the effectiveness of an instructional service-productionmanagers, university administrators, students or employees whose learning is being measured, teachers, tuition-paying parents, or other stakeholders-must understand what was learned if they are to make intelligent judgments regarding the time and monetary value of the instruction. What is learned in various instructional programs differs in substantial ways even among organizations with the same or similar educational missions. Is the primary goal the accumulation of knowledge or should the student also learn how to apply that knowledge appropriately in future work settings or citizenship roles? Should greater priority be given to enabling individuals to become highly effective learners early on or covering what specialists consider the most important knowledge in their fields? What balance between generic capabilities and occupationally specific skills should a particular instructional program achieve? To succeed, leaders of an institution need to be clear about what learning they strive to evoke and then determine by sound assessment how well they are meeting that goal.Assessing the end-of-course knowledge and skills a student or trainee possesses and establishing what part of that was learned in the course or training program are two completely different tasks. Measuring learning gains requires preinstruction measures to compare end-of-instruction results. Proving the gains takes additional time and money and presents technical challenges in legitimately ruling out causes other than the instruction. Each of these requirements of good assessment presents a challenge to those who teach. In addition, students in college and employees in training vary enormously in characteristics that affect their potential for learning. A recent study on risk and promise among working adults found that at least 18 factors besides traditionally studied demographic differences have statistically significant effects on such measures of success as grades, ratio of courses completed to courses attempted, and semesters completed. Mapping issues that teaching and support personnel face in helping such diverse students succeed reveals the inadequacy of programs built on the idea that one size can fit all.The diversity of student characteristics interacts with the diversity of environments in which students and employees learn. The normal working adult has at least three primary environments impacting the likelihood of effective learning-work, home, and school. Thus, deciphering what learning may be attributed to the education and training program is an urgent and difficult undertaking.Study ScopeThe following scope defines the content and structure of the benchmarking study. Sponsors spent a day-and-a-half collaborating with the Apqc project team and subject matter expert Dr. Morris Keeton to create this scope. Focus Area 1: Planning for Assessing Learning Outcomes- Starting with self-evaluation- Creating/improving an organizational plan for assessing learning outcomes- Deciding what learning outcomes to assess- Focusing on a variety of learning outcomesFocus Area 2: Deciding How Best to Assess- Using diverse, appropriate methods of assessment- Choosing, preparing, and supporting the assessors- Determining when and how often to assessFocus Area 3: Managing the Assessment Process- Using and applying assessment findings- Coordinating assessment with other functions within the organization- Coordinating assessment practice and results with other organizationsStudy KEY FindingsFollowing are the 11 key findings that emerged from this study, divided into three topical sections that follow the study scope. The findings will be explored in detail throughout the remainder of the report.1. Good assessment plans are strategic in nature. They clarify the purposes of the assessment activities and tie each to the organization’s mission, vision, and key goals.2. Widespread involvement of all stakeholders, established early and maintained over time, yields an organizational culture that embraces assessment.3. The adoption and implementation of an assessment plan is best begun promptly when the need is recognized and then allowed to evolve slowly. It is important to balance the need for buy-in with the time required for a sound implementation.4. In-depth analysis and periodic review of the needs and interests of internal and external stakeholders drive the choice of which learning outcomes to assess and how they are assessed.5. The use of multiple methods of assessment can enhance reliability. Additionally, to ensure that a process is valid and measures what it is intended to measure, each activity and instrument should be tied to its purpose and the strategy for achieving that purpose.6. Integrating assessment with other ongoing performance improvement efforts within an organization enhances the long-term viability of the assessment program and its usefulness to the overall organization.7. Successful organizations take a decentralized approach to assessment, pushing responsibility and ownership to those on the front lines.8. Assessment is integral to learning and most effective when included as a responsibility for each member of the organization, as opposed to being an add-on effort.9. The primary purpose of obtaining and reporting assessment findings is to improve the organization and, in particular, its employees’ and students’ learning. Accordingly, the findings are best used in non-punitive ways. 10. Educating those who will use the assessment data is the key to shifting the focus of assessment from the data to an overall process.11. Best-practice organizations continually communicate the assessment activities and results to their constituents.Benchmarking MethodologyThe past decade has seen wrenching change for many organizations. As firms have looked for ways to survive and remain profitable, a simple but powerful change strategy called “benchmarking” has evolved and become popular. Benchmarking can be described as the process by which organizations learn, modeled on the human learning process. A good working definition is “the process of identifying, learning, and adapting outstanding practices and processes from any organization, anywhere in the world, to help an organization improve its performance.” The underlying rationale for the benchmarking process is that learning by example, from best-practice cases, is the most effective means of understanding the principles and the specifics ofeffective practices. Benchmarking is not a fixed technique imposed by “experts” but rather a process driven by the participants who are trying to change their organizations. It does not use prescribed solutions to a problem. Instead, it is a process through which participants learn about successful practices in business, healthcare, government, and education and develop solutions most suitable for their own organizations.Benchmarking is not copying, networking, or passively reading abstracts, articles, or books. It is action learning, not simply a comparison of numbers or performance statistics. Numbers are helpful for identifying gaps in performance, but true process benchmarking identifies the “hows” and “whys” for performance gaps and helps organizations learn and understand how to perform at higher levels.
Note: Product cover images may vary from those shown
2 of 4


3 of 4
4 Sponsor and Partner Organizations

A listing of the sponsor organizations in this study,
as well as the best-practice (“partner”) organizations
that were benchmarked for their innovation and
advancement in assessing learning outcomes.

6 Executive Summary

A bird’s-eye view of the study, presenting the key findings
discovered and the methodology used throughout the
course of the study. The findings are explored in detail in
following sections.

15 Key Findings

An in-depth look at the 11 key findings of this study.
The findings are supported by quantitative data and
qualitative examples of practices employed by the partner

69 Partner Organization Profiles
Background information on the partner organizations, as
well as their innovative learning outcomes assessment
Note: Product cover images may vary from those shown
4 of 4
Note: Product cover images may vary from those shown