Tag: transparency

Threshold Concept: Assessment Literacy

June 22, 2017 by

Courtesy of Allison Shelley/The Verbatim Agency for American Education: Images of Teachers and Students in Action

This is the nineteenth article in a series leading up to the National Summit on K-12 Competency-Based Education. We are focusing on four key areas: equity, quality, meeting students where they are, and policy. (Learn more about the Summit here.) We released a series of draft papers in early June to begin addressing these issues. This article is adapted from Fit for Purpose: Taking the Long View on Systems Change and Policy to Support Competency Education. It is important to remember that all of these ideas can be further developed, revised, or combined – the papers are only a starting point for introducing these key issues and driving discussions at the Summit. We would love to hear your comments on which ideas are strong, which are wrong, and what might be missing.

“Student assessment is essential to measure the progress and performance of individual students, plan further steps for the improvement of teaching and learning, and share information with relevant stakeholders.” – OECD, 2013

Assessment literacy is important for practitioners but it is also important for policymakers and stakeholders throughout the system to understand the roles that different types of assessment play in student learning, how assessment and moderation are used to comparatively and fairly judge student mastery, and how the information generated by assessments can be used toward a cycle of continuous improvement in teaching and learning. The lack of assessment literacy across the system is a major blind spot. Thus, building significant capacity for assessment literacy is needed to advance new competency-based approaches and address tough issues in our current system.

An important concept in assessment today is related to the concept of comparability. Comparability is defined as the degree to which the results of assessments intended to measure the same learning targets produce the same or similar results. This involves documenting the reliability of judgments and not assuming that comparability is stable over time or invariant across multiple subgroups such as English language learners and special education students.1

There are unique circumstances in the U.S. education system that have driven the need for much greater degrees of comparability than is true in most other nations. When the federal government became involved in K-12 education with the Elementary and Secondary Education Act of 1965, it was in direct response to deep inequities that remained even after school segregation. Because of the history of inequities in education offerings among student groups, concerns for equity are much greater than in many other countries, which drives, to a significant extent, the degree to which we need to take greater care that measures are fair and have common meaning among students, schools, and districts.2 This drives the prevalence of standardized tests in our country, causing the concept of assessment to often be conflated with the end-of-year, statewide, summative accountability tests.

Practitioners working deeply in competency-based learning models realize quickly how our K-12 education systems lack systems for calibrating the quality of student work, so we know that fundamentally there is significant consistency across schools and systems. As much of a systems challenge as this would appear across the states in the U.S. today, building professional educator capacity and policymakers’ understanding of assessment literacy is fundamental to shifting to personalized, competency-based systems at scale and focusing on equity.

A common misconception about assessment literacy is that it is only about how to interpret standardized test results. In contrast, assessment literacy is a much broader and more significant concept. The New Zealand Ministry of Education defines assessment literacy as:

“the possession of knowledge about the basic principles of sound assessment practice, including its terminology, the development and use of assessment methodologies and techniques, and familiarity with standards of quality in assessment. The primary purpose of assessment is to improve students’ learning, as both student and teacher respond to the information that it provides. Information is needed about what knowledge, understanding, or skills students need. By finding out what students currently know, understand, and can do, any gap between the two can be made apparent. Assessment is the process of gaining information about the gap, and learning is about attempts to reduce the gap.”

Personalized, competency based learning requires us to reorganize systems around doing what it takes to ensure every student is attaining mastery, rather than the ranking and sorting them into high achievers and low achievers that is created through variable A-F grading practices. Redesigned systems will need to build capacity for clear evaluation criteria to make valid and reliable comparisons of students’ progress against outcomes (commonly understood outcomes) using evidence and common rubrics.

Thus, progress isn’t measured by ranking and sorting kids against each other, or through grading “curves,” but instead for each student to measure their evidence against articulated, high-level, common expectations of success and with clear depictions for what success looks like. This process of developing clear expectations for common proficiency levels is a key part of a “calibration.” Calibration is a process that allows two or more things to be compared via a common standard (e.g., a weight in the physical sciences or commonly scored papers in an education system). The purpose of common performance tasks given to students by different schools and districts is to serve as a “calibration weight;” a way to compare the way one school or district scores students on the common task, with the way other schools and districts score those same students’ work. In order to use the common performance tasks as calibration weights, districts need to re-score other districts’ common performance tasks. Calibrating expectations as well as grading and scoring processes for learning goals, is very important in competency-based learning systems. Calibration may involve groups of educators who collaborate and develop consensus around rubrics for scoring student work. The calibration process makes scoring student work consistent and more aligned to the standards upon which rubrics and scoring criteria are based, as well as creating reflective processes focused on improving student learning.

In addition to calibration processes for consistently and accurately evaluating student work, assessment literacy also includes knowing which assessments are appropriate for what purpose (e.g., formative, progress monitoring, or summative). This idea of common expectations, and evaluating evidence against common standards and rubrics to build and evaluate comparability across schools and systems, requires careful  moderation of assessment practices across the system and perhaps across the state level. Professional development of educators to assess student evidence using calibration processes and developing rubrics with scales for evaluating performance tasks against criteria, is central to building the capacity needed in a competency-based education system. A competency-based learning system that offers personalized pathways for students to meet learning goals and learning targets must rely on multiple forms of evidence against common standards and expectations.   (more…)

Transparency and Trust

March 14, 2017 by


This article is the tenth in the Designing Performance-Based Learning at D51 series. A reminder: D51 uses the phrase performance-based learning or P-BL.

Opportunity to Learn

Like most districts, transparency hasn’t been a strong point at D51 in the past. Thus, with transparency being a core value of performance-based education, there are trust issues that will have to be worked through. D51 knows it is important to provide teachers with the chance to understand and learn how to use the T&L Framework and effective practices. Any tools being developed are being designed to support growth – not evaluation. Rebecca Midles, Director of Performance-Based Learning (P-BL), noted that, “We are moving step by step and need to constantly communicate about our timelines and sequencing. Understandably, educators are wondering how the performance-based system will impact them. We are trying to be very clear about whether something is going to be evaluative or not. Eventually the T&L Framework will help us create the foundation for strengthening our human resource processes. But only when we are ready and only after teachers have had the opportunity to learn.” Opportunity to learn standards: an important piece of competency-based education for students and adults.

Preparing for Angst

Bil Pfaffendorf, a professional learning facilitator, mentioned the double edges of transparency. “The sense of trust is changing in D51,” he said. “There is more dialogue, people are sharing their opinions, and they are starting to feel confident that those at the district level are listening. We are all trying to be transparent, which is difficult in the midst of so much change. Transparency is important in building trust. It can also lead to anxiety. If teachers understand the expectations but don’t have the skills yet to do it, anxiety and angst are totally understandable feelings. So we are thinking about the the social and emotional learning of our teachers as we design the labs.”

Angst and anxiety came up several times during my visit. In a discussion, one teacher emphasized, “The level of professional engagement of our teachers is very high. Some are anxious because they recognize they have a lot to learn. Some may even be in cognitive overload as they wrap their heads around what it means to personalize their classrooms. Their can-do attitude is a beacon. It’s inspirational.”

Midles explained to me later that when educators start to feel anxious, it is often for one of two reasons. First, they may feel the expectations of their job are changing or they may not have the skills to excel. Thus, the trust-building response needs to be an assurance that there will be supports provided and that adults will not be evaluated until expectations are clear and they have had an opportunity to learn. Please note, this is the same principle used for students.

Second, anxiety and angst may build up when teachers feel out of control or that new expectations of compliance and control are being layered on top of their jobs. Midles referred to the Csikszentmihalyi model of flow in thinking about the mix of challenge and ability to strengthen educators’ and students’ relationships to learning. A middle school teacher, Darren Cook, explained to me that teachers have endured at least a decade of sweeping new reforms only to be replaced by the newest sweeping reform. With the introduction of the state-teacher evaluation policies that are not rooted in the culture or strategies of the district or their schools, teachers have become even more suspicious of changes. The trust-building response here is to make sure that teachers understand that as the district creates a more intentional common Teaching and Learning Framework, teachers will actually have more autonomy and opportunity for creativity in the strategies and learning experiences they use to help students learn. The other response is to offer opportunity to learn about performance-based learning not through memo or lecture, but through engaged reflective learning as well as opportunity to participate in creating the new system. (more…)

WordPress SEO fine-tune by Meta SEO Pack from Poradnik Webmastera