What constitutes well-designed performance-based assessment?  Can performance-based assessments be designed in such a way that requires students to demonstrate their ability to apply, analyze, synthesize and evaluate what they have learned? 

What role does technology play in performance-based assessment? 

What should the components be for performance-based assessment as related to 21st century skills.  How do you build those types of assessments and why is it important for 21st century education? 

How integral is the role of assessment when evaluating students’  performance around 21st century skills?  

Carefully designed performance assessments can contribute
greatly needed evidence of students’ progress in applying their thinking and
reasoning skills in significant, recurring academic and practical problems. Performance
assessments have been studied for decades and can currently be designed to meet
the highest standards of reliability and validity. Over the next few days, I
will share examples from performance assessment projects I have led to design
writing, reading, history and science assessments, higher order thinking
assessments, and assessments of 21st century competencies. Some
projects were for large-scale assessments, some for classroom assessments. All
focused on moving beyond recognition of disconnected facts or canned
conclusions to engaging students in extended scenarios in which they must construct
their own solutions and products by using their knowledge to solve problems and
achieve goals throughout extended scenarios.

The iconic performance assessment design begins with specification
of the knowledge and skills to be assessed and the criteria for evidence of
proficiency. The design of the assessment tasks can then focus on how to design
tasks to elicit evidence of the targeted core knowledge and strategies.
Typically performance assessments begin with an authentic, engaging problem or
goal that represents an important type of problem in the subject area.  A series of subtasks or assignments
engage students in such 21st century skills as analyzing the goal or
problem, gathering relevant information and data, organizing, analyzing, interpreting,
creating, and synthesizing findings in a culminating presentation. In language
arts, performance assessments may focus on understanding and interpretation of
literature and non-fiction or composing a range of discourse types. In mathematics,
performance assessments may target students’ abilities to apply problem solving
strategies to novel challenges. History/social science performance assessments
may assess interpretations of past or current events. Science performance
assessments may gauge students’ skills in using core ideas to investigate the
natural and designed world.

I invite participants in this forum to share their
experiences with the design and use of performance assessments to monitor
student learning. Some of you will have developed performance assessments as
summative measures of achievement, to document achievement by a point in time.
Others will have developed performance assessments to be used formatively by
teachers to monitor progress and adjust instruction accordingly. What
challenges have you faced? How are you organizing development teams?  What methods are you using? How can we
share our experience and expertise to design and use performance assessments
that will extend and enrich evidence of student learning?

Thank you, Catherine, for giving me an opportunity to
discuss these important questions.

I will be drawing on a number of resources during this
week’s online discussion, but, in order to make this less academic, I will not
be citing these resources in the text. I will post a bibliography on Friday.

I think I will start with the first two questions, but in
the opposite order:

Can performance-based assessments be designed in such a way that
requires students to demonstrate their ability to apply, analyze, synthesize
and evaluate what they have learned? 

What constitutes well-designed performance-based assessment? 

The quick answer to the first question is, yes, performance
assessments can be (and have been) designed in such a way that elicit students’
abilities to apply, analyze, synthesize and evaluate information in the 21st
century.

My answer to the second question is that a well-designed
performance assessment, like any assessment, should include tasks or situations
that elicit behaviors or performances that reveal the knowledge and skills that
are the target of measurement (e.g., ability synthesize and evaluate
information). In the end, one wants to observe performances that support the
inferences and assumptions underlying test score interpretations.

With performance assessments, the behaviors one most often
(but not always) wants observe are those that they would observe in the
“real-world” outside the testing situation – authenticity is one of the
hallmark characteristics of performance assessments. The trick, then, is
designing the assessment tasks so that they, to the extent possible, elicit
behaviors consistent with real-world behaviors. The testing context, of course,
poses many necessary constraints, some technical, some practical, that require
designers to use a principled approach to developing the performance tasks.
Technology can help overcome some of these constraints, but can also, if not
carefully integrated, lead to measurement error.

Evaluating the
trustworthiness of Internet-based or other digital information has become an
essential 21st century skill. The iSkills™ assessment, from
Educational Testing Service (ETS), purports to measure such digital evaluation
skills, along with other digital literacy skills. Examinees solve information
problems through simulated technology, and these tasks are embedded within
scenarios designed to mimic the situations in which college students
demonstrate their skill in locating, managing, and using information.

From 2004 to 2008 I worked with Educational Testing Service (ETS)
to investigate the extent to which iSkills test
scores measuring the evaluate performance area support inferences about the
ability of college students to evaluate information in a digital environment.

The
overarching goal of the work was to investigate whether the iSkills assessment
tasks and naturalistic ICT literacy tasks provided comparable measurement of students‘ evaluation skills. To accomplish this goal, I developed criterion performance tasks
to approximate the context of academic assignments in which undergraduate
students are expected to utilize ICT to evaluate information. As such, I viewed
the criterion performance tasks as “naturalistic” representations of how
students evaluate information in a technological environment. I then compared student
scores and response processes on the iSkills evaluate tasks with their
scores and response process on the naturalistic ICTL evaluate tasks.

The naturalistic evaluate
tasks differed from the iSkills evaluate tasks in two important ways.
First, students select their own ICT (within a computer lab setting) to
complete the tasks, rather than being limited to using specific web browsers
and generic software interfaces (as with the iSkills evaluate tasks).
Second, the context for the naturalistic evaluate tasks was based on in-depth
interviews with undergraduate students about how they evaluate information in a
technological environment, as well as actual assignments from college courses
in which students have to demonstrate their information evaluation skills.

I
proceeded with the development of the naturalistic tasks in an iterative
fashion. Following initial design of the tasks, I interviewed undergraduates
about how they would evaluate information in the context of hypothetical
academic assignments (summaries of actual assignments). Each naturalistic
performance task was designed to elicit multiple scorable observations and take
approximately 20 minutes to complete. Each task consisted of an opening
academic-based scenario describing one of four possible topics (The Number Pi,
Public Smoking Debate, Purchasing Computer, Critical Thinking), as well as
several follow-up steps asking students to use ICT tools on their computer to
review information sources and describe their basis for selecting and rejecting
sources (i.e., evaluate information).

Students had to make
several decisions related to evaluating information in order to complete the
tasks: (a) which information sources to review, (b) how to review the sources
(i.e., as listed in task or actual source/source excerpt), and (c) which types
of information presented with the sources (e.g., author, publication date) form
the basis for selecting or rejecting the source. It was the last of these
decisions that students were asked to describe; their descriptions were
assessed via the scoring rubric.

Well designed performance based assessmets are those criteria that can be developed in classrooms, schools and districts. There is a need to evaluate students performance, that means skills, knowledge and atitudes in the classroom and out from it. Criterion-referenced evaluation provides students with clear statements of criteria for learning so they know exactly what quantity and quality of work is expected. Student performance is compared to the established criteria based on learning outcomes developed for each grade and in that way designed the curriculum. There is a need to establish what are expected from each student to learn and to be able to do, rather than set up a curriculum based on contents where there is only assessement considering a benchmark rather than considering the student, or the performance of other students. Criteria are the aspects of a product, process and demonstration which are used to guide, monitor and evaluate learning. Teachers usually have their own internal criteria, and this is mostly based in their own educational experience, trying to standarize education in a classroom. That is why the need to develop a criteria that can be relevant, widely held and shared with the students is a need in our schools today. So, to assess learning and to guide teaching, teachers must be able to tell what degree the students are able to meet the criteria. Criteria may be established by teachers, individually, or collaboratively, and then explained to students. Or even teachers and students may work together to generate and establish criteria. Performance or rating scales delineate the criteria for different levels of performance from unsatisfactory to excellent, and that gives teachers the guide to know how to increase those week areas in their classroom.  Technology helps a lot! It gives teachers the possibility to work with different way of assignments out from the classroom and watch how children are able to cope with them. It also gives the teacher the opportunity to learn on different ways of performance of their students, not only face to face.  Using a criteria developed under the desirable outcomes, and shared with students benefit them because they feel a real sense of ownership of their learning, they know what to expect, what standard for excellence is before begining, it gives support and structure for them, they also learn self-evaluation.  For teachers it means the basis of their job because they can set clear goals on their daily practice, because they know the strenghts and weaknesses of their students and planning becomes easier. 

The way we conceive education today is different from those in our school days, where grading based on tests were all about knowledge, today there are more than that and we have the resources to do it. Our schools may be different, our mission as educators must be different, and our students will change the world where they will live in.

The new Technology and Engineering Literacy Framework for the 2014 National Assessment of Educational Progress (NAEP) presents content targets and many ideas for tasks to asess Information and Communication Technologies (ICT). The ICT targets are based on the ISTE and 21st Century Partnership frameworks. Assessment areas include Construction and Excange of Ideas and Solutions, Information Research, Investigations of Academic and Real World Problems, Acknowledge of Ideas and Informaton, and Selection and Use of Digital Tools. The 2014 NAEP will be delivered entirely by computer. It will administer long (25 minute) and short (12 minute) scenario-based tasks. There is a CD with samples of interactive tasks. (See http://www.naeptech2014.org) Both classroom teachers and assessment programs may find this framework, the specifications and CD useful resources. Technology can not only support the administration and scoring of performance assessments, it can provide opportunities to design assessments that allow studetns to engage in active problem solving, online collaboration with distant experts and peers, and multimedia communicaiton. Are there examples in your schools of taking advantage of technology to assess using technology tools to solve problems?

Edys, Maria, and Eric make excellent points about assessment.   The issue is that they join a chorus of excellent thinkers in this field, and a mountain of research that affirms their statement.  The issue is – how does one collect this, personalize it, assess it, remix it, reuse it, save it, tag it, find a community of peers on demand?  This topic, like so many brought up at gatherings like this, need a place to go.  Could be here at GetIdeas;  could be one of hundreds of other places.

Assessment of students (at least in the United States) is connected to quantitative and qualitative factors, independent and dependent variables, culture and context.  In the United States, we are clamoring for standards around these assessment.  While the debate is healthy, why not federate the search of assessment practices so that a given teacher could have access to information s/he can hone down, as well as experts who can be available?  Just as all great classrooms require both content and community, let’s not forget that the “answer” to these issues of assessment are more a function of cloning folks like Edys, Maria, and Eric (commenters, above) and building this kind of community so that every teacher has a colleague, close by.  Let’s take all these great ideas, pool them, and make them available.