Assessment Analysis, Communication, and Feedback

Item Analysis

Although time consuming, a detailed item analysis reveals helpful information about students as well as assessments.  Strategies to analyze a selected response assessment are described below.

Selected Response Analysis

Number down a sheet of paper.  After marking the assessment, tally every time a question is incorrectly answered. See Figure 5.

Figure 5: Example Item Analysis

SchoolCenter Picture


Clearly more students missed questions 5, 8, and 11 than any other.  This begs for further inquiry regarding the missed questions.  Was the content addressed by items 5, 8, and 11 taught? If so, which students learned it?  Were the skills and strategies required to answer items 5, 8, and 11 practiced?  If so, who can apply them?  Were the items, including distracters, constructed well? Were the items accessible to all students (format, language, etc.)? Which, if any, of these items require re-teaching?  What clues do students responses provide in planning any necessary re-teaching?

Rubric Analysis

Also, teachers analyze results of a rubric based assessment across the class by tallying the results. Start with a blank rubric. After scoring all the assessments, flip through and tally each time a student is rated in that standard.  Below is a sample rubric for a presentation (see Figure 6)

Figure 6: Rubric Analysis

Total ____ / 9

1

2

3

Presenter must speak in a loud and clear voice.

Presenter was unable to be heard or understood by the audience.

Presenter was loud enough but not speaking very clearly.

Presented with a loud and clear voice.

Presenter will make eye contact with the audience and use note cards only to stay on track.

Presenter read from note cards during the entire presentation.

Presenter gave some effort to make eye contact, but read off the note cards more often than not.

Presenter made good eye contact and used the note cards only to stay on track.

Presentation followed a logical order and was easy to understand.

Presentation had no logical order and had no main point.

Presentation was understood, but not in a logical order.

Presentation was presented in a logical order and understood by the audience.


Analysis: Students understand how to prepare a speech and deliver it in a logical order.  Most students are reading from their note cards during the presentation.  Finally, a teacher can infer that students are evenly split when it comes to using a clear speaking voice.

Using item analysis information

Technology can help teachers analyze assessments by performing extensive calculations instantly.  Table 6 shows various calculations that can help the teachers extract information from an assessment that informs instruction. 

Table 6: Calculations and Information Learned

Calculation

What it can tell you...

Class Average

Averaging the scores of the entire class provides very little information as it pertains to informing instruction.  When compared across classes taking the same assessment, an average score can be used as a general indicator as to which class performed better.

Percent Correct by Question

Calculating the percentage of students that responded correctly to each individual question can help a teacher determine whether the whole class has reached an understanding of the content or if re-teaching is necessary. This should be followed by a review of individual student data to determine which students need to be retaught and which students are ready to move on.

Percentage of Each Answer Responded

Within each question calculate how many students answered each of the possible responses.  For example, how many students answered A, answered B...etc.  Writing quality distracters can help a teacher gain more insight on how students came up with the answer they chose.


Feedback

In order for student assessment information to improve curriculum, instruction, and student learning, accurate analysis of assessment results must be followed by feedback to the student and teacher. In Classroom Instruction that Works, Marzano (date) identified four generalizations to guide the effective use of feedback.  Feedback that positively impacts student learning is:

criterion-referenced

related to specific knowledge and skills

student-centered, student-involved

timely with additional opportunities to perform


Feedback at the beginning, during, and at the end of the instructional process are all necessary to provide students the best learning opportunities.

Feedback at the Beginning of the Instructional Process

Information gained from pre-assessments, as well as diagnostic testing  yields impressions of students' knowledge and skills related to the content. Information from this type of assessment establishes a  starting point for the teachers. Teachers determine which studentsneed help acquiring essential skills and which students are ready for extended learning. Teachers ask themselves: Do the students have the prerequisites to plunge into the forthcoming unit of learning, or does time need to be spent on building a foundation or reviewing prior knowledge and skills? Do students already have a firm grasp of concepts and therefore need instruction that allows them to develop deeper understanding of the curriculum?  While feedback from preassessments is used primarily to inform instruction, it can also be used to help students form detailed  personal learning plans.

Timing of preassessments should be considered to assure accurate and meaningful feedback. The timing of a unit-level preassessment should be as close to the instruction as possible while also allowing the teacher enough time to monitor and adjust. 

Feedback to the Student During the Instructional Process

The goal of feedback is to increase individual knowledge, skills, and achievement. For students to learn from their confusion or miscues, they must be aware of what they have accomplished as well as what they need to work on. Providing descriptive feedback, either in written or oral format, is followed by opportunities to practice and demonstrate understanding and skill.

Feedback to the Teacher During the Instructional Process

Assessment during instruction offers teachers an analysis of teaching strategies and methods. Is the instruction sound and effective? Is re-teaching for all or intervention for some students necessary? Asking these questions can help teachers determine whether more time using the same method of instruction is enough, or if the instructional method needs to be different.

The Dufour's Professional Learning Community structure provides teachers with contexts for analyzing common assessments. Every teacher on a team works together to identify the most important common essential outcomes.  Common assessments are created to assess student performance against the common essential. After assessing, teachers share results and co-construct interpretations of these results.  From the analysis they learn about successful practices and attempt to apply them in their practice.  The Professional Learning Community conversations also inform student grouping practices and curriculum areas that require more or different attention. 

Feedback at the End of the Instructional Process

Feedback at the end of instruction informs the student, parents, teacher, and school system how well the goals of instruction have been met. For the student, reporting of summative assessments ranges from a grade on a single assignment to earning a credit for a class. For the teacher, progress of the individual student as well as effectiveness of their teaching can be measured. Analysis of trends in student performance informs teachers as they ensure all students master a guaranteed and viable curriculum.    

Student Involvement in Assessment

Student involvement in assessment is key to unlocking the potential of assessment as a learning tool. Below are questions to consider while planning to include students in assessing their learning.

How do teachers assure that assessments will be useful measures of learning?

Construction of useful assessment begins with learning targets that are specific and written in accessible language. Teachers limit the number of learning targets so that students are not overwhelmed and student have ample and flexible time with scaffolding to master essential content and skills. For example, when trying to teach 3rd graders how to write a good expository paragraph, consider only including 3 or 4 learning targets:

I know how to write a topic sentence that tells the general idea of the paragraph.

I can give specific examples and details to explain the general idea.

I can stick to the topic.

How do teachers encourage students to assess their work?

Well-written rubrics allow students to assess their work. Rubrics useful to learners:

Focuses on only a few learning targets

Contains student-friendly language

Provides descriptive detail that defines various levels of success

Descriptions of qualitative differences in the work are more important than the quantity in most rubrics. It is tempting to just use numbers to distinguish levels in a rubric. Students are taught how to use the rubric to evaluate their work.  A helpful starting point is to have students use a rubric to assess model or anchor papers. 

Begin with the extremes-a well constructed product and a poor sample of student work. Allow students to identify the strengths and weaknesses in the products and formulate a rubric from their suggestions.  Together, practice the strategies students will use to improve their work. Then, progress to analyzing mid-range products (2s and 3s on a 4-point rubric scale). Focus on one learning target at a time when examining sample products so students are able to isolate the intended learning.  Before students practice scoring and improving their work, have them work in pairs on anonymous student work.  When students learn how to give themselves descriptive feedback about their conceptual understanding of the content is realized. ( http://www.nycenet.edu/NR/rdonlyres/5CF749A8-D90F-4646-BEAF9DD3130EB82E/2716/AppendixC.pdf )

How do teachers develop relationships with students that promote student involved assessment?

It is essential that teachers first establish a close, trusting relationship with students.  The skills that DuFour describes for building a Professional Learning Community with colleagues are also useful when establishing relationships with students.  Being present, and open, listening without judging, seeking common understandings, and viewing learning as mutual all lead to the development of a positive collaborative atmosphere.  A teacher should seek to inspire students through their own passion for ongoing growth and show young people why they should be committed to their own development.  Teachers should ask themselves, "How can I develop a sense of safety that enables and encourages students to take risks and reattempt tasks at which they have failed or been incapable of completing in the past?" and  "What grading practices motivate students to continually strive for mastery and excellence?"

How do teachers set up situations so that students are more involved in determining the course of their learning?

A powerful way to involve students in their learning is to systematically engage them in shared goal setting. In order for students to set goals and monitor their progress, they must clearly understand what learning is essential.  When students are involved in setting goals and have a clear understanding of their mission, they have a sense of ownership, self-awareness, and control of their own development over time.  Some questions for students might be:  "What are your strengths and weakness in the discipline at hand?" and "What do you see as the greatest challenge for you as you move towards mastery in this area?" and "How will this goal inform your next steps as a learner?"  As a result of this inclusive process, students are intrinsically motivated to take the necessary steps to achieve those goals.

What are some ways that students can use communication skills to empower themselves in their learning?

Portfolios "provide an ideal venue for getting [students] to take notice of, keep track of, and celebrate their learning. Collecting, organizing, and reflecting on their own work builds an understanding of themselves as learners and nurtures a sense of accomplishment. Becoming reflective learners, developing an internal feedback loop, learning to set goals, and noticing new competencies and new challenges are all habits of thought we can cultivate in students through the use of portfolios." (Stiggins, 2004) Students also benefit when they communicate their parents about their progress. Student-planned and lead conferences with teachers and parents creates a shared, interdependent audience for student self-assessment. Selecting work to discuss, talking about their strengths and weaknesses, and sharing goals helps students to process learning. Teachers might guide students in preparing for the conference by supplying a framework like the one below. 


Figure 7. Framework for a Student-Led Conference

SchoolCenter Picture

(Stiggins, 2004)


SchoolCenter Picture