Application Knowledge Assessment I

Assessment Tasks, like instructional tasks, are response-demand activities that function as explicit manifestations of students' implicit and unobservable cognitive processes. Assessment Tasks for application-knowledge objectives are designed to cue students' implicit cognitive process of recognizing whether novel objects or actions are examples of the generalizations that are appropriate for the objectives.

Teachers will usually need to engage in diagnosis of students' performance of Assessment Tasks in order to determine students' understanding, which will enable them to adapt their instruction to students' understanding. Diagnosing students' performance of Assessment Tasks is less complex with a lower-order cognitive process like recall knowledge than it is with a higher-order cognitive process like application knowledge. Recall knowledge involves the retrieval of information encoded in long-term memory in much the same way it was originally presented as sensory input. Diagnosis involves simply determining whether the retrieved information is correct For instance, here are two Assessment Tasks for recall-knowledge objectives. The first is knowing about leaves, and it was shown in a more complete form in an earlier posting.

Here is an Assessment Task for the recall-knowledge objective of knowing about thermodynamics.

These two Assessment Tasks are not adequate for application knowledge because they do not ask students to recognize whether novel objects or actions are examples of a generalization. They do not require the transfer of previously encoded information to novel objects or actions, ones not encountered previously. Here is an Assessment Task for the application-knowledge objective of knowing what a summary is. Students are assigned to read an article in their science texts that is entitled, "Treasure on the Ocean Floor." It is three pages long. They have not read it before. When students finish reading the teacher projects this exercise on the screen.

Notice that the novel objects in the Assessment Task consist of examples of three possible summaries for the science article. The task is intended, first, to cue students' retrieval of information about the generalization summary from their long-term memories, and then, second, to have them transfer that information when choosing the statement that best exemplifies an adequate summary of the article. The directions identify the generalization to be retrieved. The sentences are the novel objects to which that generalization summarization should be transferred. Notice that students have a one-in-three chance of simply guessing the right answer. Or they might not use the intended generalization at all, such as picking out the third statement, which is the correct one, because it is the longest. The only way teachers are able to determine if students possess an adequate application knowledge of the generalization summary is to use an active diagnosis consisting of asking them follow-up questions, such as "Why do you think the third one is the best summary? What is wrong with each of the other two summaries?" The key to teachers' accurate diagnosis is asking questions that are based on the teachers' mental model of the implicit cognitive processes involved in students' application knowledge. Teachers use the mental model of application knowledge like a map in locating where each student is relative to the map.

Teachers' diagnostic ability is a significant factor in the effectiveness of their instruction [1]. Some even regard teachers' diagnostic ability as one of four major components of teaching expertise [2]. There is evidence that teachers' diagnostic ability contributes significantly to students' academic achievement [3]. Teachers obviously must possess an adequate knowledge of the subject matter they teach. However, teachers with high levels of subject-matter knowledge often overestimate students' actual understanding [4]. Teachers' overestimation may be caused by a tendency to use their own subject-matter knowledge as the model against which to judge students' performance, thus they think that if they understand then their students must also [5]. Teachers' diagnostic ability will be the greatest when it is based both on their subject-matter knowledge and on their mental model of the implicit cognitive processes students are being asked to use, such as application knowledge.

Incidentally, with reference to the Assessment Task for summarization described above, asking students to read the story and write their own summary, even though it is a less wordy task, stull requires the more complex implicit cognitive process of procedural knowledge. Instruction for the procedural knowledge of summarizing is most effective once students have already learned an application knowledge of summaries. They will have a great deal of difficulty learning to write their own summaries before they have even learned to recognize a good summary when they see it. For instance, think back to the novice-expert physics study described in an earlier posting, where the novices were unable to recognize the physics generalizations exemplified in novel physics problems shown on cards. Because the novices were unable to recognize the examples of the physics generalization shown on the cards, they would not be able to solve those physics problems unless they were told which physics generalization to use with each physics problem. Until they learn application knowledge of those physics generalizations, instruction for a procedural knowledge of them will be ineffective.

____________________

1. (For example). Shepard, L.A. (2001). The role of classroom assessment in teaching and learning. In V. Richardson (Ed.), Handbook of research on teaching (pp. 1066-1101) Washington DC: American Educational Research Association.

2. (For example). Weinert, F.E., Helmke, A., & Schrader, R.W. (1992). Research on the model teacher and the teaching model. In F.K. Oser, A. Dick, & J.L. Patry (Eds.), Effective and responsible teaching--The new synthesis (pp. 249-260). San Francisco: Jossey-Bass.

3. Seidel, T., & Shavelson, R.J. (2007). Teaching effectiveness research in the last decade in disentangling meta-analysis results. Review of Educational Research, 77, 454-449.

4. Nathan, M.J., & Petrosino, A.J. (2003). Expert blind spot among preservice teachers. American Educational Research Journal, 40, 905-928.

5. Nickerson, R.S. (1999). How we know--and sometimes misjudge--what others know: Imputing one's own knowledge to others. Psychological Bulletin, 125, 737-759.