���˴�Ƭ

Instructors with remote-delivery final exams may now face new challenges in order to accurately assess learning, manage accommodations and time zone differences, and address academic integrity concerns. This recorded webinar and resource describe general design considerations for remote exams and explain how to use the ICE model to design a common framework that ensures single and multiple exams are accurate assessments of student learning outcomes.

Sections:

• Webinar: Designing Remote Final Exams
• General considerations about offering exams remotely
• Using a Table of Specifications to Develop Exam Questions
• The ICE Model
• Using ICE in a Table of Specifications
• Frequently Asked Questions

Webinar: Designing Remote Final Exams

with Robin Attas, Centre for Teaching and Learning
This session was recorded on Thursday, March 26, 2020 from 2:30 - 3:30pm using Zoom Video Communications

Instructors with remote-delivery final exams may now face new challenges in order to accurately assess learning, manage accommodations and time zone differences, and address academic integrity concerns. This webinar will describe general design considerations for remote exams and explain how to use the ICE model to design a common framework that ensures single and multiple exams are accurate assessments of student learning.

General Considerations ���˴�Ƭ Offering Exams Remotely

Following are some general ideas compiled from a variety of institutions across Canada and the United States. Individual faculties, schools, and departments may have other guidelines; check with your home department/faculty for more information on specific guidelines.

Timing and accommodations logistics

If you were planning a 3 hour exam, consider reducing it to 1.5 hours so that students with time accommodations will still complete within the scheduled exam time.

Offer a longer availability window (e.g., 48 hours) and once students begin, they’ll have the correct amount of time to complete the exam. This helps students who are now in different time zones, although keep in mind that students with multiple scheduled exams may not be able to take advantage of the extended window.

Write multiple exams for different groups of students. You might group according to time zone or according to amount of time received under accommodations (e.g. no additional time, time-and-a-half, double-time).

Academic integrity

Clearly communicate what’s allowed and not allowed to students (e.g., working with other students, sharing questions or information with other students, posting questions online, using particular materials, impersonating another student for the purposes of writing the exam). Appeal to elements of transparency, fairness, and the learning relationship between instructor and students. The University of Saskatchewan has a great available.

Consider having a question at the start of the exam where students agree to follow academic integrity rules.

Make a large question bank, and randomize distribution to each student so that students are less able to share questions/answers with each other. However, keep in mind that you shouldn’t use this question bank again in future years.

Limit the number of questions displayed per page and don’t allow students to go back to previous pages once they’ve submitted answers. This is not ideal from a pedagogical perspective, but it will reduce the ability of students to share questions and answers.

Remind students that we are all supposed to be social and physically distancing, so that’s one more reason they shouldn’t get together with each other in person to write the exam.

Using a Table of Specifications to Develop Exam Questions

Whether you’re writing a single exam with a deep question pool, a take-home exam, or multiple equivalent exams, a table of specifications can help you plan out the overall format and scope of the exam.

A table of specifications is a method for planning out the format and questions in your exam so that you are measuring exactly what you want to measure, both in terms of content and the way students engage with the content. It ensures that you will assess learning to the same level of complexity across multiple exams, meaning that the exams can be considered equivalent in terms of student grades. Even if you are only planning a single exam, a table of specifications can be helpful. For instance, if you are giving students a choice between two or more questions, it’s important to write questions that assessing student learning to the same level of complexity. Or if you are developing a pool of questions that will be assigned randomly to students, you’d want to make sure all questions in each pool are assessing the same learning.

Ideally, your exam planning will be done alongside a consideration of what was emphasized in the course itself. For instance, if students practiced a lot of identification-type questions throughout the semester, you would ask them to do the same thing on the exam. If you spent ten weeks on one topic and only two weeks on another, you’d expect to have the final exam topics distribute in a similar proportion.

The ICE Model

To get both the content and the way students are expected to engage with the content all into one exam, it’s helpful to align a table of specifications with a learning framework. Here we’ll use the ICE Model, developed by Sue Fostaty Young and Robert J. Wilson (2000). ICE stands for Ideas—Connections—Extensions. Ideas are basic facts or steps in a process; connections are relationships between facts or between facts and the learner; and extensions are hypotheses, applications, or extrapolations to new situations. The diagram below gives some examples of ideas, connections, and extensions for a variety of disciplines.

Using ICE in a Table of Specifications

With the ICE model selected as a learning framework, now it’s possible to use a table of specifications to  combine content and what students need to do with the content into a single table to design an exam (PDF, 77KB).

A table of specifications has three main components:

Content column

• the topics you’ve covered in the course and that you want to test. You can be as fine-grained or as general as you wish.

Ideas–Connections–Extensions columns

• the total points you will assign to ideas-, connections-,  and extensions-type questions with each of those topics

Totals column and row

• allows you to see how your plans distribute the assessment proportionally
• does your weighting of ideas, connections, and extensions match the learning outcomes of the course? the learning activities you emphasized all semester? your hopes for this final assessment?
• does the distribution of content match the distribution within the course?
• make sure everything adds up to 100 in the bottom right corner of the table!
• By planning everything in this table, it’s possible to then write multiple exams, or multiple questions within a particular segment, that are equivalent in terms of the assessment of student learning. The table can be adapted to suit your own needs as an instructor. In the two examples below, you’ll see that individual instructors can come up with quite different results.

Business Example (PDF, 77KB)

In this example of a business exam, you can see that the exam is relatively balanced in terms of ideas, connections, and extensions overall, and in most content areas. However, note that ethics, for instance, is only extensions-type questions—perhaps not surprising since ethics often involves decision-making in novel situations.

Music Theory Example (PDF, 77KB)

In this example of a music theory exam, you can see that the content areas are more fine-grained, and that the instructor has made notes for themselves about specific question types alongside the points distribution. The exam is almost exclusively ideas-type questions, but the instructor was fine with this because connections and extensions learning outcomes were assessed with other assignments (written reflection papers and a final project).

Frequently Asked Questions