COVID-19 and the dramatic increase in remote, hybrid, and online teaching have prompted many CU faculty and instructional support staff to rethink many elements of their courses and question previously held assumptions about how people learn. How to administer exams is a particularly thorny issue, especially for courses with large enrollments for which fact-based, multiple choice exams are the norm. This guide highlights key considerations, concerns, and effective practices for remote and online exams.

A few things to keep in mind as you work with students during this time:

  1. Start with care and trust. Remember that we are still in the midst of a pandemic and that this is not the “new normal,” but rather a constantly changing environment with an overwhelming amount of information and daily disruptions contributing to additional stress. Meaningful teaching and long-lasting learning depend on the mutual trust of educator and learner. This has never been truer than it is now, when the where and how of teaching and learning are so disrupted.
  2. We can’t always see the challenges. Researchers have found that people are . It is common to have feelings of fear and anxiety, loneliness, depression, and negative impacts on existing health conditions. CU students may have trouble concentrating, sleeping, or eating, and may increase substance use during this time – all of which have substantive impacts on learning and decision-making.
  3. Be straightforward, clear, and accommodating. The pandemic has had inequitable and compounding effects on CU students and their families. A found that students – especially first-generation students – suffered financial hardships that may have prevented them from returning this fall. Many of these students do not have access to adequate technology or the internet, and often face food and housing insecurity.

Effective Practices for Online Exams

Done well, exams can be effective assessments of learning, and can be used to help students focus attention on key concepts and materials. Facing an exam in an unfamiliar format or with unusual expectations can contribute to additional stress and test-taking anxiety that can interfere with students’ abilities to effectively demonstrate what they have learned. Educators should be mindful of the ways in which exams can be designed and administered to support students and their learning.

Align Assessments to Course Learning OutcomesĚý

When deciding whether to use an exam to assess learning, consider how the assessment aligns with learning goals. Do the exam content and format allow students to demonstrate their understanding of key course concepts or ask them to apply skills learned in the course? Good assessments in any teaching modality starts with clear alignment of learning tasks to course learning goals. Assessments should focus on what instructors want students to know or be able to do and not punish students for hidden or implicit expectations (e.g., writing syntax errors).

Use Multiple Low-Stakes Assessments Rather than Fewer High-Stakes Exams

Research shows that low stakes assessments promote learning by providing frequent formative feedback and multiple opportunities for improvement. . Low-stakes assignments provide opportunities for students to demonstrate their learning throughout the semester, as well as foster a feeling among students that they are making measurable progress. Brief writing assignments such as one-minute essays, rapid response activities using tools like Clickers, Quizzes in Canvas, , peer- and self-assessments using rubrics, and online discussion forums can keep students engaged and allow instructors to check in on their learning.

Create Exams that Require Higher Order Thinking

If one of the course goals is for students to demonstrate higher-order thinking skills, create exams that allow students to . This approach asks students to think critically and tackle tough problems.

Consider Open-Book Exams

“Open-book” exams encourage students to . Open-book exams tap into students’ critical thinking, analysis, and application skills. Have students show their work, explain answers, interpret data visualizations, or draw connections between ideas. Make sure to clearly communicate your expectations for what resources students are allowed to use, and whether or not they are allowed to work with other students.

Design Assessments with Equity in Mind

Inequities in higher education persist under normal conditions and have come into sharp relief during the COVID-19 pandemic. We must be careful not to inadvertently further or exacerbate inequity through pedagogical or technological choices. The information provided here is meant to support meaningful, learner-centered assessment. Creating assessments with equity in mind means involving students in the assessment process such as using student-generated exam questions; offering flexible options for students to demonstrate their learning; and being able to .

Concerns about Cheating

Faculty may be concerned about cheating and the effects on learning and grades. While the strategies discussed in the previous section minimize the opportunity for cheating, some students will still choose this path. The following recommendations directly address these concerns while minimizing the need for proctoring.

Encourage Accountability with the CU Honor Code.

Ask students to sign a pledge or write a statement in their own words that they will uphold the Honor Code. CU’s Honor Code states, "On my honor, as a University of Colorado Boulder student I have neither given nor received unauthorized assistance." You can include the CU Honor Code statement at the beginning of the online quiz or in the exam instructions.

Use an Exam with Random Item Generator

Canvas Quizzes can draw from a bank of test questions and randomly assign questions to individual students based on set parameters. Some question types allow the instructor to populate test items with different values so that students must solve the same kind and level of problem but cannot share answers with others.

Randomize Answer Options

If your exam includes multiple choice questions, you can randomize the order of options so each student is presented with a random order. This setting is available in the Quizzes tool in Canvas. This will make it more difficult for students to share correct answers.

Set Time Limits

While it is important to provide sufficient time for students to complete the exam so that they can account for competing issues such as their living situation and access to technology, setting a time limit for students to complete the exam could also reduce the opportunities to violate academic honesty. Remember to allow extended time for students who have accommodations through Disability Services.

Canvas Tools for Online Exams

Quizzes in Canvas

Instructors can administer online exams using the tool in , CU Boulder’s learning management system. The tool has several question types available, including Multiple Choice, Matching, Numerical, Formula, and Essay. Most of the question types can be auto-graded. Instructors can control when a quiz is available and how much time students have to complete a quiz, as well as which results are released to students and when they are released. makes it easy to adjust the exam availability window, time limit, and number of attempts for individual students (e.g., if a student needs to take an exam at a different time or needs extra time to complete it). also integrates with .

Instructors can get support for setting up their exams in Canvas from . OIT offers both as live sessions and as .Ěý

Exam setting and configurations strategies to minimize academic dishonesty:

  • Set exam time limit
  • Randomize the order of multiple-choice question answers
  • Randomize the order of questions
  • Create a question bank and set the exam to pull a randomized set of questions for each student
  • Set the exam to not release correct answers to the students until all the students complete the exam

Assignments in Canvas

The Assignments tool in Canvas can be used to evaluate papers and a variety of project formats, including video. Assignments has a flexible rubric tool that can help make grading faster and more consistent. Rubrics are also a great way to set student expectations for the quality of work. allows instructors to leave comments and annotations directly on student papers, as well as to provide overall text and video comments on student submissions.

Online Exam Proctoring Software: Proctorio

The Office of Information Technology (OIT) supports , an online exam proctoring software that is available for faculty to enable in their Canvas courses. Proctorio simulates an in-person testing experience through automated user authentication and monitoring of the test taker. Proctorio requires that exam takers have a computer with camera, speaker and microphone. Proctorio is a robust platform that meets CU’s privacy and security policies; however it must be noted that some faculty and students have expressed concerns about its use in college courses.

Uses for Proctorio

Faculty may choose to use online exam proctoring out of concerns for academic integrity, for verifying the test-taker’s identity, or for administering multiple-choice exams to courses with large enrollments. CU faculty who use the tool say that it is easy to use and reduces cheating. Proctorio gives instructors granular control over the types of test taker behaviors they would like to monitor. For information about Proctorio’s features and training sessions, visit .

Concerns 91ÖĆƬł§ Online Exam Proctoring

Online exam proctoring has from educators and student privacy rights proponents over the invasive methods that the platform uses to monitor student body and eye movements, access the student’s computer, and surveil the student’s surroundings. Instructors and students have likewise noted inequities such as students’ limited access to technology or bandwidth, stressful test conditions or shared living spaces with no access to private places to take the exam. Online proctoring platforms like Proctorio can also be incompatible with assistive technology such as screen readers, tablet computers, or outdated computer systems.

In July 2020, the Remote Exam Working Group in the College of Engineering & Applied Science recommended against using Proctorio, citing privacy concerns and the sense of mistrust that its use can create between students and instructors. For more information and recommendations, read their report, Remote Exam Best Practices.

If You Decide to Use Proctorio...

Offer a practice exam before the actual test. This will help reduce stress, as well as work out any potential issues. Step-by-step instructions on how to set up a practice test as well as a pre-built practice test that you can upload to your course are available in on the OIT website.

Provide an alternate arrangement. If remote proctoring must be used in your classes, allow exemptions or alternatives for students who have limited connectivity or do not have access to the technology or equipment required for using Proctorio, and for those who need to take the exam in another time zone. Alternatives will also need to be extended to students with disability accommodations, students who do not feel safe exposing their environment, and those who express mental health concerns. Please consult Ěýfor information about limitations of the tool for users with disabilities.

Include a statement about online proctoring in the syllabus. It is recommended that instructors include information about Proctorio in the course syllabus, as well as link to the Proctorio security and privacy information and explain what to expect should students need an . OIT provides that you can adapt to your course.

Above all, do not require a higher level of proof of learning in an online class than you would in a face-to-face course.

Further Reading & Resources

For more information or assistance with creating effective assessments:

CU Boulder College of Engineering & Applied Science Remote Exam Best Practices

Office of Information Technology

Ěý

Ěý

Ěý

The Bay View Alliance is an international network of research universities exploring strategies to support and sustain the widespread adoption of instructional methods that lead to better student learning.ĚýCU Boulder is one of ten research universities in the network working to bring more effective teaching approaches into wider use by sharing what works and pooling resources. .