Online Assessment

Why is it important to align assessments with learning outcomes?

Alignment between assessments and desired learning outcomes is foundational if your assessments are to be valid. Just like in a research study where you want to make sure that your research instrument is measuring what you want it to measure, by aligning your assessments to your learning outcomes you are making sure you are assessing what you want to assess. As we mentioned in the Course Planning with Backward Design, Biggs (2003) describes the constructive alignment of three components: (a) measurable, clearly-stated learning outcomes, (b) assessment tasks that allow students to show to what extent they have reached the learning outcomes, and (c) activities (including content and practice) that help students reach the learning outcomes. Assessments that are aligned with your learning outcomes provide dependable evidence as to how well students are reaching the desired outcomes.

Clearly aligning assessments to desired learning outcomes also reinforces to students what needs to be mastered and helps them track their progress in the course. Students pay attention to what you test. For example, If your intent is for students to be able to apply, critique, or evaluate, but your assignments and exams ask students to remember, identify, and describe, then your assessments aren't aligned with your desired learning outcomes. Asking students to describe a concept doesn't encourage them to evaluate the concept in context and doesn't provide evidence that they can evaluate the concept.

The intent of Backward Design is that assignments (and everything else) are aligned to desired learning outcomes instead of creating learning outcomes based on what you are assessing. Starting with assessments and extrapolating learning outcomes from them is the definition of "teaching to the test." This may be necessary if your course is preparing students to sit for a licensure or registry exam, but those cases are the exception more than the rule.

What does assessment look like in an online context?

Just as you can't confirm a hypothesis without testing it, so, too, you can't confirm whether your students have achieved the course learning outcomes without some form of assessment. This is why assessment is the second stage of backward design - if you know where you want students to go (learning outcomes), you next need to decide how you'll know if they've gotten there (assessment). Assessment is that evidence.

Although the types of assessments that often first come to mind are a test, paper, or lab exercise, many other activities can be used for assessment, including portfolios, discussion forums, concept maps, diagrams, and presentations. Any tangible output from a learning activity can be assessed. Your choice of output—and the activity designed to generate that output—should be determined by your learning outcomes; this is just as true in the online environment as it is in the traditional classroom.

First let's look at four qualities of assessments: Formative, Summative, Authentic, and Traditional; and how those qualities can be leveraged in different ways online. On the following page we see how mixing these qualities can create a balanced, comprehensive assessment strategy that is aligned with learning outcomes for an entire course.

Formative Assessment

feedback on performance looping back to improve performance Formative assessment is designed to provide feedback to both student and instructors about how well the learning process is going. Examples of formative assessment include self-tests, think-pair-share activities, and other low risk assignments that allow students to demonstrate their knowledge, skills, and abilities.

Another option for formative assessment is to develop a larger, summative assessment and break it into smaller components that can be turned in throughout the semester. This allows you to catch and address misconceptions, challenge students’ early analyses, and provide the opportunity for them to revise and resubmit each piece in a unified whole at the end of the semester or unit. Scaffolding an assessment in this way ease your end of semester grading burden as well. Your students will hopefully produce higher quality work and you'll be able to focus on how well they've incorporated your feedback rather than checking their entire submission.

Formative assessment options such as ungraded self-tests using the Canvas Quizzes tool, think-pair-share activities in discussion forums or group spaces offer ways for students assess their own understanding of course concepts. If you are interested in embedding some understanding checks in your Canvas pages, try Quizlet which can be added by going to Settings and then Apps in your course.

An often overlooked option for formative assessment is peer review and feedback. When adequately scaffolded, peer review and critique can be a learning activity for both the student giving and the student receiving the peer review. The Assignments tool in Canvas provides options for blind peer review, or you can set up a Discussion where students post their thoughts or explanations or examples and then provide feedback to the person posting immediately above them. Students can be split into small groups where they can share an initial draft of a paper or project, each student gives feedback to all the other group members, and then they work together to synthesize their best efforts into a group report. Using the Group spaces in Canvas allows the instructor to see all of the initial drafts and student discussions while keeping each group separated from the other groups.

Summative Assessment

Summative assessment is designed to provide evidence that students have achieved a learning outcome or otherwise gained skills or knowledge throughout the course. End-of-semester exams, projects, portfolios, and presentations are often used to summatively assess students' knowledge and skills. Courses that use a blend of summative and formative assessments provide more consistent support for learning than relying exclusively on a midterm and a final exam.

Final papers, projects, and portfolios have a variety of options in an online class. It's easy to incorporate media into Assessments, Discussions, and Pages - both in project instructions such as presenting a video case for analysis, and in student work such as recorded presentations, interviews, and demonstrations. A videoconferencing tool like Canvas Conferences can be used for synchronous assessments such as oral exams in languages. Other tools such as Zoom.us or Google Hangouts record individual video presentations or interactions such as mock counseling sessions and other role play scenarios which students can submit to an assignment or share in a discussion.

Authentic Assessment

Circle labeled authentic assessment placed with some overlap on top of a circled labeled context Authentic assessment asks students to demonstrate skills and knowledge by performing realistic tasks within the discipline. It provides opportunities to practice, consult resources, get feedback, and refine performances and products. Well-designed authentic assessments:

are realistic, using real-life situations with constraints, purposes, and audiences that impact what needs to be done.
require judgment and innovation to effectively solve unstructured problems
assess students' ability to use their knowledge and skills to negotiate a complex task as a whole.

Authentic assessment commonly uses strategies such as case studies, simulations, consulting (where students work with real organization to explore a problem and recommend solutions that are evaluated by both the instructor and the organizational partner), internships, and service learning. However, depending on the discipline, authentic assessment can leverage simpler tools. For example,

situating statistical questions in the context of fantasy sports over several weeks of a season,
sending emails in a foreign language to request information about a travel destination and working with a small group to determine which destination would be the best vacation spot.

Traditional Assessment

traditional assessment sits out of context Traditional assessment (defined mainly as discrete-item testing) tends to emphasize the development of a body of knowledge or skill. Does a student know the who, what, when, and where? Traditional assessment strategies are helpful when you want students to identify one best answer and/or target isolated skills in a concrete fashion.
Something to keep in mind is that assessment methods do not have to line up with assessment approaches. For example, multiple choice test items can be developed to draw attention to contextual factors in an authentic case. In the same way artificial and minimally contextualized cases can be used to identify who, what, when, and where without asking students two work with holistic, complex problems.

Online testing using the Canvas Quizzes tool provides auto-grading and auto-feedback features with a wider range of options than blue book or scantron testing. You can provide video, audio, and images as part of a question, and students can record or upload video, audio, and images as part of their answers. You can pre-set different feedback for different incorrect answers and even re-route students to a review page explaining the question in more depth.

However, there are also drawbacks. Many faculty express concerns about the potential for cheating in an online class. Where a faculty member might make one test and deliver it once in a proctored room for a face-to-face course, a similar fully online test may be delivered over time. Remote proctoring services are available to allow students to take assessments at any time using their own computer while proctors monitor and record their webcams, physical environments, and desktops. These services provide secure authentication to help ensure assessment integrity however they are not inexpensive.

If you do use online testing features, here are some options to consider:

Create multiple question banks for a single test - break down the questions that are delivered to students, so no one gets the exact same examination, but all are tested on similar concepts and at similar levels of difficulty.
Randomize the order of questions to discourage cheating
Randomize the answers and distractors to discourage cheating
Use an honor statement at the beginning of each test, forcing the students to check "yes" or "no" that they understand the academic dishonesty policy, and they have not received assistance.
For more options specific to the Canvas Quizzes tool, please see the Canvas Quizzes Instructor Guide.

Multiple Choice Question Construction

There is a science to creating good multiple-choice questions that accurately assess student performance on your course outcomes. Here is one of the best guides to Writing Good Muliple Choice Questions from the Center for Teaching at Vanderbilt University. The guide clearly demonstrates how to write a good question stem, valid detractors, and good and poor examples of questions.

One of the common errors we see in multiple-choice tests is using 'All of the above', 'None of the above', and using combinations of detractors, such as 'Both A & D'. These should never be used. "When “all of the above” is used as an answer, test-takers who can identify more than one alternative as correct can select the correct answer even if unsure about other alternative(s). When “none of the above” is used as an alternative, test-takers who can eliminate a single option can thereby eliminate a second option. In either case, students can use partial knowledge to arrive at a correct answer" (Brame, 2021).

The guide also suggests the order in which to present the detractors and other great recommendations. Check it out, it is a quick read!

Brame, C. (2013) Writing good multiple choice test questions. Retrieved 4/9/2021 from https://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/ .