The information on this page is for those departments who want to do department-level assessment of student learning. Please reach out to and partner with the Office of Accreditation & Assessment to learn more about these assessment options before you get started. To help you understand some of the assessment tool options that the Office of Accreditation & Assessment can support you through, read below and use this information to inform your consultation. 

Assessment Tool Options

If you want to do department-level assessment with the support of the Office of Accreditation & Assessment, there are two primary tools: Canvas and Qualtrics.There are different reasons why you might choose either one, however, a good standard is to consider who your assessors are:

1. Mostly assessors internal to St. Thomas? We recommend using Canvas Outcomes.
2. Mostly assessors who are external to St. Thomas? We recommend using Qualtrics.

Next steps for each tool are below.

Canvas Outcomes Assessment Option

If you want to use Canvas to support your departmental assessment efforts, please reach out to the Office of Accreditation & Assessment. This is an overview of using Canvas for assessment.

The Office of Accreditation & Assessment will:

• Serve as the Account Admin on the Office of Accreditation & Assessment (OAA) Canvas Sub Account. 
• Make necessary staff/faculty (e.g., Program Admins) Sub-Sub-Account admins for their College or Program Sub-Accounts (if needed) 
• Generate assessment sites within the OAA Sub-Account 
• Add any necessary department staff/faculty to Canvas assessment sites in appropriate role: 
  • Program Admins added to course sites as “Teacher” (if needed)
  • Assessors added to course sites as “Assessor - OAA”
• Submit Service Tickets on the behalf of departments to Academic Technology
• Ensure correct Canvas Outcomes setup and rectify errors
• Serve as the main contact for assessment inquiries and Canvas Outcomes support

Canvas Outcomes Workflow for OAA (or Department Program Administrator): 

1. Determine Assessment Needs and Configure SIS Enrollment Feed: 
   • Identify students who require assessment. (This may necessitate requesting access to the active or past course site to gain access to artifacts and students.)
   • Configure SIS Enrollment feed using template of students to assess (instructions below, #Creating an SIS Enrollment Feed)
2. Request Program-level Access and Canvas Assessment Course (for Department Program Administrators only): 
   • Add appropriate department sub-account (if needed) and generate their copy of the Canvas Assessment Course Template 
3. Set up Outcomes at the Program-level in the Assessment Sub-Account: 
   • What are [Canvas] Outcomes? (includes 'Setting up Outcomes')
4. Confirm & Set up Assessment Course Settings: 
   • Confirm that default template assessment course settings are in place (via Settings):  
     • Course Details:
       • Term: Assessment Term
       • Participation: Start and End dates are both in the past
       • "Restrict students from viewing course before term start date" is checked
       • "Restrict students from viewing course after term end date" is checked
       • Mastery Paths: "Enable individual learning paths for students based on assessment" is checked​​​​​​
     • Feature Options: 
       • "Anonymous Grading" is enabled  
       • "Learning Mastery Gradebook" is enabled
       • "Moderated Grading" is enabled
       • "Outcome Service Results to Canvas Reporting" is enabled
   • Set up additional assessment course elements:
     • Add Program-level outcomes to "Outcomes"
     • Create rubrics specifying assessment criteria  
     • Create assignments for assessors 
       • How do I create an assignment? *Note: The checkbox next to "Do not count this assignment towards the final grade" cannot be checked. If it is not checked, you can go into the gradebook and click on the shelf to submit as student.
5. Add Students to the Assessment Course: 
   • Submit ticket to Academic Technology to upload pre-configured SIS Enrollment feed to the Assessment Course (instructions below, #Creating an SIS Enrollment Feed)
6. Prepare and Submit Artifacts for Assessment
   •  Download all original student artifacts for assessment from their original course site 
   • Clean artifacts: remove student identifiers, filenames, metadata. 
   • Submit artifacts in the Assessment Course using “Submit as Student” for each student, in each assessment assignment 
     • How do I submit an assignment on behalf of a student as an instructor?
7. Add Necessary Staff/Faculty for Grading (and/or Course Management): 
   • Assessor - Added to courses as the course role "Assessor - OAA" (role based on the "Teacher" role but without the ability to edit the course or message the students)
     • How do I use a rubric to grade submissions in SpeedGrader?
     • How do I add a moderated assignment to be graded by multiple reviewers?
   • Program Administrator - Added to courses as traditional course role “Teacher” (if needed)
8. Analyze Results: 
   • View Learning Mastery Gradebook for overall results to Outcomes
   • Department takes responsibility for interpreting results and exporting data for analysis 
9. Repeat Assessment: 
   • To repeat assessment process, OAA/Program Admins can: 
     • Set up sections within an existing assessment course, OR 
     • Create a new assessment course

Please reach out to and partner with the Office of Accreditation & Assessment to learn more about these assessment options before you get started.

Creating an SIS Enrollment Feed

This section outlines how to create the spreadsheet of student enrollment information for Academic Technology to upload into a Canvas Assessment site. The simplest way to enroll students without providing them unnecessary notifications of their enrollment is by creating an Student Information Services (SIS) Feed, which is a CSV file listing students’ Banner ID (“USER_ID”), their course role (“ROLE”) as ‘student’, the unique SIS ID of the Assessment Canvas Course you want them enrolled in (“SECTION_ID”), and their status in that course (“STATUS”) as ‘active’. Use our template SIS feed to create your own SIS Enrollment Feed spreadsheet. Once you’ve configured your SIS Enrollment Feed, then submit this service ticket to have Academic Technology enroll students into the Canvas Assessment Course.  

Before submitting your SIS Enrollment Feed service ticket, these requirements must be met: 

• At least one person needs to be already enrolled in the course (this could be the Program Admins added as “Teacher” or Assessors added as “Assessor - OAA” or any other role)
• The file is a CSV file (.csv); files saved as an Excel workbook file (.xlsx) will not be accepted  
  • Note: Our template downloads as an Excel workbook (.xlsx) file, so ensure that you “Save As” and change the file to CSV (.csv) before submitting it 
• Rename the file using this naming convention: YYYY-MM-DD_AX_YYYY[10,20,30,40]_XXXX_### SIS feed (explanation of naming convention below) 
  • YYYY-MM-DD_ = the day you’re submitting the ticket to request the SIS enrollment feed to be uploaded 
  • AX_ = “AX_” indicates it’s an Assessment course 
  • YYYY[10,20,30,40]_ = this will be the year and semester you’re assessing  
    • 10 = J-term 
    • 20 = spring term 
    • 30 = summer term 
    • 40 = fall term 
    • Or if it’s a whole year, how about instead of putting the semester indicator, put 2023-2024 
  • XXXX_### = Use the four letter Department Code (e.g., HIST, PHIL) and three number course code to indicate which course is being assessed (ensure Department Code is put in all CAPS)
  • SIS feed = indicates what kind of content will be on the file 
  • Note: SIS Id's are permanent (even if you delete a section/course associated with it). Every SIS Id needs to be unique. If you need to remove a section and/or course with an SIS Id attached to it, do NOT delete it yourself and instead submit a ticket for it to be removed. 
• On the spreadsheet itself, there must be: 
  1. The first row will be interpreted as a header defining the ordering of your columns. This header row is mandatory. DO NOT EDIT the first row of the template. 
  2. Under “USER_ID”, Banner ID numbers for each student being assessed in the Assessment course 
  3. Under “ROLE”, the word ‘student’ listed next to each Banner ID  
  4. Under “SECTION_ID”, the unique Canvas SIS ID of the Assessment Canvas Course you want them enrolled in
     • Note: The Canvas SIS ID for your assessment course will be the naming convention outlined above (excluding the date the file was created). For example, AX_202240_HIST_115 is the Canvas SIS ID for an assessment course assessing students in a HIST 115 course taken Fall 2022.
  5. Under “STATUS”, the word ‘active’ 

After my Canvas SIS ID is created for my course, where can I find it? (see the screenshot below) 

• In your Assessment Canvas course, select “Settings” from the lefthand menu 
• Ensure that you’re in the tab “Course Details” 
• Scroll down to “SIS ID” to find your SIS feed configuration 

Qualtrics Assessment Option

If you want to use Qualtrics to support your departmental assessment efforts, please reach out to the Office of Accreditation & Assessment. Additionally, review the knowledge base articles for using Qualtrics: Qualtrics: Getting Started.