Individualized Assessments

Enhancing learning via assessments unique to every student

Dividni helps to create individualized assessments. Individualization encourages students to engage in concept-oriented discussions rather than mere exchange of answers. It also enables an instructor to repeat an assessment activity with a different set of questions, thereby reinforcing student learning. In addition, individualization renders any blind copying in an assessment ineffective. Consequently, it is seen as an effective means of mitigating collusion and plagiarism.

Dividni helps us create

Trivia: Dividni is individ(ual) spelt backwards.


Installation

On any OS with Mono

This is the best option if you want to use Dividni on MacOS or Linux. Please download and install Mono first (Google Mono the open source .NET framework!). Then, download and extract Dividni from this zip archive: Dividni-v21.07.15.zip

On Windows

We need to install the latest .NET framework first. The recommended pathway to this is to install Visual Studio (the free community edition will suffice) which includes the .NET framework. But you can instead install the latest .NET framework itself or install Mono. Then download and install the Dividni MSI: DividniSetup-v21.07.15.msi

Additional step-by-step installation guide for Windows is available here.

Teleform Maker

Ulrich's Teleform Maker software creates versioned Teleform sheets for use with multiple choice examinations. Download of this software is restricted to authenticated users. It can be downloaded from here upon authentication.

Sample Scripts for Automation

Sample bash and Windows PowerShell scripts are available here to help with automation.

Additional Software

To convert HTML to PDF and to merge all PDFs into a single one, you may use wkhtmltopdf and pdftk. Both are freely available.

Useful tips on the installation and/or use of the other software is available here.


Architecture

There are two supported architectural styles. The first one is based on writing an HTML template and a number of associated macros. This style is useful for printed examinations and printed task sheets. The macros are functions that an instructor will have to define. The framework has a macro processor that takes as its input an HTML template and a library consisting of the instructor-defined macros, and outputs HTML assessment scripts where the macros have been substituted by the result of executing the macros.

Architecture

Use of HTML for the template allows the instructor to develop bulk of the assessment such as cover page, appendices, and possibly parts of question stems and/or answer options in HTML and supplement it with the code in the macros. It also allows a limited preview, without any macro substitution, of the template through any browser. HTML assessment scripts will permit digital delivery if applicable, or the scripts can be printed out for traditional paper-based assessment.

The second architectural style is to form questions as a C# class adhering to a given contract. These questions can then be exported to LMS (such as Canvas and Moodle) or online examination platforms (such as Inspera). The framework supports QTI (for Canvas and Inspera) and XML (for Moodle).

Papers

There are some academic papers on the use and value of the framework.

  1. Personalized assessment as a means to mitigate plagiarism — available at IEEE Transactions on Education. An authors' version of this paper is available to download here.
  2. Cheat-resistant multiple-choice examinations using personalization — available at Computers & Education.
  3. Contract Cheating in Computer Science: A Case Study — available at IEEE Xplore. An authors' version of this paper is available to download here.
  4. Repeat individualized assessment using isomorphic questions: a novel approach to increase peer discussion and learning — available at International Journal of Educational Technology in Higher Education.

Creating Assessments

The source code for a sample individualized examination is here. First build QHelperMcqSample.dll, and then run the following command to generate a sample (proof version of) examination.
TestGen -lib QHelperMcqSample.dll -start 42 -paper 1 -proof Exam.template.html

The source code for a sample individualized task sheet is here. Once you have built QHelperArithmeticTask.dll, you'd run TestGen as above on the task sheet template. Help is available using TestGen -help.

LMS Integration

Dividni can create question banks with individualized question variants for importing to learning management systems (LMS) such as Canvas LMS. Questions designed for use with TestGen can be turned into question banks containing individualized question variants. The command-line tool QtiGen generates a QTI ZIP file containing these question banks. The ZIP file can then be imported into LMS. For example, the following command generates a QTI ZIP file called myQti.zip containing two question banks, each with 200 variants of the questions specified in the two supplied C# files.
QtiGen -variants 200 -id myQti -qtiVersion 2.1 CsharpTemplate03.cs CsharpTemplate02.cs

QtiGen can also be used to create short answer and numerical questions. These questions are automatically marked by LMS systems, just like the multiple-choice questions. However, these questions do not have any answers to show. This means that the instructor only needs to supply the correct answers, and does not design any distractors.

QtiGen supports QTI versions 1.2 and 2.1.

Moodle LMS does not support QTI. To create individualized questions for Moodle, Dividni has a tool called MoodleGen. This creates question banks in the Moodle XML format which can be readily imported to Moodle and Moodle-based platforms such as CodeRunner. For example, the following command generates two Moodle question banks containing 200 variants of the questions in the two C# files CsharpTemplate03.cs and CsharpTemplate02.cs:
MoodleGen -variants 200 -bank myBank CsharpTemplate03.cs CsharpTemplate02.cs

Both QtiGen and MoodleGen accept HTML files as parameters in addition to C# files. The content of HTML files are converted to informative descriptions in a separate question bank. Cover pages, academic integrity statements, and appendices are examples of where such descriptive text might find use.

While multiple-choice questions work best for printed exam scripts, short answer questions work best for online quizzes, as these are well-supported by LMS and are auto-marked.

Short-answer questions offer two main advantages:

  1. An instructor does not provide distractors (i.e., incorrect answer choices) as in the multiple-choice questions. This saves time.
  2. A student not familiar with the topic has far less probability to score in a short-answer question than in a multiple-choice question.

There are two common types of short-answer questions used by LMS.

  1. Numerical Questions. Here the answer is numerical. The instructor is expected to provide a range within which the student-supplied answer is deemed correct. The lower-bound of the range is typically noted as AnsLTE and the upper-bound is typically noted as AnsGTE. The upper and lower bounds could be the same.
  2. Short-text Questions. Here the answer is a short text. The instructor is expected to provide at least one correct answer, but could provide multiple. For instance, a question asking the first name of Bond might provide the answers: James, Jimmy, Jimmie, and Jim. Note that not all LMS are case-sensitive when it comes to marking short-text questions. So please design questions taking this into account.

A step-by-step guide for importing quizzes into Canvas LMS is here.

Online Assessments

Online assignments currently work with Shibboleth's single sign-on to automatically pick a student's ID. The ID is then used to randomize the questions in the assignment.

Documentation on online assessment is here.