Viva Voce

I have struggled with the state of testing in education since I was in high school. Then, as now, I find the types of tests administered to be inadequate for the purpose of evaluating student knowledge. Common testing approaches challenge students with unresolvable contradictions and false dilemmas, which frustrate and obstruct clarity and fairness in evaluation. They are also prone to cheating, or simple rote memorization of answers.

Historically, tests in elite schools were administered in person, as an oral exam. The following two scenes, taken from the HBO/RAI adaptation of Elena Ferrante's Neapolitan novels My Brilliant Friend, present a fictionalized view of how this might have looked in the Italian education system in the 1960's. In the first clip, Elena, the protagonist, is being tested at the high school level. In the second, her friend Franco is tested Scuola Normale Superiore in Pisa.

I seek to replicate this experience, as closely as is feasible, in software. My goals are threefold:

1. To provide educators with a superior and cost-effective method to test students' knowledge.
2. To effectively thwart cheating.
3. To help students better understand what they know, and where additional study may be indicated.

Viva Voce (living voice in Italian) is made up of three modules:

A: The Exam Blueprint

Here an examiner establishes the arc of an exam, which can be as short as just a few minutes, or as long as several hours.

The process involves uploading source material -- textual documents, pdf's, images, and audio. Although optional, most exams are built from the source material students are expected to study.

The exam length, and the level at which the examiner challenges the student, are then modified as needed.

The arc of the exam is defined by "anchor" questions -- key questions around which the exam progresses -- which are be presented to the student in slightly re-worded ways. These questions are assigned stages within the arc: they typically start off with a "Foundations" stage which test basic knowledge, then proceed to a "Dialectic" stage in which the student is challenged on their knowledge, and finally, come around to a "Synthesis" stage, examining the overall subject matter.

The system can auto-generate the questions from the source material. If the generation is unsuitable, the examiner can request additional generation attempts from the system, providing feedback to help the generation better align with their vision. The examiner can also craft their own anchor questions.

Once the anchor questions are in place, the examiner can:

• (a) define a rubric for what constitutes an acceptable answer, or provide context for what they are looking for in an answer.

• (b) attach media files that had been uploaded to the exam blueprint.

• (c) change the stage of any question, or the order within the exam.

• (d) approve, reject, or delete any question.

  NOTE: An exam requires at least one approved question to be testable or releasable to the students.

After the core arc of the exam is established, it is advised that the examiner try out the exam themselves, to fine-tune it. Once the examiner is satisfied with the exam blueprint, they can "release" it, thereby finalizing its state, and making it ready to be taken by students.

B: The Student Address Book

The address book stores all of the examiner's student information. Currently, this is limited to their first and last names, and email addresses. Examiners can invite one or more students to take any exam they craft, and can review a history of their attempts at taking any exam.

C: The Examination

The exam is executed as an oral exam. Students can take the exam on a mobile device, although the examiner may wish to advise students to take the exam on desktops if careful attention to imagery is required.

First, the student is presented with a voice selection screen.

Once selected, they can begin the exam. They are presented with a countdown timer, as well as a counter displaying the total number of questions, and the number of questions answered.

The examiner then poses the first "anchor" question. Depending on the level of rigour selected, the examiner might press the student to elaborate on, or enrich, their responses.

There is no way to pause or stop the exam. Premature termination, such as closing a window, is recorded.

D: The Results Review

Once the exam is complete, the system evaluates the responses. The system will flag:

• (a) a change of tone or certainty
• (b) a change in the voice signature within any given answer.
• (c) a change in the voice signature across answers.

The examiner can review every student's results, and override the system's proposed score. A transcript of the entire exam session is presented for review. The audio of every response is also available for review. The audio clips are also automatically flagged by the system if cheating is suspected.

Once the review is complete, the examiner can send the review to the student, and can invite the student to re-take the exam.