Personal need

When I started teaching, at the College of St Mark & St John in Plymouth, Devon, I was struck by how difficult it is to assess student work.

How does one equate thirty 2500 word essays?

...and...

If sophisticated criteria are used to help equate those essays, how can one generate a final grade, without making mathematical mistakes or spending an inordinate amount of time marking?

Initital inspiration

Inspired by a spread-sheet 'marking machine', designed and used by my colleague, Tim Sayer, I set out to build upon Tim's design, using the ToolBook 3.0 CBT authoring environment.

Over subsequent weeks and months, the new machine was tested, improved, and used to mark a number of assignments. Informal comments from students indicated that they appreciated the detail the new reports provided.

At the same time as improving the detail in the reports I provided students, using a marking machine also yielded a number of other, previously unforeseen, spin-offs. For example, the development of explicit marking criteria has enabled me to investigate student perceptions of work submitted for me to mark.

Additional inspiration

Inspired by a second colleague, Malcolm McIver, MarkThis made it easy to provide students with the criteria by which their work will be assessed. This enables students to be clear about the work they need to complete and whether they are putting in the effort they will need to obtain the marks they desire.

In addition, reading the students' own self-assessment sheets enables me to rate each student's ability to be self-aware - and also enables me to be aware of potential problems, when my own rating and a students rating are wildly different.

Explicit marking criteria

Generating an explicit set of criteria, upon which work will be assessed, has many benefits.

First and foremost, generating an explicit set of criteria enables a teacher to know exactly what s/he is looking for.

Secondly, students can be provided with a clear set of criteria by which they may self-assess, which means that they can learn how to critique their own work.

Thirdly, banks of comments can be evolved over time, and across modules, enabling teaching teams to develop clear and consistent standards.

Detailed feedback

Students can be provided with computer generated reports, which indicate exactly how marks were allocated. If required to submit a self assessment form, along with their work, students can also see how their assessment of their work differs from a teacher's assessment of that same work.

Saving time

Because detailed calculations are handled automatically, in the background, reports can be more detailed than would otherwise be practicable. Giving a sub-mark for twenty different criteria is a nightmare, if the final mark then has to be calculated by hand. For a computer, such a task is child's play.

Reduced effort

Once a bank of comments has been set up, then those comments focus the teacher on what to expect of his or her students. By using comments to describe a piece of work, the quality of the work can be quickly and easily assessed.

List of students

The basis of a MarkThis file is a list of students to be assessed. This list can be generated by hand, or imported from a word processor or other application.

List of criteria

A piece of work may be judged on up to fifty separate criteria: for example: coherence or use of English. Each criterion is 'marked' using a set of comments: for instance: excellent, good, or satisfactory. MarkThis takes these comments and derives a final percentage.

Lists of weights

When generating a final mark, MarkThis does not expect all criteria to be of equal weight, therefore the teacher is given the option to alter the weightings attributed to different criteria.

Assessment screen

Looking at each student's work, the teacher selects an appropriate description of that work, from a bank of comments.

Personal notes

At any time, personal notes can be made, or selected from a common bank of notes. For example: "Remember to use the spell checker!" or "Excellent work". A standard, MarkThis is installed with a large comment bank to get you 'up-and-running' quickly and easily. Over time you can customise that comment bank to suit your own particular preferences.

Output screen

Having requested output, a report is generated and loaded into the word processor of your choice, in rich text format. The report provides a table of results, average and standard deviation, for the group as a whole - as well as detailed comments and a summative judgement for each individual. Alternately, you can have reports output as emails placed in the outbox of your emailer.

Module Leader

You can create a list of files, MarkThis will then collate the information in those files and generate a report. The report will present a table of marks, average, standard deviation, notes relating to extenuating circumstances, personal notes and notes relating to possible problems - for example, should a student be missing a mark.

Link to Orator

Those of you, who have made use of BrainBox, will know that BrainBox contains a piece of software called Orator. Orator can read to you, making use of Microsoft Windows text-to-speech technology. Now, MarkThis can send a data file to Orator at the click of a mouse. Orator can then read lists of students and the marks that they received. This is useful when you want to transcribe marks to paper, or want to check for transcription errors.

Looking ahead

We already have some plans for the next version of MarkThis Pro. Mainly, we want to replace the ToolBook file dialogs with Visual Basic ones, because the ToolBook dialogs do not always represent long files very well. However, if there's something you would like - let us know...

...MarkThis can also be easily be adapted by trainers in business...

...or even by HR managers who need to deliver staff appraisals...