CBA Research

cba_assessmentI have completed my Ph.D under the supervision of Dr.E.Foxley and Dr.C.Higgins. My thesis has the title ” A Framework for the Computer Based Assessment (CBA) of Diagram Based Coursework“. It asserts that developing free-response CBA for coursework that is solved by students drawing diagrams, is both feasible and useful. To substantiate these claims, in the almost seven years period of my phd, I’ve designed, implemented and tested DATsys, a system which significantly eases the development of automatically assessed coursework that requires diagram-based solution.

My work, part of the Ceilidh project, is now part of Ceilidh’s newest version entitled CourseMarker (previously called CourseMaster). In the past, I’ve implemented several domain dependant diagrammatic editors and attempted to capture their commonality in a object oriented framework. To grasp the complexities and a portion of the art of developing OO frameworks, I had to scratch several initial prototypes and restart…

My most recent diagrammatic framework, entitled DATsys, is based on architectures, first introduced by frameworks such as Unidraw and Hotdraw. Implemented in Java, it is using Aspect Orientation to describe meta-object functionality needed to allow its runtime dynamic evolution. By “runtime dynamic evolution” I refer to the ability of the system to change at the source-code level, recompile and relink its parts making the changes in behaviour to appear instantly and while running. Hence, by using the diagrammer we can get the next version of itself (although at the moment in very restricted ways).

Three main diagram editors are part of the of the DATsys framework:


The idea in naming these editors has been taken from Greek mythology. Daidalos, for the developer of new diagram domains, Ariadne for the author of diagram-based CBA and Theseus for the students solving the diagram-based coursework. Daidalos in greek mythology made the plans for the labyrinth, Ariadne was the only one that knew the solution to the labyrinth, and Theseus was the hero that solved the puzzle and got his way out of the labyrinth. As the metaphor implied for the development of diagram-based CBA often is not perceived, DATsys also means Diagrammatic Assessment Teaching System.

diagram showing the three levels of editors

Daidalos is a meta-diagram editor that allows the definition of tool libraries for new diagram notations. It can be considered as a diagrammatic domain modeler. Diagrammatic courseware developers use daidalos to:
a) Model graphically the diagram elements, relationships and data model of a concrete domain.
b) Create graphically the tools that allow the direct manipulation of the diagram elements
b) Change and evolve the system’s behaviour by using its dynamic meta-level facilities.

Ariadne is another diagram editor that uses domain libraries built in Daidalos to allow the definition of automatically assessed graphical course-exercises. To create such exercises the developer has to enter the exercise question and its solution, parameterise the checking oracles, program the marking script and specify the apllication functionality of the generated diagram-editor.

Theseus is the last link in the chain of diagram editors. Theseus is completely generated according to the specifications of the developer using Ariadne. It can be very simplistic, offering the minimum functionality needed by students to draw the exercise’s solution, or it can be as complex as Daidalos itself. A generated Theseus can be used for a whole range of exercises or it can be customised to a specific one.

diagram showing the architecture of DATsys

If all these sound confusing you can check out a series of animated demos that illustrate the use of all the software mentioned:

Tutorial Contents Slides Duration
concepts, Introduction
139 03:56s

Selection Editor
the View of an Element
130 03:16s
the Prototype Tools
222 04:45s
Tool Libraries
351 07:20s
the Canvas
Key Manipulations
111 02:40s



Tutorial Contents Slides Duration
and non-editable text fields
100 02:15s
pins and connection lines
247 05:33s
design, flowcharts, etc
351 07:20s
functions, configuration
82 03:04s
Tool Libraries in Theseus
tools, and using Theseus
87 02:33s
CBA using Daidalos and CourseMarker
and running CourseMarker and Theseus.
296 07:45s


Domain-libraries have been developed for coursework in the field of Electronic Engineering (Analog & Digital Circuit Design) and Computer Science (UML, FlowCharts, ER-Diagrams, Petri Nets). Exercises in these domains are currently in the development phase. My ambition is to address many more diagrammatic notations and create prototype automatically assessed exercises for diagrams such as :

Entity Relationship Structure Petri Nets DataFlow Flowcharts
Nassi Shneiderman Timeline/Fishbone State Transition Generic Layer Generic Painting
Venn Geometrical Generic Map Generic Topological Mechanical
Bond (Chemistry) Generic Trees/Graphs Mindmap/Concept Cell & Arrows Circuit Analog
Circuit Digital Logic UML Class UML Use Case UML Sequence
UML Collaboration UML StateChart UML Object UML Activity UML Component
UML Deployment Network Pert Music Transcription Visual Language

I have used Daidalos to create tool libraries for many types of diagrams:

many types of diagrm editors generated with DATsys

The development for marking tools for these domains is currently under investigation. The DATsys system is not yet available to other parties than universities doing research in CBA.

For a list of publications on CBA related subjects check my CV.

For an intro on diagrammatic notations you could have a look to a mini diagrammatics module I’ve compiled for our first year undergraduates.




Leave a Reply

Your email address will not be published. Required fields are marked *