by Abdullah Mohd Zin [1],
Saadiyah Darus [2],
Md. Jan Nordin
and Abd Malik Md Yusoff

Document Management Research Group, Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia
sdarus2002@yahoo.co.uk

In teaching a course, instructors at higher learning institutions have to carry out the following tasks: prepare for the course, deliver the course and evaluate students' achievement. For language courses, students' achievement is normally evaluated by giving them a number of coursework and one or two examinations. Coursework is always considered an important part of a course. Through coursework, instructors could evaluate students' understanding and their ability to express ideas.

With the advent of IT, computer-based systems have been developed to improve our way of doing things. A number of systems have also been developed to help in the process of teaching and learning. In Malaysia, for example, the government has played an active role in promoting the use of IT in education through the Smart School project. The development of a number of courseware has been commissioned to various companies. The goal of the project is that by the year 2010, all students at Malaysian schools will learn by using computers.

Most of the computer-based systems that have been developed by various companies in the world are geared towards helping students in their learning process. However, lately, some effort has been undertaken to develop systems that can assist instructors in teaching. Examples of such systems are WebCT (Hazari 1998) and Blackboard (Penny 1986a; Penny 1986b). These two systems have started to gain acceptance in the academic communities and are currently being used to deliver courses online. Zahedi (2000) described an experience of using WebCT in teaching engineering. The main drawback of these systems is that an institution must adopt them, as they must be installed on servers for their effective implementation. What is really needed by instructors is a stand-alone system that can assist them in fulfilling their administrative of teaching, especially in handling students' coursework.

This paper will describe an experience of using Coursework Management System (CMS) in language teaching. It is one of the systems that our research group has been developing in order to help instructors in managing coursework over the last few years. CMS was designed to run on IBM PC under Windows Operating System. Most parts of the system have been implemented by using Microsoft Visual Basic version 5.

For most of the language courses, students' coursework is normally given in the form of written assignments or essays. With the present arrangement, the instructor would ask students to write an essay on a certain issue. They would then submit the essays written on paper. The instructor would collect them, mark and return to students. As feedback, the students' achievement in the assignment needs to be disclosed.

As the number of enrolment for each course increases, this arrangement has created many administrative problems. Firstly, instructors need to handle a large number of papers submitted by the students. For example, the number of students enrolled in one of our language courses is more than 150. If they have to submit three essays in a semester, the instructor has to handle 450 written documents submitted by students. From our previous experience, there were occasions where submitted papers were missing for some reasons.

Secondly, instructors have to keep track with coursework that has been submitted as well as un-submitted ones. For un-submitted coursework, they need to respond quickly by asking the students to resubmit. In the present arrangement, the instructors will need to note down the submitted essays against the name of the students in their classes.

The next problem is marking. The main aim of marking coursework is to evaluate students' performance in the course. However, a more important role of marking is to enable instructors to gauge students' understanding of the course. Thus, marking can also be considered as a feedback mechanism in the process of teaching and learning. For this to be effective, the marking process must be done as quickly as possible so that the feedback can be given promptly.

Instructors will then need to tabulate the breakdown of the marks carefully and promptly in order to inform students of their overall achievement for the course. Since each coursework normally carries different weightage, tabulating marks does not only involve collating of marks but also includes some calculations.

Although all of these tasks are quite simple in nature, they require a lot of time on the part of the instructors. As a coursework management system, CMS helps instructors to handle the process of collecting, marking and tabulating marks of coursework and thus reduces tremendous amount of energy on the part of the instructors.

The process of teaching and learning involves two parties: instructors or teachers and students. In a traditional mode of learning, there is direct communication between instructors and students. In a computer-based learning model, a computer-based system acts as an intermediary between instructors and students.

There are a few means or protocols which can be used for students to communicate with a computer-based system. Students communicate with CMS through e-mail since this is the most common and easiest means of communication. For example, almost all students at our university have their e-mail accounts and they always use this means to communicate with their friends and instructors. The students can access their e-mail accounts either from the computer lab, library, and cyber café or from their homes.

To submit a coursework, a student has to send an e-mail to a given address. CMS will then automatically reads all e-mails submitted to that particular address and then informs the student that his or her e-mails have been received. After the instructor has read and marked the assignment, they can instruct CMS to convey the feedback about the coursework to the student, also by sending e-mail. An instructor has an option either to give individual feedback or general feedback to all students.

Before a student can submit a coursework, he or she needs to register with the system. The registration process is also done using e-mail. A student who would like to register for the course will send an e-mail describing information about himself or herself to the system. The system will acknowledge the registration through e-mail.

This section will explain how an instructor would use CMS by looking at the facilities provided by the system. The full explanation of how to use CMS can be found at http://www.ftsm.ukm.my/jabatan/tk/amz/cms/IntroCMSWeb.htm

CMS was designed as a window-based system. By calling CMS, the introduction window will be displayed. The user then has to enter user login and password. Once the user login and password has been correctly entered, the course selection window will be displayed.

This window provides four functions for the user. The first function is to configure the system, which is normally done when a user uses the system for the first time. During system configuration, the user needs to input the e-mail system, e-mail server, e-mail address and e-mail directory. The second function is to change the user's password. The third function is for selecting a current course and the fourth function is for creating a new course.

Once a course has been selected, a user will be presented with the main menu. Options that are provided in the main menu are divided into two categories: course level and assignment level. In order to use the system, the instructor would start by selecting "Course Configuration". Next he or she can go to "Students Registration" to capture information about students who are registering for the course. Facilities provided at the assignment level concerns with building up assignment, checking of submission and marking of assignments. Two functions, "listing of marks" and "e-mail to students," are provided both at course level and assignment level.

To set up an assignment, an instructor would select "Build Assignment". Setting up the assignments requires some input from the instructor such as type of assignment, source of assignment, language, date of submission, question of the assignment and the type of marking.

Generally, essays can be categorized into two types: close-ended and open-ended (Saadiyah 1999). An open-ended essay tests the student's ability to present ideas effectively. On the other hand, a close-ended essay is used to assess the student's understanding of a particular subject or concept. Various issues such as student's language ability and organization of ideas are not as important when marking close-ended essays as compared to open-ended essays. Thus, assessing an essay is a challenge since there are numerous aspects that need to be evaluated.

CMS supports all of three types of marking: holistic marking, itemized marking and off-line marking. Off-line marking requires the instructor to input the students' marks directly into the system's database.

The first type is the holistic marking, which is based on a single, integrated score of writing behavior. The human marker is required to respond to the writing as a whole and it emphasizes on what is done well and not on deficiencies (White 1985).

The type of holistic marking that is supported by CMS is based on a marking scheme that was presented by Cohen (1994). Cohen (1994) has categorized essays into five different grades.

An A grade essay is one which is well organized and coherent; its main idea is stated very clearly; the choice of vocabulary is excellent; and there are no major or minor grammatical, spelling and punctuation errors.

A-B grade essay is moderately well organized and relevantly coherent; its main idea is clear; the vocabulary is good; there are no minor grammatical errors; and only a few spelling and punctuation errors.

B-C grade essay is not so well organized and somewhat lacking in coherence; its main idea is not so clearly indicated; the vocabulary is fair; there are some major and minor grammatical errors; and there are also a fair number of spelling and punctuation errors.

C-D grade essay is poorly organized and relatively incoherent; the main idea is difficult to identify; vocabulary is weak; grammatical errors appear frequently; and there are a few spelling and punctuation errors.

E grade essay is very poorly organized and generally incoherent; the main idea is absent; use of vocabulary is very weak; and grammatical, spelling and punctuation errors are very frequent.

An analytic marking scheme uses separate scales for marking purposes. Each scale assesses a different aspect of writing for example, content, organization, vocabulary, grammar and mechanics. An itemized marking scheme is a more useful tool for the standardization of the grading than the holistic marking (Weir 1990). The inconsistencies of marking between human markers can be greatly reduced in itemized marking.

Depending on the type of essays, each aspect of writing would be allocated certain marks. For example in close-ended essays, more marks would be allocated for content, while in an open-ended essay, more marks are normally given for the organization, vocabulary and grammar. An instructor would assess an essay item by item. The total mark for an essay is the sum of the marks obtained for each of the item.

CMS supports two approaches of analytic marking. An instructor could use holistic approach, where each item is marked holistically. A typical marking scheme for analytic marking based on holistic approach is detailed by Weir (1990).

Another marking scheme is "point by point marking". In this approach, an instructor would assess an essay by looking at the presence of relevant points. Certain marks are allocated for each point. The total mark for the essay is the sum of all marks given.

Itemized marking divides the writing components into eight sub-sections, namely content, spelling, vocabulary, grammar, language style, format, idea and strength of argument. If itemized marking is selected, instructors have to allocate marks for each of the subsections. Instructors must also identify whether each subsection will be marked holistically or point by point.

When the deadline for submission has passed, the instructor could use the "Check Submission" option in the main menu to obtain answers submitted by students.

For marking assignments, the instructor could select "Marking of Assignments" option from the main menu to accomplish the task. This interface would display the course code, student’s ID, question and student's assignment and the marking tool as defined by the instructor when setting up the assignments (holistic or itemized).

For holistic marking, the itemized marking components will be disabled. To allocate a mark, the instructor needs to press "Total". A list of option will be displayed and the instructor needs to select an option.

For itemized marking as in the given example (see http://www.ftsm.ukm.my/jabatan/tk/amz/cms/UseCMSWeb.htm) "Content" is marked "point-by-point". In this case, each time "Content" is pressed, a certain mark would be added and information for that particular part of the essay is given. Marks for content will be added to the text of the essay. On the other hand, "Style" is marked holistically. In this case, if "Style" is pressed, an option list will be displayed and the instructor can then select the appropriate grade for "Style". The total mark for the essay will be added automatically.

Marks that are obtained by students will be tabulated automatically. An instructor can see the mark by selecting "List of Marks". To see the total marks for the course, "List of Overall Marks" option can be selected.

CMS has been used by some of the instructors at Universiti Kebangsaan Malaysia for the last two years. Experience in using CMS has been very encouraging. In one course, where CMS was used, many students appreciated receiving e-mails informing them that their submitted work had been received. Previously, they had to check in person with their instructor in order to determine whether the instructor had received their assignments. Students also welcomed prompt feedback from instructors. In the traditional arrangement, students might receive feedback at the end of the semester, which would be too late for them to improve in their writing ability.

Instructors who were using CMS stated that by using the system, they were more organized and could focus their time more on the content of the coursework rather than on the administrative aspects of teaching. One of them had reported that by using CMS the problem of missing coursework that had frequently occurred in the past was solved. Students who did not submit their coursework were informed automatically and hence they were able to submit their coursework if they wished to do so. Another instructor had mentioned that he took less time to mark students' assignments using this system as compared to marking these assignments manually. He also mentioned the issue that he had received fewer complaints from his students regarding their marks as these marks were more consistent and accompanied with appropriate feedback. Previously, students were only given grades without any explanation about their performance. Another positive comment received was the reduction of time in tabulating overall students' results for the course. However, the instructor concerned proposed that CMS should also include facilities for displaying students' performance by using graphs.

Since CMS is e-mail-based, the availability of a reliable e-mail system is most critical for the successful usage of CMS. When we used CMS for the first time in the year 1999, we faced many problems due to the instability of our e-mail server. Some of the students' e-mails were not received for some reasons and they had to resubmit their assignments a few times. Now, with the installation of a new e-mail server, this problem has not arised any more.

In its present form, CMS suffers some limitations that need to be corrected before it can be distributed to a wider audience. We are still experimenting and testing the system in order to identify some other weaknesses of the system. Work is currently being undertaken to improve the system. Once the system is ready, we plan to distribute it as a freeware so that it can be widely used by other fellow instructors.

Another work, which is currently undertaken by the research group, is to provide automated essay marking tools for CMS. We hope that the availability of such a tool will further enhance CMS and will ease the burden of instructors in marking students' assignments.

The authors would like to thank Zainun Romli and Norhayati Mat Kassim for implementing the system.