Newsletter No. 79

CUHK Newsletter No. 79 4th December 1995 3 Focusing on Assessment and Quality Assurance TwoMoreProjects Fundedby TeachingDevelopment Grants 1994-95 The CUHK Newsletter has so far introduced to readers five out of the seven projects that have been awarded teaching development grants. In this issue, we bring you the last two projects, both of which lay special emphasis on assessment and quality assurance. ACCESS: Assessment o f C u r r i c u l a , Co u r s e s, E x am i n a t i o n s, a n d S t u d e n ts Sy s t em ( H K $ 5 8 8 , 1 5 5) Grant recipient Prof. C. B. Hazlett, Educational Services, Faculty of Medicine H o w valid are course examinations? How effective is student evaluation of courses and teachers? This project addresses the issue of teaching quality by examining these two pertinent issues. Prof. Hazlett points out that very often examinations developed by university teachers leave much to be desired, and thus examination results are neither necessarily dependable criteria for grading and promotion, nor reliable indicators of teaching quality. Different examination formats have different shortcomings. For example, frequently multiple-choice questions assess only relatively simple cognitive skills (recall or recognition of factual data), while essay questions often fail to gauge higher problem-solving skills because of inconsistencies in marking. Prof. Hazlett also observes that evaluations of teaching quality, course structure, and examination processes by students often fail to bring about constructive changes. This is due partly to the inflexibility of the evaluation system which fails to take into account crucial factors like the number of students responding to aquestion (which will affect the precision of an instructor's or a course's rating), and the number of classes rated using the same question (which will affect the precision of comparative ratings). To deal w i th the problem of examinations, Prof. Hazlett's team is in the process of creating a large bank of multiple-choice questions that are categorized by properties such as difficulty level, content, and cognitive focus (that is, whether they test problem-solving skills, or factual recall, or both). Item banks from other medical schools are being acquired and added to those of the University. For the latter, a representative from each participating department within the Medical Faculty is helping to design additional questions. ACCESS also provides for a post-examination analysis and reporting service. A ll multiple- choice questions used will be subjected to statistical analyses, in terms of overall test reliability as well as the difficulty level and discrimination capability of each question. To increase the usefulness of course and teacher evaluations by students, the project team is also acquiring from the University of Alberta and the University of Michigan a bank of student evaluation questions which are catalogued according to four components: (1) student development which includes aspects of knowledge, skills, self-concept, and responsibility; (2) instructor abilities such as clarity of instruction, rapport with students, and ability to motivate; (3) course effectiveness which covers areas such as outline and organization, examinations and grading; and (4) instructional format: whether it is lecture/seminar, bedside teaching/clinical supervision, or assignment/ laboratory work. The instructor may select from this catalogue the most relevant items for evaluation and may add items of his/her choice. I f the item has been previously used, the rated instructor can be plotted in reference to a normative group of his/her choice. Student feedback will also be analysed and then communicated to the respective instructors, and i f requested, to their department and faculty heads. To put these results in the right perspective, information will be given on the number of respondents to each question, the median rating of responses to each question, and the comparative ratings between, for example, the median of the course in question and that of other courses in the same department. Since its approval in March 1995, the grant has been used largely for buying a computer, an optical scorer, and a precision printer, in acquiring item banks, implementing and modifying appropriate retrieval software, and in hiring a consultant from Canada, an assistant computer officer, and a research assistant. The system is expected to be tested by Christmas, and ready for implementation after March 1996. D e v e l o p m e nt o f a P r o g r a mme o f Teaching Ma t e r i a ls in Quality Assurance (HK$65,000) Grant recipients Prof. Ann Mackenzie and Ms. Anne Chang, Department of Nursing Howcan we measure the quality of service in nursing? Prof. Mackenzie feels that 'It is very difficult to measure quality.' This difficulty stems from the many perceptions of quality, and is one of the questions being explored in the education package on quality assurance being developed by the Nursing Department. The aim of the project is to improve the teaching of quality assurance in nursing by promoting student-directed learning and utilizing critical thinking in the application of theory to clinical practice. In the long run this means that students will be able to become actively involved in the promotion and development of nursing quality assurance in the workplace, be it in the community, hospitals, clinics, industries, or schools. Quality assurance is definitely an issue in nursing— it provides nurses with a form of practice appraisal, a process to assist in defining the profession, and a form of demonstrating accountability, and all of this is important in today's consumer society. The educational package aims to answer the following questions: What is quality assurance? Why should nurses be involved in it? What form should it take? How can we measure quality? How can the concept of quality assurance be applied to the provision of nursing services? How is nursing quality assurance being utilized overseas and in Hong Kong? Can we improve on what we have now, and if so, how? The project to date has seen the development of a reference resource comprising international and local papers on quality assurance in nursing and allied health professional literature, revamped teaching materials including multimedia teaching aids, and tutorial materials to guide students through segments of self-directed learning. The self-directed learning materials are based on the principle that more comprehensive knowledge comes from knowing not only 'what' but also 'how' . An authentic test of the acquisition of knowledge is the ability to apply knowledge in the actual situation. Thus the tutorials provide both thinking and practical exercises to promote the link between theory and workplace for the students. It is during these times that the students can freely discuss and analyse quality assurance developments, as well as develop suitable programmes and tools in their clinical areas (clinical areas being those areas where the students are involved in the practice of nursing, for example, the orthopaedic ward/unit, the delivery ward/unit in obstetrics, or the community). It is also during these sessions that students can reflect on their learning and identify gaps in their knowledge. The focus of the self-directed materials is on developing quality assurance programmes and tools suitable for setting and measuring standards of health care. In addition students will examine ways of using the results of measurement to reinforce or amend health care practice. Prof. Mackenzie hopes that the use of this package will lead to more interactive learning with students becoming confident and proactive in promoting nursing quality assurance in the workplace. The project itself forms the basis for further development of quality assurance using educational technology. Piera Chen