GOUTAM ROY and SAMIR RANJAN NATH wrote about the necessity a short course on Educational Evaluation and Assessment
Abstract: The aim of this study was to explore the feasibility of a new course on educational evaluation and assessment. In-depth interviews were carried out with a checklist prepared through field trials. Number of respondents was 19, 16 from the non-government organisations and three from the government education department. Junior, mid and senior levels officials were in the sample. The findings reveal that short certificate courses are needed on educational evaluation and assessment. Both the government and non-government education officials were interested in it. Duration of the course was suggested to be of four weeks. Involvement of high officials of the ministry and the NGO leaders in the course advisory board should be considered seriously.
Introduction
The government is the major provider of primary and basic education in Bangladesh. As a supplementary to the State initiative a large number of non-government organizations (NGO) run primary and basic education programmes in the country. Most of these initiatives follow the non-formal mode of education. The government programme is partly financed from the revenue budget and partly from the development budget, which mostly comes from external sources. On the other hand, the non-formal initiatives are mostly financed from the external sources. Some of the NGOs run education programmes alone and some as part of an integrated development approach.
Assessment and evaluation are two vital issues for any development programme, which needs to improve the quality of programme. A good development programme considers these as integral part of it. Continuous assessment of programme components help the managers understand whether the programme is moving towards its goal. If not, measures can be taken based on the assessment results. Such a continuous process helps achieve the goals. On achieving net enrolment rate over 85% with gender parity, quality is the major concern in primary and basic education in Bangladesh. Need of regular assessment and evaluation of various quality aspects is then become more important.
There is a scarcity of qualified human resources to assess education programmes – both in the government and outside. Government education department has few research officers who generally compile facts and figures on the programme. Due to lack of both financial capacity and qualified human resources, it is not possible for the government to undertake assessment and evaluation studies regularly. On the other hand, with few exceptions, it is not possible for the NGOs to create such facilities. District and upazila level government education offices and the majority of the NGOs lack the capacity of assessment and evaluation or do not have such facilities at all. Monitoring teams are there in some cases, but the members of these teams are not trained enough. Field experience is the only asset, which is not adequate. However, in both the cases, external evaluation is done with the initiative of the donors – the funding agencies of the programmes. Sometimes, evaluation activities are contracted out to consulting farms. Foreign consultants are also involved with mid-term and final evaluations of the education programmes. These are not done as part of the programmes, hence can rarely help in improvement of the programmes during intervention phase.
BRAC is the exception in many ways in the non-governmental development sector. It has an active Research and Evaluation Division, which has been contributing to BRAC’s programme development since 1975. It has a trained team of researchers from various disciplines. Continuous assessment and evaluation of BRAC education programme is one of its activities. Besides, evaluation of national education system is also done every year through Education Watch – a civil society initiative to monitor education system in Bangladesh.
Some universities and the teacher training colleges offer bachelor and masters level degrees in education. These courses contain topics on educational evaluation and assessment. There is no provision of short course for the young professionals of education who came from other disciplines rather than education.
Concept of educational evaluation and assessment: Evaluation is the process of determining whether an item or activity meets specified criteria. It has several distinguishing characteristics relating to focus, methodology, and function. Evaluation mainly assesses the effectiveness of an ongoing programme in achieving its objectives, relies on the standards of project design to distinguish a programme’s effects from those of other forces, and aims at programme improvement through a modification of current operations.
On the other hand, a systematic process for determining goals, identifying discrepancies between optimal and actual performance, and establishing priorities for action is called assessment. The systematic collection of information about the activities, characteristics, and outcomes of programmes to make judgments, improve effectiveness, and/or inform decisions about future development can be called as assessment.
Methods
Objective: This study aims to explore the need of a new short course on educational evaluation and assessment. The following research questions were set to fulfil the objective.
• What is the existing situation of educational evaluation in formal and NGO sector?
• What types of problems do the evaluators face and how do they solve these?
• Whether there is any need of a short course on educational evaluation and assessment?
• What should be the contents for the course?
• What structure do they want for the course?
Sample: The targeted respondents of the study were two:
1. The field and central level education managers of the NGOs; and
2. The education officers and the instructors in the upazila education offices of the government
A total of 19 people working in the field of education were consulted – 16 from 12 NGOs and 3 from the government set up. Six of the NGO respondents were at senior level working in the respective head offices, six mid-level professionals and rest from field offices. Eight of them were directly working in evaluation and the rest were working as manager, researcher, policy maker, supervisor, etc. The three government officials came from three different offices such as Directorate of Primary Education, Upazila Education Office and Upazila Resource Centre. All of them were junior staff. List of the respondents are provided in Annex 1.
Tools and data collection: Data were collected through a checklist (Annex 2). Several pre-tests were done prior to finalise the checklist. The draft instrument was discussed with government and NGO education officials and the researchers at BRAC Research and Evaluation Division. Each of the respondents was interviewed one-by-one at their places. Although the interviews followed the checklist, however, other relevant issues were also discussed. Besides, attitudes, behaviour, and the body expressions of the respondents were observed during interview in order to understand them. The researchers conducted all the interviews. A gentle and friendly environment was tried to ensure during interview. The researchers had to describe their intention first, which sometimes added by a lecture on the concepts of evaluation and assessment in education. The fieldwork was done during October 2005.
Findings
Existing evaluation system: The NGOs under this study operated non-formal primary and basic education programmes with the financial support from the large national NGOs or the foreign donors. None of these NGOs had their own set up for educational assessment and evaluation. However, they needed to do it for the donors. Donors’ advice, guidance and supervision were required for this.
Some of the NGOs prepared some formats, which they were using for collecting data on a regular basis. Some of these formats were prepared by the donors and some by the respective NGOs. A few of them were compiling the data collected from various locations or schools to get the overall situation at a glance. However, none of them really analysed those data to have valid conclusions about the progress, prospects or problems of the programmes.
Three types of stakeholders were generally brought under evaluation- the students, the teachers and the programme organisers or the supervisors. General type of information such as attendance, infrastructure, etc. were often collected. Assessment of students’ performance was one important area in which the NGOs are interested. This was generally done by the programme organisers or the supervisors through taking test on purposively selected questions. According to most of the respondents, if the students did well in the test they considered it as a reflection of good programme. Few of the organizations evaluated the teachers and other programme related persons but most of them did not do this.
Most of the respondents had no idea about the programme evaluation. Some of them thought that evaluation of the whole programme is not necessary. Annual evaluation has been done in most of the NGOs; however, the process of evaluation was not scientific. They mainly examined the students or measured some learning outcomes of the students, which is only a part of overall evaluation of the programme.
According to the senior staff of the NGOs, they usually did not find any problem while doing evaluation activities. The main reason behind this was that in most of the cases they followed the formats prepared by the donors. Again, in some cases, the donors evaluated the programmes with the help of some local experts.
Knowledge on evaluation: Most of the respondents have no clear knowledge on what evaluation is. A large number of them thought that measurement and evaluation are same. They could not distinguish the difference among evaluation, assessment, and measurement. Some of the respondents claimed that evaluation, assessment, measurement, supervision, and monitoring contain almost the same meaning. They had also no idea about formative or summative evaluation.
Although most of the respondents had a little or no knowledge on evaluation, but they realized that regular evaluation and assessment is necessary for any education programme. Some of them mentioned that it would be good to evaluate a programme at its implementation phase not at the end.
Training received: The organizations under study had no trained evaluator. Mainly the programme coordinators and supervisors evaluate some parts of the programmes. Almost all of the respondents had no formal training on evaluation. Few of them received training for 2/3 days from CAMPE, NGO Forum, and other organizations but very few of these were related to educational evaluation and assessment. Some of the junior respondents claimed that they learned some techniques of evaluation during the fieldworks and also from their seniors. However, they also added that they did not know whether they were following a scientific method or doing the right thing. Nevertheless, all of them got some orientation training when a new programme is launched.
Realization of junior professionals about needs of training: The junior staff of the organisations that we discussed felt a need of training in order to improve their skills. Some of them told that they could not perform well due to lack of knowledge about the work they were doing. They also thought that in many cases they were not being able to relate their knowledge with the fieldwork. Training could help them to bridge these two. They also mentioned that they were willing to improve their quality of work and wanted to improve the school programmes they were engaged in. Some of them, in this regard, were seeking for a short certificate or diploma course.
Realization of senior professionals about needs of training: The senior professionals also realized that if they had training on assessment and evaluation of school programme they could run their programme better. However, majority of them were not interested to do such course now. It is because they thought that they were too senior to take such course. Besides, time constraint is another factor. As senior persons in their offices they were to involve in various types of works. Some of them were afraid that they would not be able to suit with the classroom situation because they left it many years ago.
Respondents’ interest: Both the senior and junior professionals welcomed a new course on educational evaluation and assessment. They had a realization that the evaluation mechanism of school programmes in Bangladesh is not much developed. They also commented that such a strong mechanism could help programme development and ensure quality. Even the research staff in DPE opined that he could be benefited from such a course.
When the respondents were asked whether they would participate in such a course if any private university offers it? The junior professionals showed their interest to participate and told that they would encourage others to participate. A certificate or a diploma course on educational evaluation and assessment would be helpful for their career development. They added, it was important for them to know how to write an evaluation report and how to evaluate the whole programme. They hoped that a new course on evaluation would include such contents. On the other hand, the senior professionals did not show their interest to participate in such a course; however, they agreed to encourage their junior staff to participate.
Top management people of the organisations such as the executive directors and the deputy directors said that they had plan to build skilled work force on school evaluation. In order to do so they would initiate to arrange training course in their organisation. According to them, a new certificate or diploma course could be suitable mainly for the fresh staff. They also assured to think positively to send their staff to such course if any private university offers.
Barriers in participation: The main barrier for the junior staff to enrol in a course, as they said, was money. Some of them also said that there was none to look after their work if they were send for training. Again, the organisations might not be willing to spend money for their training. If they like to enrol in a self-financed course the organisation might not allow them leave with pay. On the other hand, the senior staff said that they would not have time to attend the course but money is not a serious problem for them. Officials working outside of Dhaka had another type of problem; especially those had no residential arrangement in Dhaka. Residential training would be suitable for them.
Course contents: The respondents were unable to specify actually what they wanted to learn from such a course. Most of them opined that the educational experts are the appropriate persons to specify the contents of the course. Some of them mentioned two intentions – firstly, the way of understanding the quality of the programme i.e., evaluation technique, and secondly, expressing the findings in writing.
Course duration: It was difficult for the respondents to say anything about the duration of the course because they were not aware about the course content. What primarily they could say was that it should be a short course. When the respondents were asked to think about it considering their workload and time constraints in offices then they provided some numerical answers. However, they were afraid whether they could provide a practical answer. According to the officials working in Dhaka city, the course duration can be of one month. On the other hand, respondents from outside of Dhaka mentioned that it should not be more than two weeks. A good number of them even proposed for a shorter course with one-week duration.
Class time: The Dhaka city respondents said that they would prefer an evening course – starting from 6.00 p.m. to 9.00 p.m. in the weekdays. The others preferred a daylong course.
Course fee: The respondents opined for a minimum course fee. In favour of their response they said that a minimum course fee would allow those to participate who has no sponsor. Some of them suggested for a scholarship programme for the poor. On the other hand, some of them pointed out that if the course offering university request the NGOs to send students it would be helpful for them to participate.
Opinion of the government officials: Not much variation was observed between government officials and non-government professionals’ knowledge on educational evaluation and assessment. The three government officials we talked received a general training after their appointment. Nevertheless, the training was not related to assessment or evaluation of school programme. The government officials also mentioned that they were not given the task to assess or evaluate the school programmes in their areas. As they said, they were given some prescribed forms which they filled up during school visit. The upazila education officer added that he never compiled the information gathered through the AUEOs and nobody wanted to know the overall situation of the upazila.
Government officials realized that they had lack of knowledge on educational evaluation and assessment. They also added that the education programme would be benefited if the government asked them to evaluate the programme in their areas and take necessary actions for programme improvement. Without such an interlink between evaluation and programme development there is no benefit of receiving training on assessment and evaluation. All the three respondents said that they were willing to take training on educational assessment and evaluation.
The respondents opined that the government had many institutes for educational training. These training institutes can arrange such courses. If the government do not want to do so, any NGO or university can arrange it.
The government officials mentioned that they had to face some problems to get training from NGOs. They needed permission from upper level and the permission process was complex and bureaucratic. They also added that sometimes the permission came after the programme started and thus the trainees failed to join. This loosens interest of the participants.
Conclusion and recommendation
The people working in the field of education, whether in the government or in NGOs, do not have adequate knowledge on programme assessment and evaluation. However, the respondents think that if they were trained in this area they could do better for the programme. They are also convinced that regular assessment and evaluation of education programmes are important for their sustainable development.
Specific recommendations
1. A short certificate course on educational evaluation and assessment can be initiated for junior education officials/supervisors.
2. A panel of experts can be assigned to design the course. However, it should contain practical exercise, fieldwork and case study.
3. It can start as an evening course. However, some daytime would be needed for fieldwork. A residential course can be thought later on. Duration of course should not exceed four weeks.
4. It would be difficult for the education professionals to join the course if there is no financial support. A low cost course can be planned and the organisations can be requested to send participants. Cost of the course can primarily be shared by both the initiating agency and the participating organisation.
5. External resources could be explored to provide scholarships.
6. Seminar, press conference, etc. can be arranged and leaflet be prepared to publicise the course.
7. It may be hard to bring the government officials in the course if their bosses are not consulted. Government education ministry can be involved in the advisory committee of the course.
Annex
Annex 1. List of participants
Government
1. Ms. Jannatul Ferdousi, Assistant Upazila Education Officer, Tongi, Gazipur.
2. Ms. Mojida Akhter, Instructor, Upazila Resource Centre, Demra, Dhaka.
3. Mr. Sanjay Kumar Dutta, Education Officer (Evaluation and Monitoring), Directorate of Primary Education, Dhaka.
Non-government organizations
1. Ms. Ayesha Rahman, Executive Director, Society for Action Research and Development, Dhaka.
2. Mr. Baharul Islam, Programme Coordinator, Society for Action Research and Development, Dhaka.
3. Mr. HM Showkat Ali, Programme Officer, Gana Unnayan Society, Dhaka.
4. Mr. Imraj Hossain Emu, Programme Officer, Shurovi, Dhaka.
5. Ms. Marina Akther, Programme Coordinator, Jagoroani Chakra Foundation, Jessore.
6. Mr. Jagazziban Biswas, Technical Officer, Jagoroani Chakra Foundation, Jessore.
7. Mr. Khalil Uddin, Programme Organizer, Sabalamby Unnayan Samiti, Netrakona.
8. Ms. Lamia Naznine, Training Officer, Grameen Shikkha, Dhaka.
9. Ms. Mukti Rani Saha, Technical Officer, Grameen Shikkha, Gazipur.
10. Mr. Leo Palma, Deputy Director, Bangladesh Extension Education Services, Dhaka.
11. Mr. Md. Hanif Miah, Education Officer, Resource Integration Centre, Dhaka.
12. Mr. Miron Alam, Officer, Technical Assistant for Rural Development, Dhaka.
13. Mr. Md. Motiar Rahman, Executive Director, Aid for Poor People, Dhaka.
14. Mr. Mofizul Islam, Programme Officer, Aid for Poor People, Dhaka.
15. Ms. Najrul Islam Mandal, Programme Co-ordinator, Bangladesh Association for Community Education, Dhaka.
16. Mr. Rafiqul Islam, Programme Organizer, Socio-economic and Rural Advancement Association, Dhaka.
Annex 2. Checklist for interview
A. General questions
1. How many education projects do your organization have at present? Mention the name of those projects.
2. Do you/your organization perform regular evaluation and assessment of those projects?
3. How do you do such evaluation and assessment?
4. What types of problems do you face at the evaluation period?
5. How do you solve those problems?
6. Do you get help from others at evaluation process? Who help you?
7. Do you think regular evaluation and assessment is needed for every education related works or projects?
8. Have you got any internal or external course or training on evaluation and assessment? What type of training was that or those? Was it single or short-term lectures or hands-on applied workshop or course or Professional, Masters or Doctoral Degree? Were you satisfied getting that course?
9. Do you think any training or course is needed about evaluation and assessment for research fellows or education professionals of your organization?
10. How could a new course help you for your professional development? What content do you want from the course?
11. Do you have any plan to build internal manpower at evaluation and assessment in your organization?
12. If any organization starts a short course on Educational Evaluation and Assessment, would you be interested to send your professionals for training?
B. About the course on educational evaluation and assessment
1. What should be the course duration?
2. What should be the class duration in a day?
3. What would be the suitable time for the class?
4. What should be the course fee of this short course?
5. If you have another suggestion about the course, please mention.
Authors: Goutam Roy, Staff Researcher and Samir Ranjan Nath, Research Coordinator, Research and Evaluation Division, BRAC, Bangladesh. Email: goutam.r@brac.net This e-mail address is being protected from spambots. You need JavaScript enabled to view it
This article is taken form BRAC with permission.
Leave a Comment