Introduction

The higher education reform processes in the Republic of Macedonia point out the students’ active role in gaining the necessary knowledge and competencies, as well as the role of the modern approaches in the teaching practice. These new approaches have marginalized HE teachers’ role; instead of being treated as profound and major source of knowledge, they have been given the role of mentoring students’ activities. Furthermore, modern teaching indicates that HE teachers’ teaching/lecturing should be condensed but comprehensible, clear and primarily focused on examples from the practice. More precisely, it should be based on the principles that relate theory to practice and on the relevance of providing students with knowledge that will be applicable in their work and life. In addition, the modern teaching approaches demand that the assessment process should be directed towards eliciting students’ knowledge. In this sense, both the society and the higher education institutions are interested both in developing mechanisms to measure the quality of the teaching process and implementing various teaching-and-learning self-evaluation and evaluation instruments[1].

As any process of evaluation, HE teachers’ self-evaluation is a process in which the set criteria are compared with the aspects of their own work applied to assess and grade students’ achievements. This implies planning, realization and taking an action in accordance with the results. The self-evaluation takes part during the whole teaching process, it is the beginning and the end of a single developmental cycle.

The most important goal of HE teachers’ self-evaluation procedure is to know the effectiveness of their own work, and accordingly, if a necessity arises – to introduce modifications or changes into the planning and realization of their further activities for the sake of their teaching practice enhancement.

Depending on the period when it is conducted, the self-evaluation can be either formative or summative. Through the personal self-evaluation, HE teachers check their successfulness, discern the possible poor effects of their teaching practice, analyze them, etc. Then, they make the necessary changes in order to make improvements and provide what is good and desirable for their high-quality teaching. In accordance with the results obtained from the self-evaluation, HE teachers anticipate their future steps to promote their work, and this leads to the necessary further development of the higher institution they teach in.

The teaching process is a complex activity. Its efficiency depends on HE teachers’ knowledge and skills to address various students’ motives, competencies and intellectual possibilities. In this process, HE teachers transfer knowledge and skills to students, utilizing the most suitable forms and methods that will facilitate this transfer and provide an objective assessment of students’ achievements. Besides, every teacher is interested both in the quality of his teaching and its effects on his students. Therefore, HE teachers often talk to students about various aspects of their teaching, primarily about the effects and the quality of the various activities in the teaching process. These conversations represent a type of self-evaluation. They provide HE teachers with feedback about the quality of their activities in order to direct their teaching towards students’ needs and interests.

On the other hand, every higher education/HE institution is interested in enhancing the quality of the teaching stuff. Thereby, in accordance with the HE Law, it conducts a Faculty Self-Evaluation as well. Thus, each faculty teaching staff ’s work is evaluated through a survey the students fill in about each HE course teacher. The self-evaluation in the Republic of Macedonia is regulated with the current Law for Higher Education, put into force in March 2008[2]. According to article 77 of this Law, every academic unit of the University must appoint a Self-Evaluation Committee that will conduct the self-evaluation. This self-evaluation must be done in accordance with the requirements stated both in the Statute of the University and the Statute of the Higher Education Institution. This committee consists of six members of the Faculty Education and Science Council (ESC), elected via secret ballot by the other members of the respective faculty ESC, for a four-year mandate, and of two students, with a two-year mandate, elected by the students. The self-evaluation is conducted in accordance with the Self-Evaluation Guidance for providing and assessing the quality of the HE institution, promulgated by the University.

The article 77 states that the self-evaluation is to be conducted periodically every three years. The same article instructs that students assess HE teachers at the end of the academic year. The grade of the HE teachers’ work is taken into account when promoting HE teachers to a higher rank. The grade from the students is provided through a survey that students fill in. It is obvious that students have a vital and inevitable role in assessing HE teachers and determining their future career.

However, bearing in mind such a sensitivity of this issue, this procedure leads to many questions about the validity and objectivity of the grade that HE teachers are given from the students. The questions listed below appear to be of utmost importance:

– Do the statements in the survey reflect all aspects of the multidimensional teaching construct?

– Are all of the statements in the survey equally relevant and influential regarding the teaching?

– Are the students capable of grading the HE teachers objectively regarding the way they teach and the way they manage all the activities during the teaching process; let us not forget the complex nature of the assessment, especially that one of the objectivity issue?

– Are the results from the survey valid and objective indicators for the HE teachers’ work to an extent that they should be indisputably taken into ac-count when their promotion into a higher rank is to be decided on?

In line with the above stated, the aim of this paper is to show that the self-evaluation is both a relevant segment of the activities of a Higher Education institution and a mechanism, which provides important information about the HE teaching. This information should be strictly used only formatively. Yet, there have been numerous factors beyond the authority of the HE teachers which keep emerging and displaying severe impact on the validity and objectivity of the results.

The objectivity and validity issue of a prescribed survey template, used to evaluate HE teachers’ performance, is actually a constituent of the question on the intellectual refinement of HE self-evaluation not only in Republic of Macedonia but in the Central and Eastern Europe (CEE), in EU, and in the whole world, as well.

Various aspects of this type of HE institutional self-evaluation have already been frequently discussed at respective European and global gatherings. In this regard, striving to become highly compatible with the European and the world educational systems, the Macedonian educational system puts all its efforts into elevating the cultivation of the self-evaluation implementation. Furthermore, it is exceptionally important to point out that – being a candidate for accession to the European Union since 2005, and a full member of Bologna Process since 2003, Macedonia has long since accepted the principles of Bologna Declaration and keeps implementing them, including the ones related to the HE quality self-evaluation. An equally significant fact is that – as a country often sampled in the developed countries’[3] research on the HE self-evaluation implementation, Macedonia has long been encompassed in both the European and the global educational world, especially in regard with the dilemma arising over the survey answers and the survey results interpretation. This type of re-search is considered immensely relevant as it emphasizes the importance of signifiers, indices, and relevant statements, which constitute the teaching construct in general, and accordingly reflect the HE teachers’ performance for enhancing CEE and Macedonian HE systems and their compliance with the HE systems on a global level. In addition, the universities of R. Macedonia follow the principles of the European Association for Quality Assurance in Higher Education and the European University Association principles when preparing the instruments to be used for university and HE teachers’ performance evaluation.

Background information

In the Macedonian reviewed literature, the question connected with stu-dents’ survey to grade HE teachers’ teaching/lecturing has neither been researched nor elaborated so far. In the foreign reviewed literature there is far more information. The results from several relatively new conducted research projects, related to the advantages and disadvantages of this method for assessing HE teachers’ performance, are as listed:

Otany, Kim & Cho (2012, pp. 531-544), examine "the influence of 13 aspects of the teaching" as part of the students’ evaluation of the teachers’ work in order "to identify the priority aspects", i.e. the aspects which are most influential in the evaluation. The results from their research show that not all of the 13 aspects have equal influence on the overall grade for the efficiency of the teachers’ work. The listed aspects they selected are considered as most influential: "clear explanation; effective use of the time; positive climate for learning and stimulating material for learning". According to these authors, "the identification of the priority aspects for the teachers’ work assessment improves the quality of the evaluation".

Kelly (2012, pp. 2-7), indicates that "'teachers’ assessment by students should be formative in nature", i.e. provide the evaluated teachers with feedback. Otherwise, the assessment" will not achieve the desirable effect". According to her, students’ survey "measures students’ feeling and opinion about the way the subject is taught and not about the quality and level of the knowledge they have obtained from the teacher". Further, she suggests five relevant aspects of the teaching that should be taken into consideration when students assess teachers’ work. These aspects are: "the course should have clear goal; the teachers should offer help and contribute to the learning process; the time for teaching and exercising should be balanced and the topics that have been learned should be relevant to the students’ future work". At the end of the course, students should give overall grade to evaluate these elements. Moreover, Kelly stresses out "the effects of the statements on the overall grade. The grade will surely affect the teachers, because they can suspect its validity and objectivity". She points out the possibility that "the survey will not be taken seriously from students if they feel that the results from the survey are not used for improvement of the teaching". She, also, indicates "the positive correlation between the grade teachers have given to the students and the grade that students in return give to the teachers for their work". In addition, she highlights the importance of "the way the results are managed". Finally, she concludes that "the statements should be clear and meaningful, and point out the significant aspects of teaching".

Marsh & Roche (1997, pp. 1187-1197) bring out the significant role of the teaching, "which is multidimensional construct that consists of many activities". The evaluation of teachers’ work should reflect this significant aspect, i.e. this multidimensionality. Further, according to them, "the surveys differ regarding the quality of the statements and the way the teaching construct is comprehended". The survey’s validity and effectiveness depends on "the content and the aspects of assessment that have been taken into account". "Inappropriate statements will not give satisfactory information". They highlight that in the teaching practice "many instruments for assessing teachers’ work do rely on the logics and pragmatics, but rarely include psychometric features", which additionally affects their validity and objectivity. Moreover, they suggest that "the instrument for assessing teachers’ work should be designed the way it can measure the separate components of the multidimensional teaching construct". It is hard to evaluate the validity of the instrument simply because no single criterion for effective teaching is enough.

Parpala, Ylänne & Rytkönen (2011) suggest that "there is a connection be-tween students’ beliefs what good teaching is and the grade they give to the teachers". Regarding the good teaching, they reveal that "the clear information and the teachers’ efforts to clarify things for students are relevant aspects of the good teaching".

Beran & Violato (2009, p. 3), indicate "the positive correlation between the grade teachers have given to students for the subject taught and the grade that students give to teachers". The students who are graded with high marks assess teachers with higher grade and vice versa. Further, in their research, they examine "the degree to which the features of the teaching and students’ engagement affect the way students grade their teachers". According to them, "various subjects are taught in various ways". For instance, laboratory practice, distant teaching-and-learning, practicum, etc., are run in different ways, and "this has an impact/ a different impact on the way students grade their teachers’ work". In addition, they indicate other features which influence the way students assess their teachers’ work such as duration of the course; whether the subject is compulsory or elective; teachers’ requirements regarding the subject, etc. They conclude that "the students’ engagements and teachers’ teaching are multidimensional constructs that consist of many activities". Their research confirms the positive correlation between students’ regular attendance of classes and the grade they give for their teachers’ performance. The students that regularly attend the classes are more motivated and give higher grades to their teachers.

Beran & Rokosh (2009), state the opinion of many professors at the University in Calgary. These professors claim that "the instrument for assessing their work is not designed appropriately and has no quality", thus, it cannot contribute to improving the teaching practice.

Hoyt & Pallett (1999) accentuate "the negative side of these surveys", especially in reference to "the results, which are used to compare teachers". According to them, "this comparison will be valid only if standardized surveys are used".

Seldin (1993) indicates that "students have limited knowledge and they should not assess teachers’ knowledge of the subject they teach, or the appropriateness of the materials they use to teach in class".

From the conducted research stated above, it is obvious that the researchers’ attention is directed toward revealing the relevant and influential aspects of the complex teaching construct. Furthermore, their research is aimed toward revealing the relevance and influence of the surveys’ statements, which are components of how one understands the phrase “good teaching”. This information shows the presence of many factors that directly or indirectly contribute to the differences in the results obtained from the students’ survey.

Methodology of the research

The research has a qualitative paradigm (content analysis). The sample consists of 12 statements in the students’ survey referring to various aspects of the HE teachers’ teaching and work, and they are to be assessed with five preset Likert scale-ranked answers. This survey is a template provided by the university for each academic unit i.e. faculty. These statements with preset Likert scale-ranked answers have been analyzed regarding their influence on the validity and objectivity of the results obtained about the HE teachers’ work, as well as regarding their importance in the multidimensional teaching construct.

The research has also a quantitative paradigm. The opinions and stances of 200 students of “Sv. Kliment Ohridski” University – Bitola, have been collected via a survey, on the one hand, and on the other – the effects of the students’ teacher-evaluation-survey have been analyzed. The opinions and stances of 100 HE teachers of the same university have been collected via a survey and analyzed with reference to the survey aspects influencing their teaching. The quantitative method has been used to address the qualitative analysis. The methods used for processing the obtained data and for reaching conclusions are analyses, syntheses and comparison.

The above stated university has been selected as a sample because the authors of this research paper are members of the academic staff of this university; they have easier access to the HE teachers’ self-evaluation and students’ teacher-evaluation-survey data. In addition, research bias has been bypassed by previous browsing the web pages of the other universities in R. Macedonia: their self-evaluation and HE teachers-performance-evaluation results published on their web pages on the Inter-net show little, or no fluctuations from our university results. Moreover, they show reference to the same teaching construct components. These similarities result from fact that – when preparing the HE teachers-performance-evaluation survey question-naires, the universities of R. Macedonia follow the principles of the European Asso-ciation for Quality Assurance in Higher Education (ENQA[4]) and of the European University Association[5].

Results and discussion

The students’ survey for grading their HE teachers’ performance at the “Sv. Kliment Ohridski” University in Bitola, in R. Macedonia, has been regularly con-ducted since May 2002. It has been assumed that the survey – as an instrument for gaining the necessary information about the HE teachers’ work is a relevant internal mechanism which will contribute to the quality of the higher education. This survey consists of 12 statements that refer to HE teachers’ work, and they are to be assessed with five preset Likert scale-ranked answers. What follows is the qualitative analysis of these statements. The results are compared with the results from the previous re-search on this topic, as well as with the results of the surveyed HE teachers’ opinions and stances.

The beginning of the survey consists of Introductory Statements section, that students should fill in with general data, such as to which academic unit they belong, which year they attend, the surveyed course they have attended, their respective achievements/grades, etc. One of these introductory statements is of particular interest for this research. It refers to how many times the student has sat for the exam (once, twice or more times). Although this question is not a part of the 12 statements, the implications which arise with the students’ answers – further down in the survey, are rather significant for the results of the survey. It certainly leads to the assumption that the survey’s designers have supposed the number of times the students have taken the exam might have some influence on the survey results. Whether the students passed the exam in the first exam session or they had to sit for the same exam many times before passing the exam does not depend on their HE teachers. It depends only on the students’ knowledge of the subject matter. However, the practice has shown that students most often blame their HE teachers for failing the exam, regardless of the fact that there are no reasonable grounds for such perception on behalf of any of the students. In this respect, there is a high-level probability that students who needed more attempts to pass the exam will grade their HE teachers with lower grades. This interpretation is in accordance with the surveyed HE teachers’ opinions, which already shows experiences with this kind of students’ attitude. This is also in accordance with Kelly’s (2012) and Beran & Violato’s (2009) research, and an indicator that a single factor which is beyond HE teachers’ authority can greatly affect the survey’s results.

Statement 1: The HE teacher is appropriately prepared, explains clearly and understandably, and evokes interest for the subject taught

From this statement, it is obvious that it has three components: a) teacher’s preparedness to teach; b) teacher’s clarity and comprehensibility; and c) the interest the HE teacher evokes with his teaching. There is no doubt, that clear and understandable teaching, and the interest that the teacher evokes with the students attendants of the course are important, relevant, and influential aspects of the effective teaching construct. However, teacher’s preparedness should not be a part of this statement in a survey the aim of which is to grade HE teacher’s work because there are strong arguments that indicate that every HE teacher is/must be well prepared. According to our analysis we provide the following arguments respectively for each of the three components (a; b; c) included in the first statement:

The HE teacher is appropriately prepared

All of the HE teachers who teach in HE institutions have completed appropriate academic studies and obtained the three cycle degrees. Within each of the three cycles of their education, their competences and preparedness have been confirmed by many distinguished professors, both by their signatures and the positive grades they have given to the degree candidate in the respective subject area. Finally, the HE teachers have already defended their PhD thesis before a Board of five experts in the field, and have been awarded the academic title of a doctor. Therefore, they have met the necessary knowledge requirements to be employed and to work in a HE institution. Accordingly, the (a) component in the first statement is completely an inappropriate, irrelevant and misleading aspect in a survey for grading HE teacher’s performance. This opinion is also shared and confirmed by all of the surveyed HE teachers who have been selected as sample of the research.

Furthermore, it is worth mentioning some additional information in support of the stance that the “a)” component of the statement is inappropriate and can affect the survey’s results. For instance, let us analyze the question about the criterion that students can competently use to provide a valid answer as to whether the HE teachers are well-prepared or not. It is clear that students’ basic criterion can be only their prior knowledge of the subject taught. Yet, the quality and expansiveness of students’ previous knowledge has nothing to do with the course teacher’s preparedness. In support of this claim, let us assume that a student has excellent previous knowledge of the Modern Macedonian Language subject and that he understands everything the HE teacher teaches. For example, he knows to determine the nouns, he knows to determine the grammatical categories of the nouns, etc. When the HE teacher is teaching about the Nouns, for example, such a student will most probably relate his own ability to comprehend the lecture to the HE teacher’s well-preparedness i.e. knowledge. Thus, in accordance with this perception, he will circle the higher alternative from the Likert scale.

On the other hand, a student may have modest or insufficient previous knowledge of the grammatical categories, in this example – the Nouns. In such case, when the HE teacher is teaching about the Nouns, such a student will most probably be attentively listening to the teacher’s lecture, taking notes down but experiencing difficulties to understand the HE teacher, or to relate his/her prior knowledge to the new material presented by the HE teacher. In the end, this type of a student will either circle the higher alternative offered with the Likert scale – believing in the simile “the more incomprehensible the lecture – the more knowledgeable the HE teacher”, or the lowest – believing his comprehending problems to be a result from HE teacher’s lack of knowledge i.e. preparedness. In both cases, it is all about individuals’ subjective perception, which should not be a part of a survey designed to provide valid and objective indicators about HE teachers’ preparedness.

The research conducted by Otany et al., (2012), and all of the examined HE teachers’ attitudes, confirm our finding that refers to the correlation between students’ previous knowledge and students’ grade given for the HE teachers’ work.

Moreover, this finding is confirmed with the modifier “appropriately” given in the a) component of the statement – “The HE teacher is appropriately prepared”. According to the Monolingual Dictionary of the Macedonian language (2011, p. 497), the meaning of the adjective “appropriate” is presented as follows: which fits to something; which is suitable; gives opportunities; and fulfils conditions for something.

The HE teacher explains clearly and understandably

According to all of the surveyed HE teachers, the second component of the first statement, which refers to whether the teacher explains clearly and understandably, is a highly significant aspect of the teaching. This is in accordance with the research of Otany et al., (2012); Kelly (2012); and Parpala et al., (2011), who also consider the clear teachers’ explanation a significant aspect of the teaching. However, this statement largely depends on students’ knowledge and this also might affect/ distort the survey’s results. It is obvious that students who have profound previous knowledge are more in a position to evaluate whether HE teachers’ explanations are clear and comprehensible. This goes with their previous knowledge and the degree to which HE teachers’ explanations match their pre-knowledge. Nevertheless, every student, regardless of his previous knowledge of the topic taught, can evaluate whether HE teachers do put enough effort into explaining certain concepts. Moreover, every student can assess whether HE teachers’ explanations related to the topic taught are good enough in general or not. All of the surveyed HE teachers agree that students’ previous knowledge has an impact on the evaluation of HE teachers’ clear and understandable explanations. Moreover, they also consider this type of students to be capable on adequately responsive sample to evaluate them.

Though the b) component of the first statement displays one of the most significant aspects of effective teaching, the different levels of previous knowledge of the surveyed students affect/distort the survey results. Therefore, this fact has to be taken into consideration and this component of the design of the first statement reconsidered.

The HE Teacher evokes interest for the subject taught

The third component of the first statement is also a very important segment of the teaching performance. This is confirmed by all of the surveyed HE teachers. However, this statement, once again, depends on the previous students’ knowledge and affects the validity of survey’s results. Let us assume that the student is keen on learning languages. He will take into account his previous knowledge and – depending on the quality of that knowledge, he/she will assess whether the teacher has motivated him. If a student has insufficient or modest previous knowledge about the topic, then the HE teacher’s lecture on the Nouns category, for example, may be boring for a student with profound pre-knowledge. In this case, the latter type of students can get an impression that the HE teacher does not succeed to evoke his motivation and will circle the lowest alternative from the scale, even if the HE teacher teaches very well.

On the other hand, if the students have excellent or satisfactory previous knowledge, the HE teacher can run through the Nouns category and then move onto more abstract concepts and serious aspects of this word group. In this case, the HE teacher will spur the interest of the profound students, but the students lacking the respective background knowledge might feel neglected by the He teacher. Thus, they can start feeling dislike of the subject being taught by the HE teacher rather than interest in it.

If we add the number of the HE teacher’s course students attendants with or without profound pre-knowledge, who will eventually be filling in the survey, then the correlation between the number of the surveyed students attendants with previous profound knowledge and the survey’s results, on the one side, and the number of the surveyed students attendants with previous insufficient knowledge and the survey’s results, on the other side, is obvious. If the group consists of more students with modest or insufficient previous knowledge, the survey will show one result. On the contrary, if the group consists of more students with satisfactory previous knowledge, the survey will show another result. The correlation between the level of previous knowledge of the students and the survey’s results – on the one side, and between the number of surveyed students of the above stated two types and the survey’s results – on the other side, is undoubtedly confirmed with the research by Otany et al., (2012) and with the surveyed HE teachers’ stances.

Statement 2: When teaching, the HE teacher is focused and able to hold students’ attention until the end of the class

This statement also consists of two components: a) and b). The first – a), refers to HE teacher’s concentration on the teaching, and the second – b), to the interest he has spurred throughout the lecturing. In this regard, our analysis of each of the two components – a), and b), included in the second statement, is given below:

The HE teacher is focused on the teaching

This part of the statement 2, i.e. its a) component, which refers to HE teachers’ concentration on the teaching, is without doubt connected with the a) component of the statement. 1.

More precisely, every teacher is well-prepared and dedicated to his work. After all, teaching is what he has chosen to be his vocation and career. Therefore, just like with the HE teacher’s preparedness, there are no grounds for HE teacher’s dedication i.e. focus on the teaching to be an element of the grading system of what good teaching is. This is in accordance with all of the examined HE teachers’ opinions.

The HE teacher is able to hold the students’ attention until the end of the class

This second part – b), refers only to HE teachers’ lecturing, and accordingly is inappropriate. It diminishes HE teachers’ role in all sorts of in-class activities: exercises, drills, checking homework assignments, project activities and all other activities that comprise the teaching practice and performance. The b) component of the statement 2 depends on various factors which are beyond HE teachers’ authority, but which might affect the survey’s results. Various subjects are taught with different number of "contact hours" per week in class and different number of practice in-class hours. For example, three hours are scheduled and designed for teaching and one or two for exercises or other type of practice activities. In the teaching practice, most of-ten, this number of classes designed for teaching is not separated from those designed for practice activities. They represent one integral unit and it is almost impossible for HE teachers to teach without applying practical examples and activities.

In this sense, this b) component of the second statement is affected by a highly relevant factor, i.e. the exact point of time when the teaching and the practical examples will occur, mingle, or be switched. This again does not depend on HE teachers’ ability to hold students’ attention, but on the flow of the lecture. In addition, each man’s concentration and motivation as well as students’ naturally fluctuate throughout the day. Accordingly, students with only one subject scheduled for attendance on a specific day, are more concentrated and motivated then students who have two or more subjects scheduled to attend the same day. This is common in the teaching practice. For example, one group of students have to attend only one course class scheduled at 10 30 am. Another group of students have another course class scheduled at 8 30 am and a second one scheduled at 10 30 to attend. This will largely affect the students’ perception of whether the HE teacher is able to hold their attention until the end of the class. It is more probable that the students who have to attend only one course that day will be more interested in the teaching. They will have the perception that the HE teacher manages to hold their attention until the end of the class. The other group of students who have to attend two courses that day may not have the same perception. The latter will be already tired, their concentration decreased, and therefore, their perception will be that the HE teacher has not managed to hold their attention.

Let us assume that a HE teacher has to consecutively teach two courses a day, and the courses to be taught are of the 3+2 design (3 hours of lecturing + 2 practice). If we count only the teaching hours, then the HE teacher has to teach 6 hours per day, and with the practice hours added, the figure of consecutive hours per day reaches 10!

Taking into account all the psychological aspects of the period when the class takes place and the complexity aspect of the schedule design, it is questionable whether HE teachers can objectively manage to hold students’ attention throughout the whole class. The HE teachers are only humans and they can also become exhausted. Thus, the time when the class takes place is an immensely significant factor which affects the survey’s results. This is in correlation with the research by Otany et al., (2012), Beran & Violato (2009) and the stances of the surveyed HE teachers.

Statement 3: The HE teacher encourages the students to be active and work independently, and encourages them to pose questions

Taking into account the formative nature of these activities, the help that HE teachers offer to students in order to facilitate and improve their learning is very important. It is worth pointing out that this statement is among the most significant components of HE teachers’ teaching. This has also been confirmed in the research by Kelly (2012), Beran & Violato (2009), and the opinions of the surveyed HE teachers. Regardless of students’ previous knowledge, every student can tell whether the HE teacher poses questions related to the topic being taught and whether the HE teacher encourages them to work independently and pose corresponding questions of their own. Accordingly, each one of them can eventually give an overall evaluation of the HE teacher’s work in that respect. If the student is honest and if the HE teacher really practices these activities in class, it is most certain that the student will circle the highest alternative within the scale. Admittedly, the balanced teaching is of great importance, i.e. the amount of the academic students’ learning time should be well-planned and balancing both HE teachers’ teaching and practice activities. Our conclusions in this vein are in accordance with those obtained in the research by Kelly (2012), Otany et al. (2012), and the surveyed HE teachers’ stances.

Statement 4: The HE teacher gives lectures regularly

According to the HE Law and the Labour Relations Law, HE teachers have a definite number of working hours, though it is fair to admit that every HE institution allows some flexibility, without violating the Law. If the HE teacher is absent from work because of exclusively private reasons, he is obliged to inform the dean of the institution and to request a legitimate leave, or provide appropriate documents as reasons for the same before taking a leave. When it is the latter, it is for the dean to approve the HE teacher’s request. If the teacher intends to attend a conference, seminar, workshop, etc, he must submit a written request with enclosed elaboration of the necessity to attend the specific academic event, i.e. justification for his absence from work. Moreover, he has either to inform the dean that he/she has completed the scheduled lecturing classes in advance, or to provide the dean with a new schedule with specified dates for catch-up classes i.e. for the classes that will be missed at the time of the approved leave. If the HE teacher does not come to work for more than three days without any announcement, the risk of HE teacher’s employment being terminated is at its extreme.

The HE teaching profession, though specific regarding the working hours, is a profession like any other profession. Thus, this statement on the consecutive regularity of the classes is not a relevant aspect of HE teachers’ teaching. Even more, all of the surveyed HE teachers agree with this finding. It is a fact that HE teachers have a weekly schedule for their lectures as well as for the other obligations they are assigned with by the institution. Some of these assignments are faculty meetings, Education and Science Councils, participation in various commissions, committees, and boards, conducting research and writing scientific papers, giving consulting hours, preparing and conducting exams for the students, keeping records of and making reports on their achievements, etc.

On the other hand, HE teachers have the legally approved so-called academic freedom to change the already established weekly schedule in accordance with some of their needs, either of a scientist – researcher and lecturer, or of a human. Certainly, this freedom does not mean interrupting or affecting the teaching process.

In line with the above said, the consecutive regularity of teaching, which is not an index of HE teachers’ quality performance i.e. teaching, might have a serious impact on the survey’s results. A student who is negligent or does not attend the classes regularly unlike other highly responsible students, might, for example, not know about the schedule changes. If this occurs several times during the academic year, this type of students might get a wrong impression that the teacher’s classes lack consecutive regularity. In addition, our finding of a firm connection between students’ regular attendance of classes and the survey’s results about consecutive regularity of teacher’s classes is confirmed in the research by Beran & Violato (2009) and the surveyed HE teachers’ stances.

Statement 5: The HE teacher is open and available for consultations and cooperation

The consulting hours for students are a very relevant segment of the teaching practice and indirectly connected to effectiveness teaching. It is assumed that during these conversations the teacher explains, clarifies, poses questions, gives and gains feedback, encourages, etc. Thus, during the consultation hour, the HE teacher spends an effective time in various activities with the student. This finding of ours is supported with the surveyed HE teachers’ opinions.

However, depending on the type of these consulting hours and on student’s personal perception, this statement might also affect the survey’s results validity. It is normal for HE teachers to spend time with their students in consultative conversations because such types of talks are part of their activities. The observation, browsing, and analysis of all university units’ web sites have shown that every teacher has pro-vided a consulting-hours schedule. There are HE teachers who have even provided an announcement on the faculty web page that they are available 24/7. Both of these approaches have strengths and weaknesses.

All students have various needs and the type of the consultation is different for every student. Thus, a question arises as to whether the HE teacher can manage and balance these various students’ needs and whether the consulting meetings are effective at all. Let us assume that several students want to consult the HE teacher, without previous announcement. In this case, the consulting meeting leads to ineffective time use both for the teacher and the unannounced students. The HE teacher needs to explain that he has no time because of other activities or an already scheduled meeting at that specific time. Students who have come to the faculty unannounced for a consulting hour only to ask the HE teacher for a certain type of consultation and to hear that the HE teacher has no time for them at all will certainly feel deceived. On the other side, if these unannounced students need different types of consultation, it will be impossible for the HE teacher to meet their needs. It has already been clearly noted in the reviewed literature there may be a group feedback only if the demands of all students in the group are the same. The consulting implies HE teacher’s attention for the student who needs help in the learning process. Thereby, consulting meeting will prove effective only if the student announces his visit to the HE teacher’s office i.e. gives the HE teacher sufficient information and time necessary for a design of a well-balanced and effective consultation.

When speaking about the time, both the HE teachers’ time and the students’ time should be respected. In the teaching practice, various situations occur. Although one of the requirements to be received for consultations is preparation with clear questions on the topic, rarely does a student come to the consulting hour prepared at all. Appearing before the HE teacher with various meaningful and logical questions connected to the topic that he has been taught is a feedback from the student of great importance for the HE teacher. In return, the student is also satisfied with the time teacher spends with him in high quality clarification. This is an opinion expressed and shared by all of the surveyed HE teachers. No matter how emphatic HE teachers may be with a student’s choice to come to consultation when he has time, the consultation are rarely what they should be because the student either comes to complain that the course is very difficult for him, or to ask about the content of the forthcoming final exam, or to express his worries that if he gets lower grades, he will lose his scholarship, etc. This is not an effective consulting at all. If the HE teacher decides that the student’s complaints stated above are not the cornerstone of the consulting, he risks to be graded lower by these students. What students forget is that they need to be very self-critical and aware of the relevance of the questions they pose to the HE teacher at a consulting meeting and whether they have gained the deserved answer in return. If there is no self-criticism, the student might give the HE teacher lower grade than deserved, and this will lead to lower/distorted survey’s results. Our finding has found confirmation in all of the surveyed HE teachers’ answers.

Statement 6: The HE teacher’s personal culture and attitude are appropriate

This statement implies HE teacher’s personal culture and attitude to students during his work day. It is of great relevance for the teacher to have personal culture and respectable attitude, but this is not a component of effective teaching. This is in accordance with all of the examined HE teachers’ opinions. The teacher’s personal culture and his behaviour are features of his character as a person. In the teaching process, there should be mutual respect. The teacher is not allowed to offend or humiliate the student. His correct attitude is of great importance. Thus, it is difficult to define this correctness because among other things, it depends on students’ expectations and perception of what personal culture represents is. This can also affect the surveys’ results.

Let us assume that the teacher reacts to students’ fidgeting, speaking and whispering when he is teaching, reproaching the student and stressing out the importance of the lecture as well as his own efforts to present it. In addition, let us assume that the tone of the reproach is academic, i.e. the HE teacher chooses words without offense and disrespect. This situation might be perceived from some students differently and they can circle the lower alternative, just to punish the teacher. This can also affect the survey’s results regarding this statement.

Statement 7: The HE teacher assesses student’s performance objectively

The assessment of students’ achievements is directly connected with the teaching, because the teacher has to assess those whom he teaches. The objectivity of the assessment is an imperative for every teacher and every higher education institution. The last stage of the teaching is the grading. The survey’s result might change depending on students’ perception of what a good assessment consists of. Different HE teachers assess students’ achievements in different ways because students’ activities are different. The Law anticipates that students’ achievements should be graded by taking into account all students’ activities. This means that students can achieve 100 points most. The grade is formed according to a scale from 51 to 100 points. Various factors which are not directly connected with the teaching affect the survey’s results:

Test design (Preparation of the test)

In order for the assessment to be objective, the HE teacher has to prepare an objective test to evaluate students’ achievements. The test should comprise of various questions that are measurable and objective, i.e. they should have only one correct answer which will show students’ knowledge. There are different types of objective questions. Regarding the assessment, they should be assessed in the same way, i.e. the number of requirements in the task determines the number of the points. However, this test can have only certain degree of validity, because it is not standardized. Accordingly, it is not strictly objective, and this means that the test does not consider the psychometric characteristics. Furthermore, the test prepared in this way measures only the knowledge that refers to "the three levels of the Bloom’s taxonomy, and neglects the assessment of the higher thinking processes, such as analysis, synthesis and evaluation" (TALEVSKI; JANUSHEVA, 2015). Thus, it is very hard to design a test for assessing students’ achievements that will comprise all of the Bloom’s taxonomy levels. The preparation of the test, among others, depends on the teacher’s capability and skills, his knowledge about the suggestions present in the experts’ literature on test preparation, etc. It depends on features that are not directly connected with the effective teaching. Yet, these features also affect the objectivity and validity of students’ grade. In addition, this will affect the survey’s results, too. This conclusion is confirmed with all of the surveyed HE teachers’ opinions.

Criteria

In order for the assessment to be objective, every teacher must introduce the students with the assessment criteria. Introducing students with the assessment criteria does not reduce the influence that this factor, though not directly connected with the teaching, will have on the survey’s results.

Let us assume that the student achieves 70 points and according to the scale, this equals the grade 7. The teaching practice shows various cases where the student is not satisfied and claims that he has needed one point only to be graded with 8. The HE teacher who does not diverge from the prescribed scale is, thereby, perceived not as objective, but as bad or cruel. Thus, there is a big probability that such HE teacher will be graded lower by the unsatisfied student. The HE teacher who does diverge from the preset scale is perceived as great, and most probably will be graded higher. We find confirmation of our finding in the research by Kelly (2012), and Beran & Violato (2009), and a positive correlation between the grade that the teacher has given to the student and the grade he gets from the student in return. This has also been confirmed by the stances of the surveyed HE teachers.

On the other side, the teacher who decides to diverge from the scale for one student only should do the same for all students because every student deserves equal treatment. Thus, he needs to make a correction of the points for all of the students who need only one point to get a higher grade. This will put in question the grade 5 of students who have gained, for example, 49 or 50 points. According to this divergence from the preset scale, these students have also passed the exam. However, not all students are equally treated regarding the points which guarantee them a higher grade. There is no rule of how the HE teacher should approach this issue. Nevertheless, the teacher will make the decision according to student’s overall activities and the general picture he/she has created about the student. This shows that another factor that is not directly connected with the teaching affects the survey’s results.

The activities that the HE teacher requires from the student

Some of the HE teachers assess students’ achievement summative on the grounds of the results from the two-midterm exams, without conducting an oral part of the exam. Some HE teachers assess more activities, such as oral answers, project activities, homework assignments, etc. Depending on the number of the activities the teacher has asked the student to do in order to form the summative grade, the survey’s results might change. The teacher who asks for fewer activities is graded higher than the teacher who demands more activities. This is in accordance with the research of Beran & Violato (2009) as well as with all of the surveyed HE teachers’ answers. Furthermore, if the teacher taught the student the previous year, there is a possibility for the student to circle the lower alternative on the scale. This is because the teacher did not assess him as he wanted. There is also a possibility for the student to circle the higher alternative especially if he knows that the teacher will teach him again in the next semester or academic year. This is in accordance with the research of Beran and Violato (2009) and Kelly (2012) regarding the positive correlation between the grade the teacher gives to the student and the grade the students give to the teacher.

Statement 8: There is an appropriate basic and additional literature for the course taught

This statement does not depend on HE teachers’ competences and it is not informative about the teaching quality. This finding is confirmed by all of the surveyed HE teachers. There is appropriate basic and additional literature for many course taught this information is listed both in the teaching curricula and in the posts on the faculties’ web sites. On the one hand, students react if the handbook is over 200 pages. On the other hand, there should be a balance between the material that can be managed for one semester and the offered literature. Moreover, with the Government’s project many foreign authors’ books on various fields have been translated and it is a fact that there is appropriate and modern literature.

However, there are courses taught such as Modern Macedonian Language, which are based on older but immensely relevant and significant literature, for example, the Macedonian Grammar by B. Koneski (the first edition of this book is from 1967). This fact can be abused by saying that the teacher uses old literature and, thus, affect the survey’s results. On the Internet one can find many scientific papers that elaborate a good number of problems in the modern Macedonian language. If the teacher directs the students toward all those sources, he/she risks to be graded lower. For many courses taught at the university there is no literature in the Macedonian language, though there are books in Serbian, but unfortunately, the younger generations are not familiar with this language. As a result, the HE teacher has to translate, or make some shorter version of his lectures, maybe even dictate, which will certainly reflect badly on the survey’s results. Further, there is literature in English language, but offering it as additional literature puts the teacher in risk to be graded lower.

Statement 9: Level of difficulty of the assessed course: highest, high, moderate, low and lowest

This statement asks the students to assess how difficult the course is. In the teaching practice, students speak about difficult, moderate or easy courses, but this degree depends on many factors, such as the nature of the material, the students’ previous knowledge, their interest, the grade they have got from the teacher, the number of the course attendants within a group, etc. All these factors have an additional impact on the survey’s results. This statement is not vital for the teaching quality assessment because how difficult a course is in fact a personal student’s perception. This finding is confirmed by all of the surveyed HE teachers answers.

Statement 10: The HE teachers’ requirements (exams, mid-terms, seminar papers, and project activities) are very strict, strict, moderate, or loose

This statement is tightly connected with the statement which refers to the assessment’s objectivity. As already mentioned, the number of requirements set by the HE teacher affects the survey’s results.

Statement 11: The specified course lectures, in scope and quality, were a solid base for passing the exam: yes, partly, no

This statement refers only to HE teachers’ oral presentation in class, and not to all of the activities he has included in the class. HE teachers’ teaching is always a base for passing the exam in a sense that the content he presents is always related to the requirements he sets for the students. In addition, one can pose the question: Why would a teacher teach about a concept if he does not plan to include it in the exam? Therefore, this statement, though relevant as a principle that the teacher will respect when assessing what he has taught about, is not of corresponding influence on the teaching quality assessment. This is confirmed by all the surveyed HE teachers’ stances.

The statement can only be accepted as students’ evaluation of the relevance of the content that the teacher has taught for the questions posed in the exam. This is directly connected with the assessment and the teaching practice, which shows that, unfortunately, there are HE teachers who ask question that have not been taught throughout the semester. This statement can also affect the survey results and it is connected with students’ regular attendance of classes. If the student attends the classes regularly, he will take down notes about concepts that are not mentioned in the textbook, but are of great importance. The student who does not attend the classes regularly will not know about this and he/she might get an ungrounded impression that the material taught does not contain the exam questions. This finding is in accordance with all of the surveyed HE teachers’ opinions.

Statement 12: The content and structure of the exams questions provide objective assessment of the material taught: yes, partly, no

This statement is also connected with the statement 7 mentioned above and it shows that many factors related to the exam questions lead toward differences in the survey’s results.

Results from the students’ survey

Regarding the students’ opinion about the effects of this type of self-e-valuation, the results from the survey show the following: All of the sampled students have participated in the surveys which assess HE teachers’ teaching quality. None of the students is familiar with the results of the conducted survey. More importantly, in their further education, none of the students has noticed any effects from the conducted survey regarding the aspect they have not agreed with, or have agreed completely or partially. The most important thing is that none of the students takes this survey as a serious instrument by which the teaching process will be improved. Thus, a very relevant conclusion is drawn, which is confirmed in Kelly’s research (2012), and that is: if the students do not perceive the effects of the survey, they will never again have a serious approach towards it in the future.

Conclusion

The self-evaluation is without any doubt the most significant mechanism used by the HE institutions in order to provide information about their quality and effectiveness. A relevant segment of the self-evaluation is the evaluation of the HE teachers’ work. The HE teachers’ work at the University “St. Kliment Ohridski” – Bitola, Republic of Macedonia, and worldwide is assessed with a survey filled in by the students. Although this survey offers information about the HE teachers’ performance quality, and has its advantages, it also has disadvantages and there is a need of a much serious approach to its design.

As it can be seen from the research, various factors which are beyond HE teachers’ authority affect the survey results and their objectivity and validity. The re-search results have identified several aspects advantageous and valid for the teaching quality assessment such as HE teachers’ clear and understandable explanation, ability to evoke interest with their teaching, ability to spur students’ activity and independent work, and to encourage students to pose questions.

It is of great importance to design an appropriate survey which will stress those statements that genuinely relate to effective teaching and reflect the multidimensionality of the teaching process instead of the statements which are relevant part of the HE teachers’ work, but not truly referential and not in HE teachers’ authority.

In this sense, the stances about the relevant aspects of the teaching construct, in context of the Macedonian HE, obtained from all of the surveyed HE teachers and experts in this field should be taken rather seriously. They should represent the base for further research in order to detect more/other referential aspects. The statements that will represent these aspects should be formulated in a way which will enhance the survey results validity and objectivity. On the other hand, it is very important to give a meaningful formulation to the survey statements because the formulation itself can contribute to the clarity of the survey results. The research shows that various factors can affect the grade given to the HE teachers by their students: their previous knowledge, their perception of what good teaching consists of, the number of students attendants within a course group, the assignments HE teachers give to students, etc. These findings indicate that the largeness and the structure of the sample should also be considered. For instance, when selecting a sample, it is important to make a distinction between students who depend on the grade they get from the HE teacher, and students who regularly attend their classes if more valid and objective results are aimed at.

This type of self-evaluation is conducted at the end of the education process, i.e. at the end of the academic year. This makes the formative and corrective nature of the survey meaningless, because what has been important for the group that has assessed the teacher might not be important for the next group the teacher will teach in the following semester.

The ways in which the results are managed are also very significant. An average grade is often calculated for each respective statement, and then, an average grade is calculated from the accumulated grades. This way of calculating the grade has the weaknesses of the numerical grading. Moreover, what need to be addressed are the alerting results from the students’ survey regarding the effects of the survey on HE teachers’ work. It is natural for the students to expect to see some changes they have stood for throughout the survey. Their grade is, in fact, a feedback for the HE teachers. Therefore, if the HE teachers do not take into consideration their opinions, there will be no effect in doing the survey. These problematic aspects should reflect only those which are in HE teachers’ authority.

Accordingly, although the students’ survey for grading the HE teachers’ work is an important and relevant tool for obtaining significant results, still, many aspects should be taken into consideration for the survey to be created in a way which will produce more valid and objective results.

References

BERAN, T.; VIOLATO, C. Student Ratings of Teaching Effectiveness: Student Engagement and Course Characteristics. Canadian Journal of Higher Education. V. 39, no. 1, pp. 1-13, 2009. Available at http://bit.ly/2riM5Xo. Accessed: 16 March 2017.

BERAN, T. N.; ROKOSH, L. J. The Consequential Validity of Student Ratings: What do Instructors Really Think? The Alberta Journal of Educational Research. V. 55, no. 4, pp. 497-511, 2009. Available at http://bit.ly/2safPtH. Accessed: 16 March 2016.

HOYT, P. D.; PALLETT, H. W. Appraising Teaching Effectiveness: Beyond Student Ratings. Idea Center. Idea paper, no. 36, 1999. Available at http://bit.ly/2sPrzQn. Accessed: 16 March 2017.

KELLY, M. Student Evaluations of Teaching Effectiveness: Considerations for Ontario Universities. COU Academic Colleagues Discussion Paper. 2012. Available at http://bit.ly/2s-PMqTz. Accessed: 15 March 2017.

MARSH, W. H.; ROCHE, A. L. Making Students' Evaluations of Teaching Effectiveness Effective. The Critical Issues of Validity, Bias, and Utility. American Psychologist. V. 52, no. 11, pp. 1187-1197, 1997. Available at http://bit.ly/2sakJ9X. Accessed: 16 March 2017.

OTANY, K.; JOON K. B.; CHO, J. I. Student Evaluation of Teaching (SET) in Higher Education: How to Use SET More Effectively and Efficiently in Public Affairs Education. Journal of Public Affairs Education. V. 18, no. 3, pp. 531-544, 2012. Available at http://bit.ly/2t8hW-vs. Accessed: 15 March 2017.

PARPALA, A.; LINDBLOM-YLÄNNE, S.; RYTKÖNEN, H. Students’ conceptions of good teaching in three different disciplines. Assessment & Evaluation in Higher Education. V. 36, i. 5, 2011. Available at http://bit.ly/2rO7gUO. Accessed: 16 March 2017.

SELDIN, P. How Colleges Evaluate Professors, 1983 vs. 1993. POD Network Conference Materials. Paper 110, 1993. Available at: http://bit.ly/2saliR4 16 March 2016

TALEVSKI, D. J.; JANUSHEVA, V. Aspekti na ocenuvanjeto [Aspects of assessment]. 1st.ed. Bitola: “Sv. Kliment Ohridski” University, Faculty of Education, 2015.

Tolkoven rechnik na makedonskiot jazik. [Monolingual dictionary of the Macedonian language]. Skopje: Institut za makedonski jazik “K. Misirkov”, 2011.

Zakon za visokoto obrazovanie. Konsolidiran tekst. Sluzhben vesnik na Republika Make-donija “ [The Law for higher education. Consolidated text. Official Journal of the Republic of Macedonia]. Available at https://bit.ly/2qX809w. Accessed: 15 March 2017.

Notes