Introduction

In fall 2011, there were approximately 2 million employees working at U.S. four-year universities whose primary functions were the performance of duties other than teaching (Knapp, Kelly-Reid & Ginder, 2012). Prior research has documented the importance of university nonfaculty employees and their interactions with students in fulfilling the academic missions of American post-secondary institutions. Kinzie and Kuh (2004), for example, identified the significance of a university-wide culture of caring for student success in their two-year project, Documenting Effective Educational Practice (DEEP). DEEP institutions performed better than predicted in graduation rates and on the National Survey of Student Engagement (Kinzie & Kuh). One characteristic identified across universities that performed better than predicted in graduation rates and student engagement was that all community members understood their role and contribution to the institution’s character (Kezar, 2007). Further, Schreiner, Noel, Anderson and Cantwell (2011) identified personality traits of faculty and staff that high-risk college students found beneficial in the establishment of a caring college environment. These characteristics included a deep commitment to helping students and being sincere in interactions with students (Schreiner et al., 2011).

Despite this documented importance to post-secondary education and a growing literature on this population, there is still much to be discovered about non-academic employees at U.S. universities. Szekeres (2004) initially noted that university administrative employees and their roles were unnoticed at Australian post-secondary institutions. In a later paper, Szekeres (2011) noted that, while there had been substantial growth in the literature on non-academic university employees, many of these staff still felt their worth in these institutions was not recognized. In describing the current roles of non-academic employees, Gordon and Whitchurch (2007) noted the growing need for professional employees who had the same academic credentials as faculty, and could perform work that was a blend of administrative and academic. Because of the diversity of positions within post-secondary institutions, Graham (2010) questioned whether it was time to be more precise when describing the roles of non-academic employees.

The current quantitative case study explores how supervisors at one public university selected competencies to measure non-faculty employee performance. This public post-secondary institution was chosen because it was representative of a large, urban, public research university and for convenience, as performance data for all employees at this institution were readily available. Also, as this institution represents a particular type of higher education setting, it was hoped that insights from this study could be useful to provide research-based recommendations for similarly situated post-secondary institutions.

Further, higher education is a unique workplace setting, different from the corporate sector. Universities have been described as loosely coupled organizations, dissimilar to the more tightlybound, hierarchical corporate setting (Birnbaum, 1998). Thus, non-faculty employees working in higher education settings represent a unique population. The research question that guided this study was: Within a university setting, how are employee competencies valued by job title within colleges and divisions?

This study presents the argument that non-faculty employees are an important component of the success of higher education institutions and thus it is critical that all management tools, including performance appraisal, should be used to communicate the roles, desired behaviors, and institutional values to these non-faculty employees. It is hypothesized that, in its current form, the use of competency selection in the performance appraisal is not valued as an identity regulation tool, but that with minor improvements this tool could be an important management asset in this workplace setting. Making the adjustments recommended from the analysis in this study will ensure that non-faculty employees are prepared to serve and support their institution in the important missions of educating and graduating students.

In this study, performance appraisal was defined as the workplace process conducted annually or semi-annually, during which employees are rated on their perceived strengths and failings by their supervisor (Cleveland, Murphy, & Williams, 1989; Kondrasuk, 2011; Moussavi & Ashbaugh, 1995). Performance appraisals are used to make determinations about employee: salary adjustments, promotion, and training needs (Cleveland et al., 1989; Kondrasuk, 2011; Moussavi & Ashbaugh, 1995). Kondrasuk (2011) pointed out that using the performance appraisal to make administrative decisions such as pay and promotion is at odds with using the performance appraisal for employee development.

Competencies have been defined in the research literature as core abilities, traits, or characteristics (Boyatzis, 2008, 2009; Draganidis & Mantzas, 2006; Grote, 1996; Kochanski, 1997; Woodruffe, 1993). In the workplace, they can be used to predict how an employee will respond in a particular situation (Grote, 1996). They are observed in employee behaviors, and an employee’s intent or motivation can be observed through how the employee enacts the competency in various workplace situations (Boyatzis, 2008, 2009; Gangani, McLean & Braden, 2006).

Study Guiding Ideas

There were two guiding ideas that provided the framework for this study. The first was the notion of employee identity regulation in which employers use techniques such as reward and promotion systems, training, and induction to shape an employee’s workplace identity (Alvesson & Wilmott, 2002; Swann, Johnson & Bosson, 2009). These practices cause employees to question their identity in the workplace and provide information to employees about what is valued in the organization (Alvesson & Wilmott, 2002). One such technique is the employee performance appraisal.

The second guiding idea for this study focused on employer-driven identity regulation techniques and their transition from the corporate culture into post-secondary education. This is an example of neoliberalism in higher education as described by Giroux (2002). According to researchers, corporate tools, such as zero-based budgeting and Total Quality Management, have influenced higher education, even though they are not used widely today (Birnbaum, 2000). Birnbaum (2000) referred to these corporate tools that were implemented in higher education and then later discarded as management fads. Researchers (e.g., Birnbaum, 2000) have expressed concern about the wholesale use of corporate tools in higher education settings without understanding their long-term impacts. As an example, Szekeres (2006) noted the pervasive use of technology in the higher education workplace as a way to increase staff efficiency. Challenges that university staff perceive with these efficiencies are that the staff workload may be less visible to supervisors and that these technological enhancements may decrease the amount of human interaction students receive (Szekeres, 2006). This study afforded a unique opportunity to understand how one institution implemented the corporate practice of competency usage in its employee performance appraisal. The following sections describe the guiding ideas for this study in more detail.

Processes used by employers to regulate employee identity. Researchers have noted that employers use various methods for implicitly and explicitly regulating employee identity development or how employees understand their role within the organization (Swann et al., 2009). In employee identity development processes, the employer acts as a perceiver and guides the employee to enact specific behaviors in the workplace (Swann et al. , 2009). Likewise, the employee is the target whose goal in this process is to receive confirmation of identity from the perceiver (Swann et al., 2009). The purpose of the following sub-sections is to situate the performance appraisal as one such system used by employers to standardize employee behaviors.

The employee performance appraisal as an employee identity regulation tool. One management system that encompasses many of the employee identity regulation processes used by employers to provide role definition to employees is the performance appraisal. Employee appraisals are used to make judgments about employees, such as salary increases, promotions, and training needs (Cleveland et al., 1989; Kondrasuk, 2011; Moussavi & Ashbaugh, 1995). The performance appraisal fits the mold of an employee identity regulation tool, as supervisors regulate employee behavior in many ways during this process. Supervisor scores on the appraisal direct employees on how to improve or regulate performance (Cleveland et al., 1989; Kondrasuk, 2011; Moussavi & Ashbaugh, 1995; Spence & Keeping, 2011). Similarly, pay increase and promotion decisions are often made using performance appraisals (Cleveland et al. , 1989; Kondrasuk, 2011; Moussavi & Ashbaugh, 1995; Spence & Keeping, 2011). Also, the assignment of training to those employees who need it through performance appraisal is another way supervisors cultivate desired employee behaviors (Kondrasuk, 2011; Moussavi & Ashbaugh, 1995; Spence & Keeping, 2011).

Performance appraisals can measure whether an employee has achieved a specific accomplishment or can be used in a more subjective way, such as evaluating employee competencies or traits (Kondrasuk, 2011). Competency-based performance evaluation is the process in which a supervisor evaluates an employee’s skill accomplishments and deficiencies in relation to the abilities deemed necessary for a specific occupation (Scott & Einstein, 2001). The assessment of employee performance using competencies has been labeled as a critical approach to the performance appraisal in which the supervisor evaluates employees on their distinct attributes or traits (Lunenberg, 2012). The use of competencies is also a method for the employer to define aptitudes that will be important for success in the workplace (Pulakos, 2004). The competency assessment process compels the manager to conjecture about employee abilities and aptitudes by examining employee workplace performance (Kline & Sulsky, 2009).

The performance appraisal as a corporate tool in a higher education setting. Some researchers (e.g., Giroux, 2002) would likely argue that the use of a corporate tool such as the performance appraisal, in general, and in particular the measurement of employee competencies, provides another example of how the corporate culture has invaded higher education. Szekeres (2004) noted that the infusion of corporate tools, specifically Total Quality Management, in the university stems from a perceived notion that they can help the university be more accountable. To address a focus on accountability, universities have established priorities where measurable outcomes have become institutional priorities and corporate language such as efficiency and marketing have become part of everyday university vernacular (Szekeres, 2006).

Corporate tools appear at face value to be logical as information produced from their use is presented quantitatively (Birnbaum, 2000). One challenge becomes that quantitative data turn out to be more valued than those outcomes that are difficult to measure (Birnbaum, 2000). For some, the introduction of corporate instruments and terminology into higher education reflect an inappropriate deference to corporate culture that must be resisted (Giroux, 2002). This is especially concerning as one recognizes that the performance appraisal is used to regulate employee identity. The use of competency measurement in the performance appraisal is a way to draw attention to the types of behaviors valued in the workplace (Pulakos, 2004).

Literature Review

In this section, the following literature will be described and discussed: (a) an overview of how corporate tools have been adopted in higher education settings; (b) an overview of the employee performance appraisal and its uses; (c) an overview of competency use in the employee performance appraisal process, including how competencies have been used in higher education settings; and (d) gaps in the literature that were addressed by this study.

Overview of the Use of Corporate Tools in Higher Education

This study examined how the corporate tool of the performance appraisal, and specifically the use of competencies within the performance appraisal, were implemented at one post-secondary institution. Giroux (2002; 2013) expressed concern that the infusion of the corporate tools and terminology into higher education moves higher education further away from its mission of a democratic public good. According to Birnbaum (2000), it is even more concerning that certain management processes developed for other environments are adopted wholesale by higher education institutions with the promise of resolving a management challenge. Conversely, Ewell (1999) argued that the implementation of a process or term from another sector into higher education forces people and departments to constructively examine current processes and practices. For example, Ewell) noted that even a conversation about students as higher education customers may prove fruitful as it may evolve into a discourse about faculty and student roles in learning and how it actually happens.

According to Birnbaum (2000), a problem with the introduction of management tools from other settings is that they are often implemented without having a complete understanding of their potential long-term organizational impact, even if these tools are later abandoned. Some management tools that have been implemented and then phased out of popular use in higher education have included zero-based budgeting in the 1970s and 1980s and business process reengineering in the 1990s (Birnbaum, 2000; Lamal, 2001). Similarly, Szekeres’ (2006) study of Australian employees found that the infusion of a market-driven culture and treatment of students as customers significantly increased staff workload.

Overview of the Employee Performance Appraisal and its Uses

Researchers point to two specific ways that employee performance is rated by a reviewer (typically a supervisor): behavior- or trait-based formats (Kline & Sulsky, 2009). In a behavior-based format, the rater makes a determination about the frequency an employee exhibits a specific behavior or how the employee has met a particular behavioral standard in performing the task or duty (Kline & Sulsky, 2009). In a trait-based format, the rater is observing employee behaviors and determining if the way an employee exhibits the behavior is indicative of a particular trait (Kline & Sulsky, 2009; Scott & Einstein, 2001). These two different ways of measuring employees are rarely supported by a single system (Pulakos, 2004). The following sub-sections will discuss the differences between behavior-based systems and competency systems in measuring employee performance.

Behavior-based employee performance measurement systems. Behavior-based employee performance appraisal formats are used to rate the frequency with which an employee is observed exhibiting a specific behavior or to measure the quality of the outcome of a particular goal or task (Kline & Sulsky, 2009). Typically, these systems are quantitative and objective (Kondrasuk, 2011). Common instrument forms used for these types of appraisals are the Behavioral Observation Scale (BOS) and the Behaviorally-Anchored Rating Scale (BARS) (Kline & Sulsky, 2009). The BOS format is used by raters to document the frequency with which an employee is observed completing a behavior (Kline & Sulsky, 2009). The BARS provides scale points and behavioral illustrations for each scale-point to delineate expected performance (Kline & Sulsky, 2009). When rating an employee on the BARS, the rater is expected to choose the behavioral example that best matches the employee’s performance (Kline & Sulsky, 2009).

To understand the ease of use of different performance instruments in the workplace, Wiersma and Latham (1986) conducted a study that compared the use of BARS, BOS, and a traitbased performance appraisal. They found that employees and supervisors at a financial institution as well as attorneys who specialize in employment litigation preferred the BOS instrument (Wiersma & Latham, 1986). Attorneys in this study believed the BOS was more defendable than the other performance appraisal forms (Wiersma & Latham, 1986). One year after implementing a BOS appraisal format, senior management reported use of the BOS reduced personality disputes over appraisals, provided a way to defend lower employee ratings, and enhanced feedback employees received from their supervisors on the appraisal (Wiersma & Latham, 1986).

Competency-based employee performance measurement systems. Competencies selected for performance management may be used to measure all employees or a specific job family within an organization (Grote, 1996). One way of developing competencies to be used in employee performance appraisals is when managers develop examples of model performance, and those that emerge out of this process are adopted for use (Kline & Sulsky, 2009). Another approach used is competency modelling, or when competencies that generalize across jobs are used and specific behaviors that exhibit these competencies are identified and described (Kline & Sulsky, 2009). For example, in a case study describing how a competency management system was implemented at one corporation, Gangani et al. (2006) described how the firm used job analysis interviews, focus groups, and current job descriptions to develop competencies. This firm also assigned higher weightings to competencies that were viewed as more important to the firm (Gangani et al., 2006).

An Overview of Competency Measurement in the Performance Appraisal

In this section, the following discussion of competencies is presented: (a) definition of competencies and competency management; (b) history and evolution of competency use; (c) competency usage in higher education; and (d) study findings on competency usage for non-faculty employees in higher education settings.

Definition of competencies and competency management. Competencies in the context of employee performance appraisal have been defined by researchers as an underlying ability, trait or characteristic organized around a specific concept (Boyatzis, 2008; Boyatzis, 2009; Draganidis & Mantzas, 2006; Gangani et al., 2006; Grote, 1996; Kochanski, 1997; Woodruffe, 1993). Grote (1996) noted competencies are similar to an individual’s personality and can be used to predict how an employee will respond in a particular situation. Competency measurement is observed through monitoring employee behaviors (Boyatzis, 2008; Boyatzis, 2009; Gangani, et al., 2006). Some researchers have noted that intent or motivation can be observed through how the employee enacts the competency in various workplace situations (Boyatzis, 2008; Boyatzis, 2009; Gangani, et al., 2006). According to Lievens, Sanchez, and De Corte (2004), competency systems are used by employers to: (a) identify potential candidates for positions; (b) promote employees; (c) conduct performance appraisals; (d) make compensation decisions; and (e) to determine employee development needs.

Similarly, competency management has been described as the formalized process of identifying competencies or traits that are used to distinguish outstanding from average employees (Kochanski, 1997; Martone, 2003). This identification of competencies results in competency models, or developing competency groupings, that relate to success in a specific position within the organization, and is connected to overarching organizational goals (Draganidis & Mantzas, 2006; Lievens et al., 2004). A competency model is composed of a group of competencies where behavioral examples are provided to illustrate excellent and inferior competency performance (Draganidis & Mantzas, 2006). A concern mentioned by Grote (1996) and Pulakos (2004) is that employers should not use too many competencies in the development of a competency model. Grote (1996) warned that, when too many competencies are used, the importance of any one competency might be devalued.

Researchers and practitioners noted that one of the most critical components of the competency development process is the development of behavioral descriptions that will be used to evaluate employees on each competency (Campion, et al., 2011; Grote, 1996; Martone, 2003; Pulakos, 2004; Woodruffe, 1993). Woodruffe (1993) further added that the wording describing the behavior associated with each competency should be familiar to employees in the organization and easy for all to understand. Providing behavioral descriptions of each competency also helps employees understand expectations and provides a standard that managers can use when evaluating employees (Pulakos, 2004).

History and evolution of competency use. The use of competencies as a way to measure performance has been credited to McClelland (1973), who stated that employee skill level is best observed in the environment. McClelland (1973) noted if one wanted to measure the skills and abilities of an outstanding employee in a particular field, it was important to observe this employee and make a list of the employee’s qualities or characteristics to understand the competencies needed to be an exceptional performer. The practice of determining competencies of outstanding performers later evolved into the behavioral event interview (Boyatzis, 2009).

The behavioral event interview process included using a nomination process, in which top performers and average performers were selected (Boyatzis, 2009). This process involved asking both groups to describe a recent event in which the employee felt successful and to provide narrative details about performance (Boyatzis, 2009). McClelland (1998) described the behavioral event interview differently from Boyatzis’ (2009) process. McClelland would ask each employee to describe six different episodes, three in which the employee felt successful and three in which the employee felt unsuccessful. These interviews were then coded to determine competencies for outstanding employees (McClelland, 1998).

When developing competency models, Shippmann et al. (2000) noted concerns about the broad view applied in the behavioral descriptions for competencies, as not all descriptors account for the distinctiveness across positions in an organization. Competencies used to measure individual employee performance should be connected to the organizational mission and provide a common organizational vernacular about employee expectations (Lievens et al., 2004). As noted by Campion et al. (2011), competencies should be desired employee traits and skills that help the workplace accomplish its mission. Pulakos (2004) recommended that, when using them to measure performance, between 5 to 10 competencies should be selected at the organizational level and that they be connected to organizational strategic initiatives and performance success in a particular job. Pulakos (2004) further noted appraisals are sometimes constructed with a laundry list of competencies, which can complicate the rating process and may not be helpful to supervisors who have many employees. Studies have also suggested that competencies can be used to differentiate high-and low-performing employees (Dreyfus, 2007; Hopkins & Bilimora, 2008; Scudder & Guinan, 1989).

Competency usage in higher education. Several higher education professional associations have actively discussed the use of competencies for non-faculty employees. For example, the Association of College and University Business Officers (ACUBO), which represents the five coalitions of business officer organizations[1], completed the Competency Survey in 2005 (Jurow, 2006). The research team pre-selected competencies for institutional chief financial officers, employees who work in the university financial or business offices, and employees who have financial responsibilities and/or work in academic or other offices at a university (Jurow, 2006).

Jurow (2006) noted that these pre-selected competencies were arranged into four groups: “core, technical, organizational, and leadership” (para. 8). Core competencies were those skills required by all employees who work in the business sections of a higher education institution, and technical competencies included skills such as having knowledge of accounting terminology and being able to prepare spreadsheets (Jurow, 2006). Competencies related to managing performance were categorized as organizational competencies (Jurow, 2006). Leadership competencies were those skills needed by business staff tasked with leading institutional initiatives (Jurow, 2006). The national surveys were completed in 2005 by more than 900 individuals, and the pre-selected competencies were validated in this process (Jurow, 2006).

Similarly, the College and University Professional Association for Human Resources (CUPA-HR, n.d.a.) developed a listing of core competencies for higher education human resource professionals. These core competencies are part of a larger CUPA-HR (n.d.a.) “Learning Framework” (para. 1) that was developed by the organization using a volunteer taskforce of higher education human resources professionals (CUPA-HR). According to CUPA-HR (n.d.b.) in summary, these core competencies include such skills as an: (a) awareness of the unique nature of higher education; (b) ability to manage risk; and (c) understanding the importance of human resources to institutional success.

Student affairs professional organizations have spent a significant effort developing professional competencies for employees in student affairs. The two largest professional organizations—American College Personnel Association (ACPA) and the National Association of Student Personnel Administrators (NASPA)—formed the Joint Task Force on Professional Competencies and Standards in 2009 with the purpose of developing competencies and associated behavioral descriptors of these competencies for professionals in student affairs (ACPA/NASPA, 2010). The intent of these competencies was to be useful to leadership in student affairs for the development of job descriptions and to help guide the learning outcomes of student affairs programs, as many entry-level student affairs positions require a master’s degree (ACPA/NASPA).

Within each competency, there are behaviors that are associated with basic, intermediate, and advanced levels of the competency (ACPA/NASPA, 2010). The expectation is that student affairs professionals will be able to meet basic standards for each competency area (ACPA/NASPA, 2010.). These competencies were evaluated and updated by the Professional Competencies Task Force and were approved in August 2015 by ACPA and NASPA (NASPA, n.d.). These 10 professional competencies include such skills as the importance of the student affairs profession to higher education and learning and development of students.

Across these discrete higher education professions, there are some similar competencies. A focus on ethics was a competency for both student affairs and chief financial officers. An understanding of budgeting and finance were similar competencies for professionals in human resources and university business officers. An understanding of human resource was a similar competency across all three professions. Likewise, the three professional organizations reported developing professional competencies for similar reasons, mainly so that individuals in these professions might be able to use these competency listings for individual growth and development (ACPA/NASPA, 2010; CUPA-HR, n.d; Jurow, 2006). A new application for the ACPA/NASPA (2015) competencies was that these competencies should be used to provide a framework for institutional performance evaluations.

Studies on the usage of competencies for non-faculty employees. Studies on the usage of employee competencies in post-secondary education have focused predominantly on the student affairs profession. Burkard, Cole, Ott and Stoflet (2005) found that supervisors in student affairs valued individual qualities, such as adaptability and communication, most in entry-level student affairs employees. The second most valued competencies identified in this study involved the entrylevel employee’s abilities to relate to and interact with others, such as skills of cooperation and mediation (Burkard et al., 2005). Likewise, Hoffman and Bresciani (2010) noted that entry-level student affairs position descriptions frequently contained references to competencies of advising, technological knowledge, and practical abilities while senior-level position descriptions referenced management and financial competencies. Differences in the importance of certain competencies for student affairs employees were observed between student affairs officers and the program faculty who prepare these professionals (Kuk, Cobb & Forrest, 2007). Program faculty believed that management skills and serving as change agents were less important than did student affairs officers and managers (Kuk et al., 2007).

Researchers have also identified competency skill shortcomings in recent graduates of student affairs graduate programs. According to mid- and senior-level student affairs officers, recent graduates of student affairs master’s programs were least prepared in the competency areas of budget and financial oversisght, which corresponded with graduates’ perceptions of their own deficiencies (Cuyjet, Longwell-Grice, & Molina, 2009). Waple (2006) noted similar findings. Student affairs professionals reported skills such as budget and finance and computer technology were used frequently in the first professional assignment, but training in these skills was a low priority in graduate school (Waple, 2006). This corresponded with findings from Dickerson et al. (2011), who noted that faculty in student affairs programs and senior student affairs officers observed new student affairs professionals were least prepared in graduate study in competencies and skills for financial oversight, legal rules, and evaluation.

Gaps in the Current Literature

As discussed, employee identity development practices, such as the performance appraisal, provide valuable opportunities to transmit information to employees about desired organizational values and principles. Similarly, supervisor competency selection further communicates to employees their role in an organization and can offer guidance to employees about valued workplace skills and behaviors (Pulakos, 2004). Thus, in large organizations, such as universities, the use of competencies in the performance appraisal can be a valuable asset to the organization, communicating to the employee his or her specific role and how to perform it.

Indeed, a web search revealed that many universities reference terms such as competencies or core competency measurement in their performance appraisal on their human resources web site. Likewise, several higher education professional organizations have actively developed competencies with the stated intent to help employees grow and develop. More recently, it has been proposed that student affairs professional competencies be used in the performance appraisal framework (ACPA/NASPA, 2015). Similarly, more than any other profession, there has been active student affairs literature focused on understanding the important competencies for student affairs professionals and even the specific competency needs for specific institution types.

It has also been asserted that non-faculty employees, while not directly teaching students, do play an important role in the success of students at post-secondary institutions (Kinzie & Kuh, 2004; Kezar, 2007). Thus, it is important to understand the management processes that are currently used to communicate valued institutional principles and behaviors to these employees with the intent of offering recommendations for improvement. Importantly, this quantitative case study sought to understand how one public post-secondary institution used the selection of competencies to regulate non-faculty employee identity development through its performance appraisal. This study offered empirically based recommendations that can be applied to this and similarly situated post-secondary institutions.

Method

The study method used in this case study is presented in the following sections: (a) the case study as an approach for context-specific research; (b) study setting and participants; (c) background and use of the performance appraisal; and (d) variables and statistical analyses used to answer the research question.

The Case Study Approach for Context-Specific Research

Case studies have traditionally been connected with an in-depth examination of many forms of data (Creswell, 2002; Yin, 1981). Yin (1981) noted that the method of analysis in a case study can be qualitative or quantitative and that what makes the case study research approach unique is that it is an in-depth analysis of a particular incident or phenomenon in context. Likewise, Eisenhardt (1989) noted that case study research may combine qualitative and quantitative data and can use multiple forms of analysis to understand the phenomenon.

The case study lens provided an appropriate framework for this research as it involved an assessment of one university’s implementation of competency evaluation in its non-faculty employee performance appraisal. This study described the institution’s current policy and intentions with its performance appraisal and analyzed how this system was utilized, using institutional non-faculty employee performance and demographic data from calendar year 2012. The intent of this study was to understand how supervisors were using competencies as an employee identity development tool in this workplace setting as a way to communicate institutional values and desired employee behaviors.

Setting and Participants

The study setting was a public, primarily non-residential university with highest research activity (Indiana University Center for Postsecondary Research, n.d.). This institution is the flagship university within a university system located in an urban setting in the Southwestern United States. The post-secondary institution’s enrollment was reported at about 41,000 students for fall semester 2012. Its student enrollment is reported as primarily undergraduate and is reported as selective with a high transfer student population (Indiana University Center for Postsecondary Research, n.d.). This institution reported in its on-line plan and budget for fiscal year 2012, the same period when data were collected for this study, that its operating and restricted funds budget was approximately $860 million, with about $210 million (24%) allocated to non-faculty employee salaries. Additionally, this institution reported 3,360 full-time equivalency non-faculty employee positions and 1,150 fulltime equivalency faculty positions in its on-line plan and budget for fiscal year 2012.

Total participants were 2,401 university employees, classified as non-faculty employees who were administered a performance appraisal for calendar year 2012. Participants represented all the employees at this post-secondary institution who were administered a performance appraisal during this calendar year and comprised a variety of professions including: accountants, human resources professionals, student affairs professionals, attorneys, skilled trade employees in its physical plant, and clerical staff employed in all departments and colleges at this post-secondary institution. The descriptive characteristics for these employees are presented in Table 1.

As presented in Table 1, there were 1,836 non-supervisory employees evaluated through the performance appraisal process at this institution in calendar year 2012. These non-supervisory employees were predominantly female at 60%. The largest ethnicity of non-supervisory employees was White at 33.5%. The majority of these employees had been employed at this institution 10 years or less. The average age of non-supervisory employees was 45. About 25% of non-supervisory employees had earned a Bachelor’s Degree as their highest level of education. About 50% these non-supervisory employees had earned less than a Bachelor’s Degree.

As also presented in Table 1, there were 565 supervisory employees evaluated in the performance appraisal process at this institution in calendar year 2012. These supervisory employees were predominantly female at 60%. They were predominantly White at 58%. The majority of supervisory employees were employed more than 10 years. The average age of supervisory employees was 48. A Bachelor’s Degree was the highest education for about 33% of supervisory employees and 41% had education beyond a Bachelor’s Degree.

Table 1

Characteristics of Employees as a Percentage of the Sample

Background and Use of the Performance Appraisal

In its policy on performance appraisal, this institution described the three goals of its performance appraisal process:

1. To provide clearly defined performance standards based upon the employee’s current job description to ensure that employees know what is expected of them.

2. To encourage supervisors and employees to have face-to-face discussions and provide employees feedback about their job performance.

3. To express appreciation for outstanding contributions and performance; conversely, to discuss performance areas where improvement is possible or needed and to outline plans for improving performance. (Human Resources, 2009, p.2)

According to its policy, performance appraisals are to be used by supervisors in determining employee promotion, training, development, and salary increases (Human Resources, 2009). All university employees who have been assigned to work at least 20 hours per week are required by policy to have their performance reviewed annually by their supervisor, except student and faculty positions (Human Resources, n.d.a.).

Four elements of a successful performance appraisal were identified by Human Resources and include: (a) the performance appraisal must be completed by the supervisor using the official form; (b) an in-person meeting must be held by the supervisor with the employee to discuss the appraisal; (c) the appraisal must be reviewed and ratified by the senior manager in the department; and (d) it must be approved by human resources (Human Resources, 2009; Human Resources, n.d.a.). Additionally, the appraisal must be completed during the timeframe specified by human resources (Human Resources, 2009).

At this institution, an electronic performance appraisal instrument was launched in 2010 to measure individual university staff performance of all non-faculty employees and represented the institution’s centralized implementation of its performance policy for non-faculty employees. It is an employee tool that is provided in an integrated human resources enterprise solution (Human Resources, n.d.c.). The performance appraisal instrument is an electronic form that was divided into four components: (a) job goals; (b) job responsibilities; (c) customer focus; and (d) competencies (Human Resources, n.d.a.). Supervisory employees were rated on a fifth component, manager responsibilities (Human Resources, n.d.a.). This electronic form is housed in a web-based, password protected human resource management system (Human Resources, n.d.a.).

According to human resources, there are four stages of the performance appraisal process that occur at two discrete time points (Human Resources, n.d.a.). During the first quarter of each calendar year, supervisors establish the rating and weighting criteria within the four sections of performance appraisal for non-supervisory employees and five appraisal sections for supervisory employees (Human Resources, n.d.a.). Supervisors are expected to meet with their employees during this timeframe to discuss performance goals and expectations for that calendar year (Human Resources, n.d.a.). Within the first three months of the subsequent calendar year, the supervisor should rate the employee’s performance for the prior year and submit the performance appraisal document for review to the departmental senior manager (Human Resources, n.d.a.). Following approval by the senior manager, the supervisor is required to meet with the employee to review prior year performance (Human Resources, n.d.a.). The final step in the process is that the performance appraisal document should be routed to human resources for review (Human Resources, n.d.a.).

Background and Use of Competencies in the Performance Appraisal

To develop the competency section in the performance appraisal, the human resources department contracted with a human resource consulting firm and created a competency library from which supervisors selected competencies when rating their employees (Human Resources, n.d.b). Campion et al. (2011) discussed benefits and challenges with using competency libraries to develop competency models for positions or role functions. The benefits for using these predeveloped competencies are that they have been mass-produced and are easy to implement (Campion et al., 2011). A downside of using pre-developed competency libraries is that the competency language may be unfamiliar to employees in the organization as these competencies are often developed from the study of multiple industries and organizations (Campion et al., 2011). Also, one of the challenges in using a list of generic competencies is that supervisors may select too many competencies (Campion et al., 2011). To help facilitate the competency selection process for supervisors at this university, human resources pre-loaded 12 competencies based on the position type for each staff member into the appraisal and noted that they clustered these competencies according to job level (Human Resources, n.d.a.). These classifications by job level are director, manager, supervisor, lead, professional, trades, athletics, research, and public safety (Human Resources, n.d.a.).

Table 2

Classification of Positions for Pre-loaded Competencies in Performance Appraisal

Human Resources, 2010.

Pre-loaded competencies for position classifications are presented in Table 3 in the exact order as prescribed by human resources.

Table 3

Pre-loaded Competencies by Position Classification

Human Resources, 2010

According to information from this institution’s human resources department, each employee was required to be rated on the performance of two to three competencies, and to facilitate this process, 12 competencies were pre-loaded for each employee during the performance appraisal process (Human Resources, n.d.b.). In total, supervisors selected 78 different competencies to rate employees at this public institution during the performance appraisal process. Even though empirical literature and the higher education professional organizations such as ACUBO, CUPAHR, and ACPA/NASPA have determined competencies based on specific positions, this organization made no differentiation among competencies used to rate different positions. This means that, even though the competency knowledge and skills may have different behavioral descriptions about the types of knowledge and skills required (i.e. employees in student affairs, finance, or plant operations departments), only one definition of this competency was provided to supervisors who chose to rate their employees on this competency.

On average, each employee was rated on 9.32 competencies. The lowest number of competencies selected for an employee in performance appraisal was 1 and the highest number of competencies selected was 24. The median number of competencies selected was 11 and the mode was 12.

A frequency analysis was conducted to understand how competencies were selected by supervisors. The most frequently selected competency was communication. This competency was selected to rate 1,912 employees (80% of employees at this institution). The next most frequently selected competency was knowledge and skills, which was used to rate 65% of employees. The top 10 competencies, their frequencies, and the percentage utilized to rate employees are included in Table 4.

Table 4

Top 10 Competencies Used to Rate Employees by Percentage

Variables and Statistical Processes Used to Answer the Research Question

In this section, the following will be discussed: (a) dependent and independent variables studied and (b) the statistical method used to answer the research question.

Independent and dependent variables. The research question this study sought to answer was: Within a university setting, how are employee competencies valued by job title within colleges and divisions? Employee competencies used in the analysis were operationalized using the competencies each supervisor selected to rate individual employees. The independent variables used in this analysis were employee: (a) gender; (b) ethnicity; (c) college or division in which employed; (d) function of the college or division; (e) years of service; and (f) highest education attained. The function of the college or department was determined by a researcher-assigned function code, as defined by the National Association of College and University Business Officers (NACUBO). For example, if an employee was employed in a college, this was assigned the function of instruction, or if an employee was in the department of human resources, this was assigned to the NACUBO function of institutional support.

In terms of college or division, the only two divisions treated differently than other divisions were the provost division and division of administration and finance in order to properly assign NACUBO function codes. When assigning NACUBO function codes, it was determined that the division of administration and finance had eight major departments that functioned as unique divisions with four different NACUBO functions: auxiliary enterprise, institutional support, plant, and public service. Likewise, the provost’s division had three major departments functioning as divisions with two unique functions: academic support and institutional support. The dependent variables were the frequencies by which particular competencies were used to rate each employee. These variables were categorical, which impacted the statistical analysis selected.

Statistical method used to answer the research question. The variables used in this study were categorical as the question of interest was about the choices that supervisors were making to select competencies for the appraisal. Understanding the choices that supervisors made to select competencies was important for understanding how supervisors used the evaluation of competencies to regulate employee identity in the performance appraisal. Knowing how competencies were valued was also important for predicting potential residual impacts of the measurement of competencies on this particular institution’s workplace culture. A statistical method recommended for use when the majority of variables are nominal, categorical, or qualitative is multiple correspondence analysis (Hoffman & de Leeuw, 1992). This type of statistical method clusters large amounts of categorical data to view relations between data (Hoffman & de Leeuw, 1992).

In order to prepare data from this study for multiple correspondence analysis, the first step was to create the database of qualitative information, as described by Hoffman and de Leeuw (1992). For example, this meant that, for each of the 2,401 employees and each of the competencies used in the analysis, a “yes” or “no” was indicated depending on whether this competency was selected by the supervisor to rate that employee. For the remainder of the database construction, participants were placed in their appropriate categories as illustrated in Tables 5 and 6.

As the goal of the analysis was to understand differences by job title, job title was operationalized by whether the employee was a supervisory or non-supervisory employee. Differences in competencies for supervisory and non-supervisory employees were expected as these supervisory employees had management responsibilities, which were not assigned to nonsupervisory employees. These employees were identified by whether they were evaluated on the category of manager responsibilities, a category in the performance appraisal. Following this categorization, employees were placed into one of two categories and separated into a spreadsheet for supervisory and non-supervisory employees.

Following the construction of the database, multiple correspondence analysis was conducted for supervisor and non-supervisory employees using XLSTAT 2015 to understand the number of dimensions necessary to explain these data and dominant eigenvalues as described by Hoffman and de Leeuw (1992). Because a substantial number of the 78 were only used infrequently, the study chose to focus on only those competencies utilized to evaluate a notable proportion (operationalized as 20% or more) of the university staff. This decision provides exploratory insight while also allowing more fully for model stability (Beh, 2004). The total number of competencies used in the analysis was 17 for non-supervisory employees and 20 for supervisory employees.

Tables 5 and 6 (in Appendix A) provide the listing of variables and supplementary variables used for non-supervisor and supervisor employee multiple correspondence analyses. Using the symmetric plots provided in the multiple correspondence analysis output, these plots were examined to determine relations among these data.

Limitations

There were several limitations that should be noted in this study. Importantly, as this was a case study of one institution, findings may only be generalizable to this institution and similar public post-secondary institutions. Future study is needed of multiple institutions and institution types to understand similarities and differences among different populations of university employees. Similarly, the study was limited by the institution’s own processes. Since this institution utilized 78 competencies in the rating of employees in the competency section of the performance appraisal, analysis and measurement of the process was further complicated. An additional limitation was that this institution made no differentiation among competencies used to rate different positions. If the same competency was selected to evaluate the performance of a director and a stationery engineer, both employees were rated using the same competency definition, even though these positions are different. The study was also limited by its use of quantitative data. Typically, in a case study design, multiple forms of data are synthesized to provide context about a phenomenon (Creswell, 2002).

Results

As described by Williams and Vogt (2011) and Abdi and Valentin (2007), multiple correspondence analysis is interpreted based on visual distances in the map. This means that the distance between the variables reflects differences between the variables, and those variables that are closer together are more similar (Abdi & Valentin, 2007; Hoffman & de Leeuw, 1992). The symmetric plots from the multiple correspondence analyses are presented in the following sections: (a) results from the multiple correspondence analysis for non-supervisory employees and (b) results from the multiple correspondence analysis for supervisory employees.

Results from the Multiple Correspondence Analysis for Non-Supervisory Employees

The first dimension (the horizontal axis) accounted for 65.13% of adjusted inertia, or explained variance. The second dimension (vertical axis) accounted for 23.89% of adjusted inertia, or explained variance. By reviewing the symmetric plots and comparing this to frequencies of competencies utilized to rate non-supervisory university employees, it appears that the first dimension represents competencies. The second dimension, or vertical axis, represents employee demographic data.

In Figure 1, competencies that were frequently used to rate non-supervisory employees are near the center of the figure, including: communication and knowledge and skills. Divisions and colleges that are located in the center of the figure represent those that selected the most frequently utilized competencies to rate their employees. These include: finance, administration, architecture, student affairs, humanities, research, and optometry. The primary functions of these colleges and divisions vary, including auxiliary enterprise, institutional support, and instruction.

Conversely, those competencies that were used less frequently are located closer to the border of the plot and included tenacity, applied learning, engagement readiness, and innovation. Likewise, using the symmetric plots, outlier colleges and divisions can be identified. These outlier colleges and divisions that were less similar than those colleges and departments in the center of the figure included: provost, social work, business, and education.

One can also observe uniqueness in specific colleges and divisions by reviewing the figure. Plant operations was one of the few departments for which contributing to team success was a key competency. This indicates that this competency may be a representative pre-loaded competency unique to these employees, as this competency was specific to this employee group and its function.

Likewise, in the top left-hand quadrant, the division of information technology was located near one competency building work relationships. This indicates that non-supervisory employees in this division were frequently rated on this competency. In the left-hand quadrant, there are three outlier colleges and divisions that were close together, meaning one would expect similarities in the competencies selected. These included athletics, education innovation, and the public TV/radio department. These divisions were located near the competencies of continuous learning and work standards, meaning one would expect that this competency was selected to rate these employees. Figure 1 below illustrates the output for non-supervisory employees.

Figure 1
Relations between Competencies, College/Division, and Demographic Characteristics for Non-Supervisory Employees

Results from the Multiple Correspondence Analysis for Supervisory Employees

The first dimension (the horizontal axis) accounted for 86.57% of adjusted inertia, or explained variance. The second dimension (vertical axis) accounted for 8.26% of adjusted inertia, or explained variance. By reviewing the symmetric plots and comparing this to frequencies of competencies utilized to rate supervisory university employees, it appeared that the first dimension reflected competencies used to rate these employees. The second dimension reflected demographic characteristics, such as ethnicity and college/department in which the employee works. Figure 2 provides a presentation of this output.

The major observed difference between supervisory and non-supervisory employee plots was that there was more variability in the competencies selected to rate supervisory employees (86.57%) as compared with non-supervisory employees (65.11%). This is reflected in the figure by the large number of squares (demographic characteristics) located in the center and the greater number of diamonds (competencies) outside the center of the figure. This may indicate that there was more specialization among supervisory employees. Additionally, it is also of note that there was less variability in the demographics of supervisory employees (8.26%) when compared with nonsupervisory employees (23.89%). Those frequently selected competencies for supervisory employees that were located toward the center of the figure included communication, coach and develop others, decision-making, and building trust.

The vertical axis overlaid demographic characteristics on the competency dimension of supervisory employees, and this provided additional detail about how competencies were selected to rate supervisory employees. For example, supervisory employees rated on the competency selling the vision were also likely to have been rated on the competency of influence. Supervisory employees employed in an area with a NACUBO function code of public service, which is mostly the TV/radio department, were likely to have been rated on the competencies of coach and develop others. Employees in the hotel college were most likely to have been rated on the competency communication. It was also noted that the honors and architecture colleges and information technology were outliers in the selection of competencies as these colleges and division were further away from all other colleges and divisions. Other outliers included finance, public safety, human resources, business, and president.

Figure 2
Relations between Competencies, College/Division, and Demographic Characteristics in Supervisory Employees

Discussion

The purpose of this study was to understand how competencies were valued in the performance appraisal process of one public, post-secondary institution in the Southwestern United States. As the selection of competencies in the performance appraisal represents the use of a corporate employee identity regulation tool in higher education, it is important to understand how this tool has been applied in real-world higher education settings. Likewise, as processes do leave a lasting impact on the workplace culture (Birnbaum, 2000), it was important to see a real example of this process in action to understand its potential ramifications on the workplace culture. Using evidence from the synthesis of information about this institution’s implementation as well as an analysis of descriptive and multiple correspondence analysis, the major theme that emerged was that the competency selection process by supervisors in the performance appraisal was shallow and mechanical. This is in contrast with a process that could be helpful in advancing non-faculty employee role definition and professional growth and development at this institution.

In the following sections, evidence of the mechanical process will be presented by comparing this institution’s process with best practices from human resources and higher education literature. Likewise, study findings were examined in concert with the study’s guiding ideas of employee identity regulation and neoliberalism in higher education. Additionally, ways to improve the use of competencies at this institution and other post-secondary institutions will be described. Likewise, next steps for further study and research will be discussed.

The Mechanical Use of Competencies in This Institution’s Performance Appraisal Process

One indication of the mechanical nature of the competency selection process at this institution was the use of a competency library developed by a consulting firm. As Campion et al. (2011) noted, competency libraries can provide organizations an efficient way to create competency models, but it is important to customize these competencies to the organizational culture. Likewise, competency libraries created by consulting firms reflect competencies developed by examining multiple industries (Campion et al., 2011). This was reflected by Birnbaum’s (2000) assertion that, in order to even have a possibility for success, corporate tools should be modified to reflect the institutional culture. Likewise, according to Lievens et al. (2004) and Pulakos (2004), the way a competency is described should fit organizational values and be in wording familiar to employees.

The use of a competency library may have contributed to the large number of competencies selected by supervisors. On average, 9.32 competencies were selected. Similarly, the most frequently selected number of competencies was 12. As Campion et al. (2011) maintained, when there is a laundry list of competencies, there is a tendency to select more. This author noted that, in order for supervisors to select fewer competencies with these long lists, the competency selection process must be structured (Campion et al., 2011). Further, Pulakos (2004) recommended that between 5 and 10 competencies be used to rate employees. Also, Campion et al. (2011) asserted that it was preferable to use fewer and more descriptive competencies. As noted in the institution’s performance appraisal instructions to supervisors, supervisors were instructed to select 2 to 3 competencies and delete the remainder. This could be an indication that the instructions were not clear to supervisors or that supervisors did not understand the value of competency selection in the performance appraisal.

Likewise, competency models, described as pre-loaded competencies by the institution, were assigned in most cases to generic position titles, such as director, manager, lead, and the like. There was some attempt to develop position-specific competency models for public safety, athletics, trades, and research; however, it was not described how these competencies related to those developed by professional organizations or if those competencies were even considered. Importantly, in this study it was presented that several higher education professional organizations have developed role-specific competencies, some of which may overlap. This was not discussed in this institution’s implementation of competencies. As Birnbaum (2000) further noted, when implementing a corporate tool in higher education, it is important to have a full awareness of the existing theory and knowledge about the topic prior to launching an implementation.

Another indication of the mechanical implementation of competencies in this institution’s performance appraisal was the dual function of the appraisal. This institution’s performance appraisal policy noted that the performance appraisal functioned in an administrative way, to make decisions for raises and promotions (Human Resources, 2009). It also functioned to help supervisors identify opportunities for professional development and training (Human Resources, 2009). This dual function was illustrated in the multiple appraisal sections, which included: (a) job goals; (b) job responsibilities; (c) customer focus; and (d) competencies. Managers were rated on a fifth section, manager responsibilities. Some researchers have noted that it is difficult for one performance appraisal instrument to serve a dual function (Boswell & Boudreau, 2002; Kondrasuk, 2011). Kondrasuk (2011) noted that the two goals of the performance appraisal – administrative and developmental – should be separated so that it is clear to both the employee and supervisor which process is occurring. Kondrasuk (2011) further argued that this separation helps with rater training so supervisors can be equally capable to serve as mentors and evaluators.

The mechanical nature of the use of competencies was also evident in the multiple correspondence analysis. For example, even when analyzing the competencies selected to rate employees at least 20% of the time, there were still competency outliers. For non-supervisory employees, these included the competencies of applied learning, tenacity, and engagement readiness. These competencies were selected so infrequently that they could not be connected to a particular college, department, or an employee demographic. This further illustrates the importance of selecting competencies that are connected to the culture of the college, department, or institution for them to be valuable to regulate employee identity. Organizational competencies used to rate employees should be connected to the employee mission and reflect organizational values and language (Lievens et al, 2004; Pulakos, 2004).

Similarly, there was more observed variability in the number of competencies used to rate supervisory employees as compared with non-supervisory employees. This may be an illustration of why the student affairs profession determined to use levels within each of its professional competencies, as it was noted by ACPA/NASPA (2015) that all employees are expected to be competent at the foundational level. As employees gained experience, these levels provide insight to employees about how to progress within each competency area (ACPA/NASPA, 2015). It could be that supervisors at this institution are using more competencies to account for these fine-grained differences because there are no levels within each competency. In the current process, a nonsupervisory stationery engineer would be rated on the same level as a supervisory employee in student affairs if the same competency was selected to rate each of them.

However, even though an analysis illustrated that the process of competency evaluation in the performance appraisal was mechanical, this does not mean that there were no valuable lessons to be learned about identity regulation that may be occurring in this process. For example, nonsupervisory employees in the plant operations division were frequently rated on the competency of contributing to team success, as noted in Figure 1 of the multiple correspondence analysis symmetric plot. In any further evolution of competency usage, it would be important to understand more about the meaning of this competency for this group and how this skill is exhibited. Likewise, for non-supervisory employees, the divisions of athletics, public TV/radio department, education innovation, and information technology were located close to one another and the competencies of work standards, building work relationships, and continuous learning. It would be important to understand more about the similarities between these employees to further develop competency models for these employees. As another example, supervisory employees from information technology and human resources were outliers at opposite ends of Figure 2. Understanding that there are specialized competencies for higher education human resources professionals may explain why human resources is an outlier. It would be important to understand if there are specialized competencies that have been developed for information technology to explain why it is an outlier

Examining the mechanical nature of competency selecting and study guiding ideas. The framework for this study included the dual guiding ideas that the selection and use of competencies in the non-faculty performance appraisal was both a corporate tool and a way for employers to regulate non-faculty employee identity. As a corporate tool in higher education, the use of competencies in this institution’s performance appraisal provided an example of the continued encroachment of neoliberalism in higher education. The mechanical nature of the selection and application of competencies by this post-secondary institution presented a concern when considered alongside the guiding ideas. If the application and selection of competencies are mechanical, then employees may de-value the use of competencies and their own identity as an employee. Likewise, if the selection of competencies by supervisors is rote, the process will become just another form or task to complete. As such, there will be little meaning connected with how competencies are intended to be valued as a potential tool to enhance and develop employee identity in the higher education workplace.

Next Steps for Policy and Practice

The key study finding that the selection of competencies in the performance appraisal was mechanical and that there are potential higher-education specific competency alternatives available illustrates that now may be the time to launch major reform at this university. Potential holistic uses of role-based, higher education-specific competencies at this and other institutions should be to: (a) evaluate employee performance; (b) develop institution-wide succession plans; (c) orient employees; and (d) develop role-based and institution-wide professional development (ACPA/NASPA, 2015). Similar to potential applications of student affairs competencies proposed by ACPA/NASPA, competencies could also be used to explain the important roles of non-faculty employees in their institutions. As a web search indicated a number of institutions have been using competencies to evaluate employee performance, it may be important that these institutions also examine their performance appraisal process and consider adopting higher-education specific competencies as well.

Future uses of competencies utilized to measure and evaluate non-faculty employees should be higher education-specific and should describe the key knowledge and skills required in higher education and institution-specific language. As noted by Birnbaum (2000), changes in the use of competencies at institutions should be thoughtfully developed and pilot-tested before a large-scale implementation is launched. Further, the impact of these changes in a pilot study should be measured so that improvements can be made before a large-scale roll out is considered, as recommended by Birnbaum (2000).

Finally, another important recommendation to refine the process of performance appraisal for non-faculty employees at this and other post-secondary institutions is to consider using separate performance appraisal instruments for administrative and developmental employee decisions. As researchers have indicated, it is difficult for one instrument to fulfill these discrete purposes (Kondrasuk, 2011; Pulakos, 2004). Likewise, it is difficult for supervisors to serve as evaluator and mentor in one setting (Kondrasuk, 2011). Separating these functions also offers the important opportunity to evolve the performance appraisal from a once-a-year activity into an on-going process. Similarly, prior research has indicated that employees desire more direct feedback from their supervisors than they actually obtain (Roberts, 1994) and university administrators seek appreciation from senior leadership for their efforts (Rosser, 2004).

Continued Evaluation of Tools Used to Evaluate Non-Faculty Employee Performance

This study reinforced the importance of assessing the current value of the usage of competencies to evaluate non-faculty employee performance and for engaging higher education leadership in a discussion about the role of higher education specific competencies in employee development and evaluation. As a web search revealed that a number of universities are using some type of competency measurement in their performance appraisal instruments for non-faculty employees, it would be important for these institutions to understand how their own instrument is actually being used by supervisors. Birnbaum (1988) noted that higher education has a loosely coupled, decentralized culture, different than the more tightly-coupled corporate setting. Birnbaum (2000) further asserted that, when corporate tools are introduced into the higher education culture, assessment of their impact is important. Thus, it is critical that these institutions understand how and if their particular performance appraisal is functioning as a tool to assist managers in managing employee performance. Importantly, future study would help higher education institutions develop a performance appraisal process that separately serves the functions of an administrative tool and an employee developmental tool.

Future study of the use of competencies should evaluate how leaders anticipate that the evaluation of employee competencies could be an important management tool. As described by Birnbaum (2000), it is important that higher education leaders recognize that corporate tools should not replace but supplement good management techniques. A future study might include a pilot test in a department at a post-secondary institution using some of the higher-education specific competencies that have been developed by one of the professional organizations. It would be important to understand more about how competencies can be applied in these settings and the benefits perceived by administrators with these applications. Such studies might look at separating the performance development process from the administrative functions of the employee performance appraisal, using higher-education specific competencies and measuring whether this provides a more valued process than is currently in place.

Conclusion The purpose of this study was to provide insights into how competencies were selected by supervisors to rate non-faculty employees at one public post-secondary institution. There were several goals for this analysis. The first was to understand how the selection and use of competencies functioned as a tool to regulate employee identity at this post-secondary institution. A second goal was to understand the potential impact of this corporate tool on this higher education setting. A third goal was to understand if this tool, which represented an encroachment of neoliberalism in higher education, could be modified to be more reflective of the unique higher education workplace setting.

Through this study, it was observed that competency selection at this university was mechanical. A challenge with a mechanical process is that it devalues the process in the minds of the employer and the employee. By having a rote process in place for competency selection, employees may feel that their role is not valued. A review of current practices illustrated that there are higher education specific examples of the use of competencies from NASPA, ACUBO, and CUPA-HR that could provide a starting point for this institution to use competencies in its performance appraisal that might be more relevant and more valued by employees and supervisors. This study finding illustrated the importance for public post-secondary institutions to use their available data to analyze how a tool is utilized to understand if it is being implemented as intended. As recommended by Birnbaum (2000), it is important to understand how a corporate tool should be modified to fit the higher education environment prior to implementing the tool wholesale from a corporate setting, such as using a competency library.

Prior studies have demonstrated the importance of this population to the institutions and the faculty and students they serve. By having a more valued performance appraisal tool in place, this institution, and others like it, may find their performance appraisal communicates desired organizational values and behaviors to their employees. This may improve these employees’ abilities to provide valuable service to their higher education communities and the students and faculty they serve.

References

Abdi, H., & Valentin, D. (2007). Multiple correspondence analysis. Encyclopedia of measurement and statistics, 651-657. http://dx.doi.org/10.4135/9781412952644.n299

Alvesson, M. & Willmott, H. (2002). Identity regulation as organizational control: Producing the appropriate individual. Journal of Management Studies, 39(5), 619-644. http://dx.doi.org/10.1111/1467-6486.00305

American College Personnel Association & National Association of Student Personnel Administrators. (2010). ACPA/NASPA professional competency areas for student affairs practitioners. Retrieved from https://www.naspa.org/images/uploads/main/Professional_Competencies.pdf

American College Personnel Association & National Association of Student Personnel Administrators. (2015). Professional competency areas for student affairs educators. Retrieved from http://www.naspa.org/images/uploads/main/ACPA_NASPA_Professional_Competencies_. pdf

Beh, E. J. (2004). Simple correspondence analysis: A bibliographic review. International Statistical Review, 72(2), 257–284. http://dx.doi.org/10.1111/j.1751-5823.2004.tb00236.x

Birnbaum, R. (1988). How colleges work: The cybernetics of academic organization and leadership. San Francisco: Jossey-Bass.

Birnbaum, R. (2000). The life cycle of academic management fads. Journal of Higher Education, 71(1), 1-16. doi:10.2307/2649279

Boyatzis, R. E. (2008). Competencies in the 21st century. Journal of Management Development, 27(1), 5- 12. http://dx.doi.org/10.1108/02621710810840730

Boyatzis, R. E. (2009). Competencies as a behavioral approach to emotional intelligence. Journal of Management Development, 28(9), 749-770. http://dx.doi.org/10.1108/02621710910987647

Bresciani, M., Monzon, R., & Oyler, J. (2011). A comparison of senior student affairs officer and student affairs preparatory program faculty expectations of entry-level professionals’ competencies. Journal of Student Affairs Research Practice, 48(4), 463-479.

Burkard, A. W., Cole, D. C., Ott, M., & Stoflet, T. (2005). Entry-level competencies of new student affairs professionals: A Delphi study. Journal of Student Affairs Research and Practice, 42(3), 545- 571. http://dx.doi.org/10.2202/1949-6605.1509

Campion, M. A., Fink, A. A., Ruggeberg, B. J., Carr, L., Phillips, G. M., & Odman, R. B. (2011). Doing competencies well: Best practices in competency modeling. Personnel Psychology, 64(1), 225-262. http://dx.doi.org/10.1111/j.1744-6570.2010.01207.x

Cleveland, J. N., Murphy, K. R., & Williams, R. E. (1989). Multiple uses of performance appraisal: Prevalence and correlates, Journal of Applied Psychology, 74(1), 130-135. http://dx.doi.org/10.1037/0021-9010.74.1.130

Creswell, J. W. (2002). Educational research: Planning, conducting, and evaluating quantitative. Prentice Hall.

CUPA-HR. (n.d.a.). The CUPA-HR learning framework. Retrieved from http://www.cupahr.org/learning/framework.aspx.

CUPA-HR. (n.d.b.). CUPA-HR learning framework. Retrieved from http://www.cupahr.org/learning/files/CUPA-HR-Learning-Framework.pdf.

Cuyjet, M. J., Longwell-Grice, R., & Molina, E. (2009). Perceptions of new student affairs professionals and their supervisors regarding the application of competencies learned in preparation programs. Journal of College Student Development, 50(1), 104-119. http://dx.doi.org/10.1353/csd.0.0054

Draganidis, F., & Mantzas, G. (2006). Competency-based management: A review of systems and approaches. Information Management and Computer Security, 14(1), 51-64.

Dreyfus, C. R. (2007). Identifying competencies that predict effectiveness of R&D managers. Journal of Management Development, 27(1), 76-91. http://dx.doi.org/10.1108/09685220610648373

Eisenhardt, K. M. (1989). Building theories from case study research. Academy of management review, 14(4), 532-550.

Ewell, P. T. (1999). Imitation as art: Borrowed management techniques in higher education. Change: The Magazine of Higher Learning, 31(6), 10-15. http://dx.doi.org/10.1080/00091389909604228

Gangani, N., McLean, G. N., & Braden, R. (2006). A competency-based human resource development strategy. Performance Improvement Quarterly, 19(1), 127-140. http://dx.doi.org/10.1111/j.1937-8327.2006.tb00361.x

Giroux, H. A. (2002). Neoliberalism, corporate culture, and the promise of higher education: The university as a democratic public sphere. Harvard Educational Review, 72(4), 425-464. http://dx.doi.org/10.17763/haer.72.4.0515nr62324n71p1

Giroux, H. A. (2013). The corporate war against higher education. Workplace: A journal for academic labor, (9). http://dx.doi.org/10.17763/haer.72.4.0515nr62324n71p1

Gordon, G., & Whitchurch, C. (2007). Managing human resources in higher education. Higher Education Management and Policy, 19(2), 1-21. http://dx.doi.org/10.1787/hemp-v19-art14-en

Graham, C. (2010). Hearing the voices of general staff: a Delphi study of the contributions of general staff to student outcomes. Journal of Higher Education Policy and Management, 32(3), 213-223. http://dx.doi.org/10.1080/13600801003743315

Grote, D. (1996). The complete guide to performance appraisal. New York: American Management Association.

Hoffman, D. L., & de Leeuw, J. (1992). Interpreting multiple correspondence analysis as a multidimensional scaling method. Marketing Letters, 3(3), 259-272. http://dx.doi.org/10.1007/BF00994134

Hopkins, M. M. & Bilimoria, D. (2008). Social and emotional competencies predicting success for male and female executives. Journal of Management Development, 27(1), 13-35. http://dx.doi.org/10.1108/02621710810840749

Human Resources. (n.d.a.). ePerformance FAQs. (Link available upon request)

Human Resources. (n.d.b.). ePerformance (Link available upon request)

Human Resources. (n.d.c.). ePerformance training for managers. (Link available upon request)

Human Resources. (2009). Manual of administrative policies and procedures 02.01.04 staff performance appraisal. (Link available upon request)

Indiana University Center for Postsecondary Research (n.d.). The Carnegie Classification of Institutions of Higher Education, 2015 ed. Bloomington, IN: Author.

Judge, T. A., & Ferris, G. R. (1993). Social context of performance evaluation decisions. The Academy of Management Journal, 36(1), 80-105. http://dx.doi.org/10.2307/256513

Kezar, A. (2007). Creating and sustaining a campus ethos encouraging student engagement. About Campus 11(6), 13-18. http://dx.doi.org/10.1002/abc.190

Kinzie, J. & Kuh, G .D. (2004). Going DEEP: Learning from campuses that share responsibility for student success. About Campus, (November-December), 2-8. http://dx.doi.org/10.1002/abc.105

Kline, T. J., & Sulsky, L. M. (2009). Measurement and assessment issues in performance appraisal. Canadian Psychology, 50(3), 161-171. http://dx.doi.org/10.1037/a0015668

Knapp, L. G., Kelly-Reid, J. E., & Ginder, S.A. (2012). Employees in Postsecondary Institutions, Fall 2011 and Student Financial Aid, Academic Year 2010-11: First Look (Provisional Data) (NCES 2012-156rev). U.S. Department of Education. Washington, DC: National Center for Education Statistics. Retrieved 07/02/15 from http://nces.ed.gov/pubsearch

Kochanski, J. (1997). Competency-based management. Training and Development-Alexandria-American Society for Training and Development, 51, 40-45.

Kondrasuk, J. N. (2011). So what would an ideal performance appraisal look like? The Journal of Applied Business and Economics, 12(1), 57-71.

Kretovics, M. (2002). Entry-level competencies: What student affairs administrators consider when screening candidates. Journal of College Student Development, 43, 912-920.

Kuk, L., Cobb, B., & Forrest, C. S. (2007). Perceptions of competencies of entry-level practitioners in student affairs. Journal of Student Affairs Research and Practice, 44(4), 1061-1088.

Lamal, P. A. (2001). Higher education: Social institution or business? Behavior and Social Issues, 11(1), 65-70. http://dx.doi.org/10.5210/bsi.v11i1.101

Lievens, F., Sanchez, J. I., & De Corte, W. (2004). Easing the inferential leap in competency modeling: The effects of task-related information and subject matter expertise. Personnel Psychology, 57, 881-904.

Lunenberg, F. C. (2012). Performance appraisal: Methods and rating errors. International Journal of Scholarly Academic Intellectual Diversity, 14(1), 1-9.

Martone, D. (2003). A guide to developing a competency‐based performance‐management system. Employment Relations Today, 30(3), 23-32. http://dx.doi.org/10.1002/ert.10095

McClelland, D. C. (1973). Testing for competence rather than intelligence. American Psychologist, 28(1), 1-40. http://dx.doi.org/10.1037/h0034092

McClelland, D. C. (1998). Identifying competencies with behavioral-event interviews. Psychological Science, 9(5), 331-339. http://dx.doi.org/10.1111/1467-9280.00065

Moussavi, F., & Ashbaugh, D. L. (1995). The effects of participative, goal-oriented performance appraisal: A field study in public agencies. Journal of Public Administration Research and Theory, 5(3) 331-343.

National Association of Student Personnel Administrators. (n.d.). About student affairs. Retrieved September 30 from https://www.naspa.org/about/student-affairs.

Orton, J. D., & Weick, K. E. (1990). Loosely coupled systems: A reconceptualization. Academy of Management Review, 15(2), 203-223.

Pulakos, E. D. (2004). Effective practice guidelines: Performance management. Retrieved from https://www.pdri.com/images/uploads/Performance_Management.pdf

Roberts, G. E. (1994). Maximizing performance appraisal system acceptance: Perspectives from municipal government personnel administrators. Public Personnel Management, 23(4), 525-549. http://dx.doi.org/10.1177/009102609402300402

Rosser, V. J. (2004). A national study on midlevel leaders in higher education: The unsung professionals in the academy. Higher Education, 48(30), 317-337. http://dx.doi.org/10.1023/B:HIGH.0000035543.58672.52

Scheeres, H., & Rhodes, C. (2006). Between cultures: Values, training and identity in a manufacturing firm. Journal of Organizational Change Management, 19(2), 222-236. http://dx.doi.org/10.1108/09534810610648924

Schreiner, L. A., Noel, P., Anderson E., & Cantwell, L. The impact of faculty and staff on high-risk college student persistence. Journal of College Student Development, 52(3), 321-338. http://dx.doi.org/10.1353/csd.2011.0044

Scott, S. G., & Einstein, W. O. (2001). Strategic performance appraisal in team-based organizations: One size does not fit all. Academy of Management Executive, 15(2), 107-116. http://dx.doi.org/10.5465/AME.2001.4614990

Scudder, J. N., & Guinan, P. J. (1989).Communication competencies as discriminators of superiors’ ratings of employee performance. Journal of Business Communication, 26(3), 217-229. http://dx.doi.org/10.1177/002194368902600302

Shippmann, J. S., Ash, R. A., Battista, M., Carr, L., Eyde, L. D., Hesketh, B., . . . Sanchez, J. I. (2000). The practice of competency modeling. Personnel Psychology, 53, 703-740. http://dx.doi.org/10.1111/j.1744-6570.2000.tb00220.x

Spence, J. R., & Keeping, L. (2011). Conscious rating distortion in performance appraisal: A review, commentary, and proposed framework for research. Human Resources Management Review. 21, 85- 95. http://dx.doi.org/10.1016/j.hrmr.2010.09.013

Swann, W. B. Jr., Johnson, R. E., & Bosson, J. K. (2009). Identity negotiation at work. Research in Organizational Behavior, 29, 81-109. http://dx.doi.org/10.1016/j.riob.2009.06.005

Szekeres, J. (2004). The invisible workers. Journal of Higher Education Policy and Management, 26(91), 7- 22. http://dx.doi.org/10.1080/1360080042000182500

Szekeres, J. (2006). General staff experiences in the corporate university. Journal of Higher Education Policy and Management, 28(2), 133-145. http://dx.doi.org/10.1080/13600800600750962

Szekeres, J. (2011). Professional staff carve out a new space. Journal of Higher Education Policy & Management, 33(6), 679-691. http://dx.doi.org/10.1080/1360080X.2011.621193

Waple, J. N. (2006). An assessment of skills and competencies for entry-level student affairs work. NASPA Journal, 43(1), 1-18. http://dx.doi.org/10.2202/0027-6014.1568

Wiersma, U., & Latham, G.P. (1986). The practicality or behavioral observation scales, behavior expectation scales, and trait scales. Personnel Psychology, 39, 619-628. http://dx.doi.org/10.1111/j.1744-6570.1986.tb00956.x

Yin, R. K. (1981). The case study crisis: Some answers. Administrative science quarterly, 58-65. http://dx.doi.org/10.2307/2392599

Appendix A

Appendix

Table 5

ariables Used in the Non-Supervisory Multiple Correspondence Analysis

Table 6

Variables Used in the Supervisory Multiple Correspondence Analysis

Notes