Effects of Personalized Versus Generic Implementation of an Intra-Organizational Online Survey on Psychological Anonymity and Response Behavior: A Field Experiment

Purpose

In the present study, we sought to investigate the effects of online survey implementation strategies on perceived anonymity and employee response behavior in organizational surveys.

Design/Methodology/Approach

A field experiment was conducted to compare two commonly used online survey implementation strategies (N = 815). One group of employees received a personalized invitation to the survey and a log-in password, while the other group received a general invitation and did not have to provide a password.

Findings

The results showed that the applied implementation strategies had no substantial effects on perceived anonymity. Moreover, there were no significant effects on nonresponse or the responses of survey participants to closed-ended and open-ended survey questions.

Implications

The present study supposes that online surveys are not a uniform phenomenon and that differences in the implementation of online surveys need to be considered. However, the findings indicate that the use of specified-personalized implementation strategies does not necessarily lead to a substantial decrease in perceived anonymity or automatically result in reduced data quality. Thus, in many cases, the investigated online survey implementation strategies are unlikely to cause serious reductions in perceptions of anonymity and quality of responses to organizational online surveys.

Originality/Value

In spite of the frequent use of online surveys in organizations, little is known about the consequences of online implementation strategies for perceptions of anonymity and response behavior. This study is one of the few empirical examinations of the psychological consequences of different online implementation strategies frequently used in organizational surveying.     

The Impact of Managerial Coaching Behavior on Employee Work-Related Reactions

Purpose

The purpose of this study was to examine the relationships between perceived managerial coaching behavior and employee work-related outcomes.

Design/Methodology/Approach

Data were collected from 482 employees in a Korean public organization. The collected data were analyzed by structural equation modeling with a two-step approach.

Findings

The hypothesized conceptual model was adequately supported by the sample data. Further investigations suggested managerial coaching, which had a direct impact on employee satisfaction with work and role clarity and an indirect impact on satisfaction with work, career commitment, organization commitment, and job performance.

Implications

Findings provide empirical support to the hypothesized conceptual model of managerial coaching outcomes in organizations. Study findings offer evidence regarding prospective, but unexamined, benefits of managerial coaching. Such knowledge can be also used by practitioners for selecting and developing effective managers and leaders and understanding and managing employee attitudes and behaviors in organizations.

Originality/Value

This article is one of the first studies to provide evidence for the influence of managerial coaching behavior on employee role cognition, work attitudes, and performance. Since there is no commonly acknowledged theory or conceptual model for managerial coaching outcomes, this finding of the current hypothesized model can notably contribute to the research on managerial coaching. Furthermore, to date, no study of managerial coaching in Asian cultural contexts has been identified.     

A Dispositional Approach to Work�CSchool Conflict and Enrichment

Purpose

The purpose of this study was to investigate a dispositional model of the work?Cschool interface. In particular, we examined the relationship between core self-evaluations (CSEs) and proactive personality with both work?Cschool conflict (WSC) and work?Cschool enrichment (WSE) as well as a variety of work and school outcomes.

Design/methodology/approach

314 employed college students were surveyed about their experiences managing work and school roles.

Findings

Structural equation modeling showed that CSEs were related to both WSC and WSE as well as school outcomes, whereas proactive personality was related to WSE and job and school outcomes but not WSC. WSC was negatively related to school performance, whereas WSE was positively related to school and job satisfaction as well as school performance.

Implications

Organizations, universities, and researchers should be aware that dispositional variables influence perceptions of work and school roles as well as important outcome variables, and that further intervention efforts may be needed to help students manage work and school roles.

Originality/Value

Past studies have demonstrated that job characteristics influence the work?Cschool interface, but this study is among the first to demonstrate that dispositional factors also relate to WSC and WSE.     

Job Design and the Employee Innovation Process: The Mediating Role of Learning Strategies

Purpose

The purpose of this article is to examine whether employee learning strategies is a mechanism through which job design affects the employee innovation process. In particular, we test whether work-based learning strategies mediate the relationship between job design characteristics (job control and problem demand) and key components of the innovation process (idea generation, idea promotion, and idea implementation).

Design/Methodology/Approach

Data were collected from a survey of 327 employees in a UK manufacturing organization.

Findings

Structural equation modeling confirmed the mediating role of learning strategies in the relationship between job design and idea generation. The effects of job control on idea generation were mediated by work-based learning strategies and the effects of problem demand on idea generation were partially mediated by work-based learning strategies. Problem demand also had a direct relationship with idea generation and idea promotion. The findings provide support for the general idea that learning is a mechanism thorough which job design affects outcomes.

Implications

The results of the study show practitioners that creating jobs with high control or high problem demand can help to promote the employee innovation process; and that this is partly due to the role that such jobs play in stimulating the use of learning strategies at work.

Originality/Value

This article develops and tests a new theoretical model that explains how learning is a route through which job design influences employee innovation.     

Employees’ Adaptability and Perceptions of Change-Related Uncertainty: Implications for Perceived Organizational Support,Job Satisfaction,and Performance

Purpose

The literature on organizational change has increasingly recognized that characteristics of change recipients influence their reactions to workplace change. Yet little is known about the influence of employees’ adaptability and change-related uncertainty on their interpretation of organizational actions. We examined these antecedents and the mediating role of perceived organizational support as explanations for employees’ job satisfaction and performance.

Design/Methodology/Approach

A survey was administered to material handling employees from two organizations. Employees completed measures of individual adaptability, uncertainty experienced regarding changes in the workplace, support received from the organization, and job satisfaction. Performance data were collected from the records of one organization.

Findings

Results from both samples support the role of perceived organizational support as a mediator of the relationship between employees’ adaptability and perceptions of change-related uncertainty and employees’ satisfaction and performance.

Implications

Change is a frequent occurrence in today’s workplace; thus, improving employee satisfaction and performance requires the consideration of change-related perceptions and individuals’ dispositions relevant to change. The present study offers insights regarding how organizations may help improve perceptions of organizational support by reducing perceived uncertainty as well as identifying employees who may need assistance to adapt to workplace changes.

Originality/Value

Despite practitioners’ expressed interest, there is scant research examining employees’ adaptability and change-related uncertainty. We provide the first evidence explaining how and why these variables impact important workplace outcomes and extend existing theory by identifying appraisals of the organization (and not the self) as a mechanism explaining stressor–strain relationships.     

A Meta-Analytic Review of the Consequences Associated with Work–Family Enrichment

Purpose

This study investigated the relationship between work-to-family enrichment (WFE) and family-to-work enrichment (FWE) with work-related, non work-related, and health-related consequences using meta-analysis.

Design/methodology/approach

We conducted a meta-analytic review of 21 studies (54 correlations) for WFE and 25 studies (57 correlations) for FWE.

Findings

We found that both WFE and FWE were positively related to job satisfaction, affective commitment, and family satisfaction but not turnover intentions. WFE was more strongly related to work-related variables, whereas FWE was more strongly related to non work-related variables. We also found that both WFE and FWE were positively related to physical and mental health. Additionally, relationships appear to depend on moderating variables including the proportion of women in the sample as well as the construct label (e.g., enrichment, facilitation, positive spillover).

Implications

Our work indicates that organizations need to consider ways to not only reduce conflict, but also increase enrichment, which will drive many important outcome variables.

Originality/value

This is the first meta-analysis on the positive side of the work–family interface.     

Does Dispositional Aggression Feed the Narcissistic Response? The Role of Narcissism and Aggression in the Prediction of Job Attitudes and Counterproductive Work Behaviors

Purpose

The purpose of this study was to extend the personality and work-related outcomes literature by examining: (1) the effects of narcissism on job satisfaction, organizational commitment, and counterproductive work behaviors and (2) the moderating effects of dispositional aggression on the narcissism–counterproductive work behaviors relationship.

Design/Methodology/Approach

Multi-wave data were collected from 381 workers employed in a variety of work settings within the United States.

Findings

Narcissism had consistent main effects on job satisfaction, organizational commitment, and counterproductive work behaviors after controlling for the personality characteristics agreeableness, conscientiousness, neuroticism, and aggression. Aggression moderated the relationship between narcissism and counterproductive work behaviors, such that the positive relationship between narcissism and counterproductive work behaviors was stronger when dispositional aggression was high.

Implications

This study provides greater clarity regarding the role of narcissism within job attitudes and behaviors, shows that dispositional aggression strengthens or amplifies the narcissistic response for counterproductive work behaviors, and provides a platform to further explore narcissism and aggression within the workplace.

Originality/Value

This study found that narcissism consistently predicted unique variance in work-related criteria after the effects of agreeableness, conscientiousness, neuroticism, and aggression were controlled. It also found evidence that aggression moderates the effects of narcissism on counterproductive work behaviors. This study is the first to directly examine these effects.     

Disaggregating Job Satisfaction: Effects of Perceived Demands,Control, and Support

Purpose

Our purpose was to inductively examine how employees’ perceptions of stressor and resource work characteristics relate to nine distinct facets of job satisfaction, in accordance with the demand–control (–support) [JDC(S)] theory. Job satisfaction is a multidimensional construct composed of various facets that differ greatly from one other. However, most stress studies have examined job satisfaction at the global level. This may be problematic for managers looking to redesign the workplace to increase employee job satisfaction based on established recommendations from previous research.

Design/Methodology/Approach

Data were obtained from employees of a public sector human services organization that provides services to protect children from neglect and other forms of abuse (n = 343). A series of t tests for dependent correlations determined facet-level differences in satisfaction with demand and resource work characteristics.

Findings

We found different patterns of correlations across facets for all seven demand, control, and support workplace characteristics.

Implications

Those re-balancing types of demands, control, and support for workplace redesign should not presume consistent effects on all aspects of satisfaction with work. By doing so, expected positive outcomes may fail to materialize, perhaps to the detriment of workplace redesign as a whole in the minds of management. A more detailed approach to workplace redesign is recommended.

Originality/Value

This is the first study which adopts the JDC(S) framework to examine job satisfaction at the facet level. Our findings shed new light on how workplace characteristics relate to different aspects of satisfaction.     

Deleterious Consequences of Change in Newcomers’ Employer-Based Psychological Contract Obligations

Purpose

This study examines the issue of change in newcomers’ employer-based psychological contract obligations over time, viewing change as a potentially important determinant of perceived contract breach and subsequent employee attitudes and behaviors.

Design/Methodology/Approach

Data were collected using a three-wave longitudinal design from newly hired faculty members (N = 106).

Findings

Newcomers’ perceptions of employer-based relational obligations significantly decreased during their first year on the job. Newcomers reacted negatively to these changes, subsequently reporting increased contract breach and more negative work attitudes (i.e., increased turnover intentions and reduced job satisfaction and organizational loyalty).

Implications

This study provides evidence of the negative effects of perceived changes to a newcomer’s psychological contract. Practitioners should implement interventions to ensure a realistic set of psychological contract obligations are developed from the start in order to minimize the likelihood that newcomers will modify these obligations downward; and, therefore, experience these negative attitudes toward the organization.

Originality/Value

Drawing from the realistic job preview and socialization literatures, this study examines a topic that has received little empirical attention in the extant psychological contract research, yet has important implications to the management of employees’ psychological contracts. Using both a three-wave longitudinal field design and a more rigorous statistical analysis for assessing change (i.e., latent growth curve modeling), we add a unique contribution to the extant research by identifying the negative consequences of psychological contract change on newcomers’ subsequent work perceptions and attitudes.     

Work Passion Through the Lens of Culture: Harmonious Work Passion,Obsessive Work Passion,and Work Outcomes in Russia and China

Purpose

This cross-cultural study with employee–supervisor dyads in Russia and China examines links between harmonious and obsessive work passion and four job- and organization-focused outcomes (job satisfaction, intentions to quit, job performance, and organizational citizenship behaviors) and two career-focused outcomes (career satisfaction and occupational commitment).

Design/Methodology/Approach

Data were collected from employee–supervisor dyads in Russia (N = 223) and China (N = 193). We undertook a series of hierarchical regressions to examine the hypothesized relationships.

Findings

We found considerable support for the harmonious passion–work outcome relationships and less support for the obsessive passion–work outcome relationships. In both Russia and China, harmonious passion predicted all six hypothesized outcomes. However, obsessive work passion predicted job satisfaction and occupational commitment in Russia, but was unrelated to any of the hypothesized outcomes in China. We also identified several culture-specific work passion–outcome relationships.

Implications

Our research extends the duality of the work passion construct to non-Western cultures. The examination of a variety of work passion outcomes provides a finer-grained approach to how two types of passion uniquely link to different work consequences. Several culture-specific findings refute the traditionally held assumption that harmonious passion relates to solely positive outcomes, whereas obsessive passion relates to solely negative outcomes. Collectively, the results augment the nomological framework of the passion construct. The study informs managerial practices by suggesting when work passion needs to be encouraged or tamed.

Originality/Value

This is the first study that examines a variety of job-, organization-, and career-focused outcomes of work passion in non-Western organizations.

     

Got Resources? A Multi-Sample Constructive Replication of Perceived Resource Availability’s Role in Work Passion–Job Outcomes Relationships

Purpose

The purpose of this study was to investigate the moderating effect of perceived resource availability on the relationship between work passion and employee well-being (i.e., job satisfaction and job tension) and performance (i.e., job performance and citizenship behaviors) using self-determination theory.

Design/Methodology/Approach

Data were obtained through surveys distributed via an online platform (Sample 1) and to employees of three professional organizations: a municipal agency (Sample 2), an engineering firm (Sample 3), and an advertising organization (Sample 4).

Findings

The interaction between employees’ work passion and their perceptions of available resources was associated with employees’ well-being and performance, such that greater work passion was associated with positive outcomes when resources were perceived as available. Conversely, heightened work passion was associated with job tension and fewer positive benefits when perceived available resources were low.

Implications

Work passion is often touted by employers as a valuable characteristic for employees, but, as these findings suggest, there are conditions that must be met in order for employees to experience positive well-being and performance outcomes. This information will likely prove invaluable for those employers seeking to best support their passionate employees.

Originality/Value

Research into the area of work passion is small but growing, and this study provides valuable insight into a key boundary condition for the effectiveness of passion: perceived resource availability. Additionally, this study identifies circumstances in which passionate employees actually experience a negative work outcome. Further, the multiple samples and constructive replication employed help provide confidence and a strong empirical foundation for the results.

     

Relations of Interpersonal Unfairness with Counterproductive Work Behavior: The Moderating Role of Employee Self-Identity

Purpose

Following the job demands–resources model, this study investigated the role of self-identity, or how employees define themselves in relation to others, in the relations between interpersonal unfairness and counterproductive work behavior (CWB). Self-identity, an important self-regulatory and resource-related variable, was proposed to moderate the unfairness–CWB relations.

Methodology

A sample of 361 Chinese airline industry employees completed measures of identity, interpersonal unfairness, and CWB directed at people or the organization. We conducted a series of moderated regression analyses to test the hypotheses.

Findings

We found as expected relational identity (identity based on dyadic connection to another person) and collective identity (identity based on membership in a social group) buffered unfairness–CWB relations, such that positive relations were weaker when relational or collective identity was high (vs. low). Support was not found for the proposed moderation effect of individual identity (identity based on uniqueness from others) on unfairness–CWB relations. Supplementary relative weight analyses indicated that multiple identity levels and interactions between them accounted for considerable proportions of explained variance in CWB.

Implications

These findings suggest that different levels of employee identity seem to play different roles in the interpersonal unfairness–CWB relations, and it is important to continue studying employee identity profiles in the context of predicting CWB occurrences.

Originality/Value

This study not only advances our understanding of potential antecedents of CWB, but also underscores the importance of simultaneously studying all three levels of employee identity.     

Personal and Social Facets of Job Identity: A Person-Centered Approach

Purpose

The purpose of this study was to examine ego-identity (Erikson, Psychol Issues 1:1–171, 1959; Identity, youth and crisis, Norton, New York, 1968; Marcia, J Pers Soc Psychol 3:551–558, 1966) and social identity (Tajfel and Turner, In: Austin WG, Worchel S (Eds.) The social psychology of intergroup relations. Brooks/Cole, Monterey, pp 33–47 1979; Turner et al., Rediscovering the social group: A self-categorization theory. Blackwell, Oxford, 1987) theories within the organizational literature. We adopted a person-centered approach to analyze whether employees classified in various identity statuses and identification profiles exhibited differences in job outcomes (i.e., burnout, job satisfaction, and organizational citizenship behaviors). We also analyzed interconnections among identity statuses and identification profiles.

Design/methodology/approach

Participants were 515 employees (85.4 % women) between 24 and 64 years old. They completed self-reported questionnaires assessing personal identity, social identity, and job outcomes.

Findings

Cluster analysis indicated that participants could be classified into four identity statuses (i.e., achievement, early closure, moratorium, and searching moratorium) and into four identification profiles (i.e., orthogonal combinations of high vs. low organizational and group identification, respectively). Employees classified in the various identity statuses and identification profiles reported meaningful differences on job outcomes. Further, findings highlighted significant associations between identity statuses and identification profiles, giving rise to various identity configurations associated with job outcomes.

Implications

This study highlights the importance of integrating different facets of job identity. These findings have relevant implications in terms of suggesting which dimensions of identity should be promoted in order to reduce workers’ burnout, and enhance their satisfaction and organizational citizenship behaviors.

Originality/value

This study provides evidence for integrating ego-identity and social identity theories. In doing so, it bridges developmental psychology literature on personal identity with social and organizational psychology literature on social identity, setting the basis for a comprehensive line of research.     

Predictors of Occupational and Organizational Commitment in Information Technology: Exploring Gender Differences and Similarities

Purpose

Despite growing demand, the information technology (IT) field suffers from a labor shortage compounded by the underrepresentation of women in the IT field. This study examined predictors of occupational and organizational commitment outcomes among IT professionals and explored gender differences and similarities in the relative importance of predictors.

Design/Methodology/Approach

1,229 IT professionals provided web-based survey data, which were analyzed using hierarchical linear modeling and relative weight analysis.

Findings

Satisfaction with growth opportunities, job security, job stress, and work–family culture generally were related to commitment outcomes as expected, accounting for the greatest variance in organizational commitment. Relative weights for men and women differed in the prediction of occupational commitment, where growth satisfaction and work–family culture were weighted more strongly by women and job stress was more strongly weighted by men. For organizational commitment, the relative weights among predictors were similar for men and women, except that men weighted job security more strongly than women.

Implications

Findings suggest that garnering commitment to the IT field requires a gender-specific approach, highlighting growth and work–family support for women and addressing job stress for men. Increasing organizational commitment for both men and women calls for an emphasis on opportunities for growth and development.

Originality/Value

This is the first study to simultaneously examine predictors for organizational and occupational commitment among IT professionals and to assess the relative importance of predictors discussed in the extant literature. Results inform efforts to increase retention in the IT field and provide guidance for improving the representation of women in IT.     

Transformational and Transactional Leadership and Followers’ Achievement Goals

Purpose

Achievement goals, or the standards of competence employees pursue in their work, have far-reaching consequences for employee and organizational functioning. In the current research, we investigated whether employees’ achievement goals can be predicted from their supervisor’s leadership style.

Design/Methodology/Approach

A multilevel study was conducted in which followers of 120 organizational leaders completed measures of their leader’s transformational leadership (focusing on individual needs and abilities, on intellectual development, and on a common team mission), transactional leadership (focusing on monitoring and achievement-related rewards), and their own mastery goals (aimed at learning, developing, and mastering job-relevant skills), and performance goals (aimed at doing better than others).

Findings

Group-level transformational leadership predicted followers’ mastery goals, whereas group-level transactional leadership predicted followers’ performance goals. Within-group differences in transformational leadership also predicted mastery goals.

Implications

These findings suggest that leadership style plays an important role in the achievement goals followers adopt. Organizations may promote transactional leadership in contexts requiring that employees outperform others. In contrast, in contexts requiring learning and development, organizations may promote transformational leadership.

Originality/Value

This research is the first to examine the relationships between leadership styles and specific follower goals, and the first to highlight the role of leadership as a social variable involved in employees’ adoption of achievement goals.     

Generational Differences in Work-Related Attitudes: A Meta-analysis

Purpose

Differences among generations on a wide variety of outcomes are of increasing interest to organizations, practitioners, and researchers alike. The goal of this study was to quantitatively assess the research on generational differences in work-related attitudes and to provide guidance for future research and practice.

Design/Methodology/Approach

We conducted a meta-analysis of generational differences on three work-related criteria: job satisfaction, organizational commitment, and intent to turnover. Our review of published and unpublished research found 20 studies allowing for 18 generational pairwise comparisons across four generations (Traditionals, Baby Boomers, Generation Xers, and Millennials) on these outcomes using 19,961 total subjects.

Findings

Corrected mean differences for job satisfaction ranged from .02 to .25, for organizational commitment they ranged from ?.22 to .46, and for intent to turnover the range was ?.62 to .05. The pattern of results indicates that the relationships between generational membership and work-related outcomes are moderate to small, essentially zero in many cases.

Implications

The findings suggest that meaningful differences among generations probably do not exist on the work-related variables we examined and that the differences that appear to exist are likely attributable to factors other than generational membership. Given these results, targeted organizational interventions addressing generational differences may not be effective.

Originality/Value

This is the first known quantitative review of research on generational differences in the workplace.     

Relationships Between Leader-Contingent and Non-contingent Reward and Punishment Behaviors and Subordinates’ Perceptions of Justice and Satisfaction, and Evaluation of the Moderating Influence of Trust Propensity, Pay Level, and Role Ambiguity

Purpose

This research evaluates how reward and punishment behaviors are related to employee satisfaction, and how perceptions of justice mediate the effects of these behaviors. This study also seeks to examine the connection between some boundary conditions and the efficacy of reward and punishment behaviors.

Design/Methodology/Approach

Data were collected using questionnaires, and hypotheses were tested using a sample of 3,065 managers from 41 business units.

Findings

We found that contingent financial and social rewards were positively related to fairness and satisfaction, whereas punishment behaviors exert a negative influence. Findings reveal that high-trust propensity and low-pay level strengthen the positive influence of rewards on distributive justice, whereas high role ambiguity increases the positive effect of reward behaviors and the negative influence of non-contingent punishment (NCP) on superior satisfaction.

Implications

This study provided evidence that using contingent rewards is more effective in promoting positive attitudes at work than using contingent punishments. Our findings add to the evidence that organizations would benefit from encouraging their managers to use financial and social rewards and reducing the administration of NCP among performing employees.

Originality/Value

This study represents a first attempt to examine an integrated model of the effects of reward and punishment behaviors. It also contributes to this field by empirically evaluating the impact of individual and situational boundary conditions in the efficacy of these leader behaviors.     

How Today’s Shocks Predict Tomorrow’s Leaving

Purpose

This research explores “shocking events” as part of the unfolding model of turnover, extending our understanding of the influence of various types of shocks on future voluntary employee separations.

Design/Methodology/Approach

1536 new hires at a large financial institution reported shocks monthly during their first 8 months at work as well as their job satisfaction and perceptions of marketability. We used event history to estimate the basic distributional properties of the shocks and Cox proportional hazards models to examine the effects of shocks on job satisfaction and turnover over the subsequent year as reported by the organization.

Findings

Organizational shocks generally occur earlier than personal shocks. Further, unexpected shocks have a stronger impact than expected shocks on subsequent turnover. Finally, the effects of organizational shocks on turnover are mediated by job satisfaction, whereas personal shocks have direct effects on turnover.

Implications

Our findings offer evidence for the utility of the shock construct in the unfolding model of turnover and speak to the importance of encouraging managers to monitor shocks on an ongoing basis in order to predict when different types of shocks will occur and their likely influence on turnover.

Originality/Value

Ours is the first study to examine shocks as they occur. This is a contrast to prior studies that relied on retrospective accounts. Thus, we are able to test new hypotheses (e.g., direct effects vs. mediation) that expand the unfolding model of turnover.

     

Reactions to Recruitment Web Sites: Visual and Verbal Attention, Attraction, and Intentions to Pursue Employment

Purpose

We investigate job seeker visual and verbal attention, and perceptions regarding company web sites in the applicant generation phase of recruitment.

Design/Methodology/Approach

We report three studies using varied methodological approaches including eye-tracking, verbal protocol analysis (VPA), and survey data.

Findings

Eye-tracking results suggest Web-based job seekers focus visual attention on information containing hyperlinks and on text more than graphic images or navigation tools. VPA suggests Web-based job seekers focus verbal attention on content more than design, especially job opening information. Survey results suggest content, design, and communication features are all related to applicant attraction. Design explains the most incremental variance in web site evaluation, while perceptions of communication features explain the most incremental variance in attitude toward the organization and intentions to pursue employment.

Implications

We provide multi-method evidence concerning features that attract applicant visual and verbal attention, and influence attraction and intentions to pursue employment. The findings also provide practical implications for designing recruitment web sites.

Originality/Value

We report three methodologically distinct perspectives on an important and timely issue: Web-based recruitment. We are aware of no other psychomotor eye-tracking studies in the recruitment literature, and only one other VPA. Combining multiple methods in this way provides unique perspective.     

Estimating Group-Level Relationships: General Recommendations and Considerations for the Use of Intraclass Correlation Coefficients

Purpose

In this investigation, we argue for why and how available intraclass correlation coefficients and other types of reliability estimates can be employed as sample-based reliability estimates within primary and meta-analytic studies when relationships between group-level phenomena are of interest.

Design/Methodology/Approach

Group-level correlations and reliability estimates were obtained from 46 studies examining organizational climate–performance relationships. We illustrate how the group-level reliability estimates can be used to correct correlations for predictor and criterion unreliability. Procedures are presented for computing the sampling variances of individually corrected correlations that account for sampling error in the group-level reliability estimates.

Findings

Support was found for the conservative nature of meta-analytic parameter estimates when group-level reliability information is sample-based as opposed to assumed population values. In addition, our analyses indicated that conclusions about substantive relationships between group-level variables can change based on availability of sample-based reliabilities within both primary and meta-analytic studies.

Implications

Results from this study suggest that researchers should rely on sample-based meta-analytic procedures when examining the generalizability of group-level relationships. This study also demonstrates the importance of using all available reliability information and accounting for sampling error in the reliability estimates when conducting meta-analyses at the group level of analysis.

Originality/Value

This study breaks ground by systematically examining the use of intraclass correlation coefficients as reliability estimates within group-level meta-analytic studies. Furthermore, illustrative analyses provide guidance to primary and meta-analytic researchers in regard to how to correct group-level correlations for unreliability in the predictor, criterion, or both whenever and in whatever proportions the artifact information is available.