Studying group and time invariance in maximal reliability for multiple‐component measuring instruments via covariance structure modelling

A method for examining invariance in maximal reliability for weighted combinations of congeneric measures is described. The approach is developed within the framework of covariance structure modelling and allows one to ascertain whether a multi‐component instrument consisting of homogeneous measures is associated with the same minimal relative error variance in distinct populations or over time. The procedure yields as a by‐product an interval measure of discrepancy in maximal reliability across independent groups or assessment occasions, and is illustrated with two examples.     

Estimation of maximal reliability: A note on a covariance structure modelling approach

A one‐step covariance structure analysis procedure for estimation of maximal reliability of linear composites with congeneric measures is outlined. The approach is readily employed within a single modelling session using popular covariance structure analysis software, and permits simultaneous estimation of the optimal measure weights with standard errors. The method is illustrated by a numerical example.     

Incorporating prior theory in covariance structure analysis: A bayesian approach

A Bayesian approach to the testing of competing covariance structures is developed. The method provides approximate posterior probablities for each model under consideration without prior specification of individual parameter distributions. The method is based on ayesian updating using cross-validated pseudo-likelihoods. Given that the observed variables are the samefor all competing models, the approximate posterior probabilities may be obtained easily from the chi square values and other known constants, using only a hand calculator. The approach is illustrated using and example which illustrates how the prior probabilities can alter the results concerning which model specification is preferred.     

Alternative test criteria in covariance structure analysis: A unified approach

In the context of covariance structure analysis, a unified approach to the asymptotic theory of alternative test criteria for testing parametric restrictions is provided. The discussion develops within a general framework that distinguishes whether or not the fitting function is asymptotically optimal, and allows the null and alternative hypothesis to be only approximations of the true model. Also, the equivalent of the information matrix, and the asymptotic covariance matrix of the vector of summary statistics, are allowed to be singular. When the fitting function is not asymptotically optimal, test statistics which have asymptotically a chi-square distribution are developed as a natural generalization of more classical ones. Issues relevant for power analysis, and the asymptotic theory of a testing related statistic, are also investigated.This research has been supported by the U.S.-Spanish Joint Committee for Cultural and Educational Cooperation, grant number V-B.854020. The author wishes to express his gratitude to P. M. Bentler who provided very helpful suggestions and research facilities—with an stimulating working environment—at the University of California, Los Angeles, where this work was undertaken. Thanks are also due to W. E. Saris who provided very valuable comments to earlier versions of this paper. Finally, it has also to be acknowledged the editor's and reviewers suggestions which led to substantial improvements of this article.     

Bootstrap approach to inference and power analysis based on three test statistics for covariance structure models

We study several aspects of bootstrap inference for covariance structure models based on three test statistics, including Type I error, power and sample‐size determination. Specifically, we discuss conditions for a test statistic to achieve a more accurate level of Type I error, both in theory and in practice. Details on power analysis and sample‐size determination are given. For data sets with heavy tails, we propose applying a bootstrap methodology to a transformed sample by a downweighting procedure. One of the key conditions for safe bootstrap inference is generally satisfied by the transformed sample but may not be satisfied by the original sample with heavy tails. Several data sets illustrate that, by combining downweighting and bootstrapping, a researcher may find a nearly optimal procedure for evaluating various aspects of covariance structure models. A rule for handling non‐convergence problems in bootstrap replications is proposed.     

Examining change in maximal reliability for multiple‐component measuring instruments

A method for examining change in maximal reliability for pre‐specified sets of congeneric measures when developing a multi‐component instrument is outlined. The approach is applicable for purposes of estimation and testing of gain or loss in the maximal reliability coefficient as a consequence of adding or dropping one or more measures from a homogeneous composite with uncorrelated errors, as well as when one is concerned with optimal component choice for highest increase or correspondingly smallest drop in maximal reliability. The method is compared with a procedure for ascertaining change in unweighted sum score reliability, and implications for instrument construction and revision are discussed. The approach is illustrated with a numerical example.     

Confidence in work‐related goals and feelings of exhaustion during a therapeutic intervention for burnout: A time‐series approach

This study investigated recursive relations between confidence in achieving work‐related goals and work exhaustion among employees who participated in an intervention to reduce their burnout. Thirty‐six employees of age 33–59 years suffering from severe burnout (28 females and 8 males) filled in burnout and well‐being measures before and after a 10‐month therapeutic intervention. They also filled in weekly measures of confidence in work‐related goals (progress and capability) and work exhaustion throughout the intervention, as well as 4 weeks before and 4 weeks afterwards. Intra‐individual variation was modelled using dynamic factor analyses. The results showed that, for most participants, confidence in work‐related goals and work exhaustion during a given week could be predicted from those of the preceding week. Moreover, high self‐esteem predicted low weekly stability in goal confidence, whereas high burnout contributed to high weekly stability in work exhaustion. The intra‐individual stabilities in goal confidence and exhaustion also predicted which of the participants benefited from the intervention in terms of increasing well‐being and decreasing work‐related stress.     

Memory for targets in a multi‐level simulated‐environment: A comparison between able‐bodied and physically disabled children

Two groups of children, one able‐bodied and the other with physical disabilities, explored a symmetrical three‐tiered virtual building that contained six distinctive target objects, two on each storey. In a subsequent test, the target objects were removed and participants were asked to make judgments of the directions to the former target locations from each floor in turn. At each test site, judgments were required for targets that were formerly on the same floor and for those on higher and lower floors. Relative tilt error scores suggested a bias for both groups, in that targets that were higher than the test location were judged as consistently lower than their actual position, whereas targets that were lower than the test location were judged as higher than their actual position. Absolute tilt errors revealed an asymmetry in both groups, with more accurate tilt errors for judgments directed to lower than higher floors. The relevance of these results for the source of the asymmetry is discussed.     

Personality and religious values among adolescents: A three‐wave longitudinal analysis

Using three waves of data, we assessed the relationships between endorsement of religious values, some of the major personality dimensions, and social and emotional well‐being amongst teenagers. Participants were 784 high school students at Time 1 (382 males and 394 females; 8 did not indicate their gender) and 563 provided data at each of Time 1, Time 2 and Time 3. We examined the impact of changes in (Eysenckian) psychoticism and conscientiousness from Time 1 to Time 2 on religious values assessed at Time 3. Both personality and personality change predicted religious values and the specific effects depended on gender. Participants higher in hope, joviality, psychological acceptance and mindfulness also tended to be higher in religious values. The implications of these results for adolescent well‐being and resilience are discussed.     

Proportion of third‐level variation in multi‐level studies: A note on an interval estimation procedure

An interval estimation procedure is outlined that can be used for evaluating the proportion of observed variance in a response variable, which is due to the third level of nesting in a hierarchical design. The approach is also useful when it is of concern to address the necessity of including a third level in analyses of data from a multi‐level study, relative to an alternative of proceeding with two‐level modelling. The proposed method is illustrated with an empirical example.     

How the 2SLS/IV estimator can handle equality constraints in structural equation models: A system‐of‐equations approach

Parameters in structural equation models are typically estimated using the maximum likelihood (ML) approach. Bollen (1996) proposed an alternative non‐iterative, equation‐by‐equation estimator that uses instrumental variables. Although this two‐stage least squares/instrumental variables (2SLS/IV) estimator has good statistical properties, one problem with its application is that parameter equality constraints cannot be imposed. This paper presents a mathematical solution to this problem that is based on an extension of the 2SLS/IV approach to a system of equations. We present an example in which our approach was used to examine strong longitudinal measurement invariance. We also investigated the new approach in a simulation study that compared it with ML in the examination of the equality of two latent regression coefficients and strong measurement invariance. Overall, the results show that the suggested approach is a useful extension of the original 2SLS/IV estimator and allows for the effective handling of equality constraints in structural equation models.     

Predictors of young dating adults' inclination to engage in extradyadic sexual activities: A multi‐perspective study

Using a multi‐perspective vignette design, we explored predictors of young peoples' (N=119) propensity to engage in unfaithful activities while dating. Demographic measures, a datding investment model, and measures of functional and dysfunctional impulsivity were used to predict inclination to engage in each of two extradyadic activities (kissing and sexual activity). The results of moderated multiple regression analyses revealed that a respondent's number of sexual partners, level of dysfunctional impulsivity, satisfaction with current relationship, and quality of relationship alternatives significantly predicted inclination to engage in both of the extradyadic activities. Consistent with previous findings, gender only showed significant predictive value in relation to extradyadic sex inclination. Moreover, the association between sex, love, and marriage interacted with gender in the prediction of both extradyadic activities and interacted with commitment in the prediction of extradyadic sex inclination. Suggestions for future research in this area are offered in light of these new findings.     

How to study the ‘quality of psychoanalytic treatments’ and their long‐term effects on patients' well‐being: A representative,multi‐perspective follow‐up study3

How can we study the ‘quality of psychoanalytic treatments’? The authors attempt to answer this question by discussing a naturalistic, multi‐perspective and representative follow‐up study of psychoanalyses and long‐term psychoanalytic psychotherapies. We studied a representative sample (n = 401) of all the patients who had terminated their psychoanalytic treatments with members of the German Psychoanalytical Association (DPV) between 1990 and 1993. Between 70 and 80 per cent of the patients achieved (average 6.5 years after the end of treatment) good and stable psychic changes according to the evaluations of the patients themselves, their analysts, independent psychoanalytic and non‐psychoanalytic experts, and questionnaires commonly applied in psychotherapy research. The evaluation of mental health costs showed a cost reduction through fewer days of sick leave during the seven years following the end of long‐term psychoanalytic treatments. The results achieved using non‐psychoanalytical instruments are complemented by the richness of the idiosyncratic findings, gained by the psychoanalytic research instruments.     

The effects of service climate and the effective leadership behaviour of supervisors on frontline employee service quality: A multi‐level analysis

A supervisor's behaviour may not be the only factor that determines the performance of team members ( Kerr & Jermier, 1978 ). Taking this postulation as a basis, we formulated a model to describe how service climate moderates the effects of the leadership behaviour of supervisors. When the organization and working environment are not conducive to providing a good service to colleagues and customers, the supervisor's leadership behaviour makes an important difference. However, when the service climate is good, a supervisor's leadership behaviour makes no substantial difference. This hypothesis was supported in an examination of the service quality of 511 frontline service providers as sampled from 55 work groups in 6 service organizations. The employee service quality was low when both the service climate and the supervisor's leadership behaviour were lacking. However, when the service climate was unfavourable, effective leadership behaviour played a compensatory role in maintaining performance standards towards external customers. When the leadership was ineffective, a favourable service climate nullified the negative effect on service quality to internal customers.     

Getting the most out of multidisciplinary teams: A multi‐sample study of team innovation in health care

Driven by the assumption that multidisciplinarity contributes positively to team outcomes teams are often deliberately staffed such that they comprise multiple disciplines. However, the diversity literature suggests that multidisciplinarity may not always benefit a team. This study departs from the notion of a linear, positive effect of multidisciplinarity and tests its contingency on the quality of team processes. It was assumed that multidisciplinarity only contributes to team outcomes if the quality of team processes is high. This hypothesis was tested in two independent samples of health care workers (N = 66 and N = 95 teams), using team innovation as the outcome variable. Results support the hypothesis for the quality of innovation, rather than the number of innovations introduced by the teams.     

The team‐level model of climate for innovation: A two‐level confirmatory factor analysis

The level structure of West's (1990) four‐factor model of team climate for innovation was assessed by means of multi‐level confirmatory factor analysis (MCFA). The sample consisted of 1,487 individuals (195 teams) from a wide range of professions. Results showed that a considerable portion of the variance in the data was explained on the team level with intra‐class correlations ranging from .30 to .39. Furthermore, the results demonstrated that the overall measurement model fitted the data well at both the team and individual levels, while the factor loadings were slightly different across the levels with item loadings showing partial invariance. Results from confirmatory factor analyses conducted on separate levels, however, showed that the four‐factor model displayed the best fit to the data for both individual and team levels. A second‐order one‐factor model also fitted the data well on both levels. The results indicate that the team climate for innovation model can be used as a team‐level consensus model of team climate for innovation.     

Deliberate practice predicts performance over time in adolescent chess players and drop‐outs: A linear mixed models analysis

In this study, the longitudinal relation between deliberate practice and performance in chess was examined using a linear mixed models analysis. The practice activities and performance ratings of young elite chess players, who were either in, or had dropped out of the Dutch national chess training, were analysed since they had started playing chess seriously. The results revealed that deliberate practice (i.e. serious chess study alone and serious chess play) strongly contributed to chess performance. The influence of deliberate practice was not only observable in current performance, but also over chess players' careers. Moreover, although the drop‐outs' chess ratings developed more slowly over time, both the persistent and drop‐out chess players benefited to the same extent from investments in deliberate practice. Finally, the effect of gender on chess performance proved to be much smaller than the effect of deliberate practice. This study provides longitudinal support for the monotonic benefits assumption of deliberate practice, by showing that over chess players' careers, deliberate practice has a significant effect on performance, and to the same extent for chess players of different ultimate performance levels. The results of this study are not in line with critique raised against the deliberate practice theory that the factors deliberate practice and talent could be confounded.