Studying group and time invariance in maximal reliability for multiple‐component measuring instruments via covariance structure modelling

A method for examining invariance in maximal reliability for weighted combinations of congeneric measures is described. The approach is developed within the framework of covariance structure modelling and allows one to ascertain whether a multi‐component instrument consisting of homogeneous measures is associated with the same minimal relative error variance in distinct populations or over time. The procedure yields as a by‐product an interval measure of discrepancy in maximal reliability across independent groups or assessment occasions, and is illustrated with two examples.     

Reliability if deleted,not ‘alpha if deleted’: Evaluation of scale reliability following component deletion

This paper demonstrates that the widely available and routinely used index ‘coefficient alpha if item deleted’ can be misleading in the process of construction and revision of multiple‐component instruments with congeneric measures. An alternative approach to evaluation of scale reliability following deletion of each component in a given composite is outlined that can be recommended in general for scale development purposes. The method provides ranges of plausible values for instrument reliability when dispensing with single components in a tentative composite, and permits testing hypotheses about reliability of resulting scale versions. The proposed procedure is illustrated with an example.     

Estimation of maximal reliability: A note on a covariance structure modelling approach

A one‐step covariance structure analysis procedure for estimation of maximal reliability of linear composites with congeneric measures is outlined. The approach is readily employed within a single modelling session using popular covariance structure analysis software, and permits simultaneous estimation of the optimal measure weights with standard errors. The method is illustrated by a numerical example.     

Estimation of maximal reliability for multiple‐component instruments in multilevel designs

A procedure for point and interval estimation of maximal reliability of multiple‐component measuring instruments in multi‐level settings is outlined. The approach is applicable to hierarchical designs in which individuals are nested within higher‐order units and exhibit possibly related performance on components of a given homogeneous scale. The method is developed within the framework of multi‐level factor analysis. The proposed procedure is illustrated with an empirical example.     

Proportion of general factor variance in a hierarchical multiple‐component measuring instrument: A note on a confidence interval estimation procedure

A confidence interval construction procedure for the proportion of explained variance by a hierarchical, general factor in a multi‐component measuring instrument is outlined. The method provides point and interval estimates for the proportion of total scale score variance that is accounted for by the general factor, which could be viewed as common to all components. The approach may also be used for testing composite (one‐tailed) or simple hypotheses about this proportion, and is illustrated with a pair of examples.     

Defining and Assessing Problem‐Solving Style: Design and Development of a New Tool

VIEW: An Assessment of Problem Solving Style (Selby, Treffinger, & Isaksen, 2002) is a new instrument for assessing problem‐solving style, for use with individuals from ages 12 through adult. It measures three dimensions of style relating to creative problem solving and change management. In this article, we discuss the construction of the instrument, the initial evidence supporting the instrument's reliability and validity, and a very brief overview of the instrument's foundations. Our reliability data involve both stability and internal consistency. We report evidence for the criterion‐related validity, based on correlational studies with relevant measures of learning style, cognitive style, and psychological type. We also conducted principal components factor analyses that support our three‐factor structure. Researchers and practitioners studying and applying Creative Problem Solving and change management methods can use VIEW in several ways. Finally, we identify several research directions that will contribute to the refinement and development of the instrument as well as to a better understanding of the “problem‐solving style” construct.     

A method for testing group differences of scale validity in multiple population studies

A method for testing equality in validity of multi‐component measuring instruments across populations is outlined. The approach is developed within the framework of covariance structure modelling and complements earlier research on examining group differences in scale reliability. The procedure is particularly useful for purposes of ascertaining comparability of validity when constructing and developing measuring instruments. The method also provides ranges of plausible values for differences in composite validity across several populations and allows one to evaluate group discrepancies in validity of behavioural scales. The approach is illustrated using data from a cognitive intervention study.     

Alpha if item deleted: A note on loss of criterion validity in scale development if maximizing coefficient alpha

This note is concerned with a validity‐related limitation of the widely available and routinely used index ‘alpha if item deleted’ in the process of construction and development of multiple‐component measuring instruments. Attention is drawn to the fact that this statistic can suggest dispensing with such scale components, whose removal leads to loss of criterion validity while maximizing the popular coefficient alpha. As an alternative, a latent variable modelling approach is discussed that can be used for point and interval estimation of composite criterion validity (as well as reliability) after deletion of single components. The method can also be utilized to test conventional or minimum level hypotheses about associated population change in measurement quality indices.     

A Method for Analyzing Videotaped Genetic Counseling Sessions

This study describes the development and evaluation of a multi-item scale for analyzing the genetic counseling process, the Manchester Observation Code (MOC) for genetic counseling. The instrument is specific to the field of genetic counseling and is designed for analysis of the communication between counselor and client. Coding is done directly from videotaped sessions. Because communication is the means by which genetic counseling is accomplished, the method measures four relevant components of communication: (1) grammatical form, (2) purpose, (3) subject, and (4) cue source. The instrument enables an observer to code the counselor's statements into these four components. Three videotaped sessions were used to measure interrater reliability, or the consistency of rating for each of the four communication domains using this method. Three videotaped sessions were also used to measure test-retest reliability, or the consistency of the designed method from one time to another. A total of 21 videotaped sessions were tested using the method. A statistical measure of reliability established consistency of the designed method; Cohen's kappa yielded 0.7 for interrater reliability and 0.79 for test-retest reliability. These findings suggest this instrument may be used to identify important elements of the genetic counseling process.     

Reliability of multiple‐component measuring instruments: Improved evaluation in repeated measure designs

A covariance structure analysis method for improved point and interval estimation of composite reliability in repeated measure designs is outlined that accounts for specificity variance. The approach also permits the testing of time‐invariance in reliability of multiple‐component instruments in terms of the ratio of ‘pure’ measurement error variance to observed scale score variance. In addition, the procedure allows interval estimation of the difference in composite reliability coefficients across assessment occasions. The method described is illustrated with data from a cognitive intervention study.     

Response–reinforcer dependency and resistance to change

The effects of the response–reinforcer dependency on resistance to change were studied in three experiments with rats. In Experiment 1, lever pressing produced reinforcers at similar rates after variable interreinforcer intervals in each component of a two‐component multiple schedule. Across conditions, in the fixed component, all reinforcers were response‐dependent; in the alternative component, the percentage of response‐dependent reinforcers was 100, 50 (i.e., 50% response‐dependent and 50% response‐independent) or 10% (i.e., 10% response‐dependent and 90% response‐independent). Resistance to extinction was greater in the alternative than in the fixed component when the dependency in the former was 10%, but was similar between components when this dependency was 100 or 50%. In Experiment 2, a three‐component multiple schedule was used. The dependency was 100% in one component and 10% in the other two. The 10% components differed on how reinforcers were programmed. In one component, as in Experiment 1, a reinforcer had to be collected before the scheduling of other response‐dependent or independent reinforcers. In the other component, response‐dependent and ‐independent reinforcers were programmed by superimposing a variable‐time schedule on an independent variable‐interval schedule. Regardless of the procedure used to program the dependency, resistance to extinction was greater in the 10% components than in the 100% component. These results were replicated in Experiment 3 in which, instead of extinction, VT schedules replaced the baseline schedules in each multiple‐schedule component during the test. We argue that the relative change in dependency from Baseline to Test, which is greater when baseline dependencies are high rather than low, could account for the differential resistance to change in the present experiments. The inconsistencies in results across the present and previous experiments suggest that the effects of dependency on resistance to change are not well understood. Additional systematic analyses are important to further understand the effects of the response–reinforcer relation on resistance to change and to the development of a more comprehensive theory of behavioral persistence.     

Group Session Rating Scale: Preliminary psychometrics in substance abuse group interventions

Background: Although ultra‐brief outcome and process measures have been developed for individual therapy, currently there are no ultra‐brief alliance measures for group therapy. Method: The current study examined 105 clients in group therapy for issues related to substance abuse or with issues related to the substance abuse of a significant other. We tested whether a newly developed group therapy alliance measure – the Group Session Rating Scale would be related to other commonly used group process measures (Working Alliance Inventory, Group Cohesion, Group Climate) and early change (change over the first four sessions of group therapy). Results: The findings provided support for reliability based on Cronbach alphas and test‐retest coefficients. Additionally, the GSRS was a one‐factor measure that was related to other group process measures as well as predicted early change. Discussion: Clinical implications for how to utilise ultra‐brief outcome and alliance measures are provided.     

Maximal reliability and power in covariance structure models

In covariance structure modelling, the non‐centrality parameter of the asymptotic chi‐squared distribution is typically used as an indicator of asymptotic power for hypothesis tests. When a latent linear regression is of interest, the contribution to power by the maximal reliability coefficient, which is associated with used latent variable indicators, is examined and this relationship is further explicated in the case of congeneric measures. It is also shown that item parcelling may reduce power of tests of latent regression parameters. Recommendations on weights for parcelling to avoid power loss are provided, which are found to be those of optimal linear composites with maximal reliability.     

A Direct Method for Obtaining Approximate Standard Error and Confidence Interval of Maximal Reliability for Composites With Congeneric Measures

Unlike a substantial part of reliability literature in the past, this article is concerned with weighted combinations of a given set of congeneric measures with uncorrelated errors. The relationship between maximal coefficient alpha and maximal reliability for such composites is initially dealt with, and it is shown that the former is a lower bound of the latter. A direct method for obtaining approximate standard error and confidence interval for maximal reliability is then outlined. The procedure is based on a second-order Taylor series approximation and is readily and widely applicable in empirical research via use of covariance structure modeling. The described method is illustrated with a numerical example.     

The Score Family Assessment Questionnaire: A Decade of Progress

This paper reviews a decade of research (2006–2016) on a family assessment instrument called the Systemic Clinical Outcome and Routine Evaluation (SCORE). The SCORE was developed in Europe to monitor progress and outcome in systemic therapy and has been adopted by the European Family Therapy Association as the main instrument for assessing the outcome in systemic family and couple therapy. There are currently six main versions of this instrument: SCORE‐40, SCORE‐15, SCORE‐28, SCORE‐29, Child SCORE‐15, and Relational SCORE‐15. It has also been translated into a number of European languages. Fifteen empirical studies of the SCORE “family of measures” have been conducted. Most have aimed to establish psychometric properties of these instruments in English and other languages. Others have used the SCORE to document the level of family adjustment in clinical samples or evaluate outcome in treatment trials. There is now sufficient evidence for the reliability and validity of the SCORE to justify the use of brief versions of this instrument to monitor progress and outcome in the routine practice of systemic therapy.     

Preliminary Validation of a Korean Version of the Acquired Capability for Suicide Scale‐Fearlessness About Death

The interpersonal‐psychological theory of suicide proposes that the desire for suicide must be accompanied by the capability to do so in order for an individual to engage in suicidal behavior. The Acquired Capability for Suicide Scale‐Fearlessness about Death (ACSS ‐FAD ) measures fearlessness about death, a core component of the capability for suicide. This study aimed to validate a Korean version of the ACSS ‐FAD in a college student sample. We administered the ACSS ‐FAD and measures of suicide ideation, fear of suicide, death anxiety, pain anxiety, and depression in a sample of Korean college students (N  = 301) and analyzed its reliability, factor structure, invariance across genders, convergent validity, and discriminant validity. The one‐factor model achieved satisfactory model fit. Across genders, partial metric invariance and partial scalar invariance were established. The results also revealed that the ACSS ‐FAD has good internal consistency, convergent validity (positive correlations with fear of suicide, death anxiety, and pain anxiety), and discriminant validity (no relation with depression). The Korean version of the ACSS ‐FAD presents adequate psychometric properties and may be considered to be a promising instrument for measuring fearlessness about death in college students.     

Qualitatively driven mixed‐methods approaches to counselling and psychotherapy research

Many researchers and clinicians take an ‘either, or’ position regarding factors responsible for change when conducting research. Qualitatively driven mixed methods privilege the qualitative approach and offer the opportunity to generate multi‐dimensional material, permitting a more holistic insight into experiences that can be understood from a combination of epistemological and ontological stances. A qualitatively driven mixed‐methods approach apply a ‘both, and’ position when exploring the elements that produce change or that are under investigation, which can be of particular value to counselling and psychotherapy research. There are various ways of engaging with qualitatively driven mixed methods. Some designs include both qualitative and quantitative components, where the former is the core element. The secondary component may also be qualitative, known as a multimethod design. Yet, other designs mix different qualitative approaches, through the application of different qualitative analyses to the same data (pluralistic qualitative research). This paper discusses the application and value of qualitatively driven mixed methods in counselling and psychotherapy research through the presentation of two research case studies; one which mixes both qualitative and quantitative components to investigate the experiences of pain alleviation following a CBT pain management programme; and one which applies a pluralistic approach to a counselling psychology doctorate exploring the experiences and meanings attached to self‐harming. Through illustration and discussion of the case studies’ mixed‐methods approaches, this paper demonstrates that qualitatively driven mixed methods produce a more enhanced and holistic understanding into phenomena, and therefore a more balanced perspective to counselling and psychotherapy research.     

The psychometric properties of The Religious Status Inventory—‘Being Ethical’ subscale among Northern Irish adolescents

There are few self‐report measures of morality. The Religious Status Inventory—‘Being Ethical’ subscale represents one approach. However, at present there is limited information on the psychometric properties of either the original 20‐item version (RSInv‐20) or the shortened embedded 10‐item version (RSInv‐S10). The aim of the present study was to provide psychometric data on the internal reliability of these two versions of the ‘Being Ethical’ subscale. As part of a larger study, 595 Northern Irish adolescents, drawn from both Grammar and Secondary schools, completed the RSInv‐20. An unsatisfactory level of internal reliability was found for the RSInv‐20 (Cronbach’s alpha = 0.42), but a satisfactory level of internal reliability was found for the RSInv‐S10 (Cronbach’s alpha = 0.70). Subsequent item analysis produced an alternative 10‐item version (RSInv‐A10) that provided the optimum level of internal reliability for a 10‐item measure in the present sample (Cronbach’s alpha = 0.74). In addition, on all three versions of the measure (RSInv‐20, RSInv‐S10, and RSInv‐A10), differences were found in levels of internal reliability among Grammar and Secondary school respondents, with the former producing higher levels of internal reliability.     

The test-retest reliability of the Form 90-DWI: an instrument for assessing intoxicated driving.

Although driving while intoxicated (DWI) is a pervasive problem, reliable measures of this behavior have been elusive. In the present study, the Form 90, a widely utilized alcohol and substance use instrument, was adapted for measurement of DWI and related behaviors. Levels of reliability for the adapted instrument, the Form 90-DWI, were tested among a university sample of 60 undergraduate students who had consumed alcohol during the past 90 days. The authors administered the instrument once during an intake interview and again, 7-30 days later, to determine levels of test-retest reliability. Overall, the Form 90-DWI demonstrated high levels of reliability for many general drinking and DWI behaviors. Levels of reliability were lower for riding with an intoxicated driver and for variables involving several behavioral conjunctions, such as seat belt use and the presence of passengers when driving with a blood alcohol concentration above .08. Overall, the Form 90-DWI shows promise as a reliable measure of DWI behavior in research on treatment outcome and prevention.     

Effects of Dohsa‐hou relaxation on body awareness and psychological distress1

Relaxation is one of the most frequently used methods for treating psychological and physical problems. Many relaxation methods are effective treatments for various psychological problems. Body awareness could be an important component of body‐oriented psychological treatment. In this study, a self‐report measure for the assessment of body awareness was developed, and its psychometric characteristics and relationships with other psychological constructs were examined. The effect of Dohsa‐hou relaxation was investigated using the developed Body Awareness Scale (BAS). The results showed that the BAS had good internal consistency and test‐retest reliability, and that it was associated with related psychological constructs. Dohsa‐hou relaxation led to increased body awareness and decreased psychological distress. The BAS can be a useful instrument in the field of body‐oriented psychotherapy research and related treatment methods, and increased body awareness has an important role in Dohsa‐hou relaxation therapy.