Specification of Scales in Biodata Form Development: Rational vs. Empirical and Global vs. Specific

Although empirical keying has been the most popular scoring strategy with biodata, researchers have increasingly argued that rational approaches are better for advancing theory. Higher validities and less faking with empirical keys, however, have made many reluctant to abandon them. Research in the personality field provided support for the notion that many rational biodata scales may be multi-faceted. Development of more specific scales was suggested as a technique for creating rational scales with validities that more closely approximate that of empirical keys. Three different strategies (rational, internal, and external/empirical) were used in the scoring of a biodata inventory for use with mechanical equipment franchise owners. The rational and internal approaches were investigated within two samples and at two levels - very specific constructs and global constructs. In addition, two types of criteria were used, including an objective measure of sales and supervisory performance ratings. The specific rational scales were as predictive as the empirical item key. Strategies for developing and scoring a biodata form using a rational approach are discussed.     

Unavailability Effects on Message Processing: A Theoretical Analysis an Empirical Test

This study is concerned with the ability of standard indexes, of socioeconomic status and similar devices to measure major dimensions of social stratification for whites and blacks. Factor analyses of survey data for samples of both races uncovered five matching factors and two others unique to a single sample, all of which seem to represent important components of stratification. Most of the indexes were relevant to three matching factors, and some of these measures consistently loaded them in both samples. However, several of the pertinent indexes only loaded white versions of these factors, and some failed to load any. Few, if any, indexes were relevant to or loaded the remaining factors. Additionally, some indexes were not pertinent to and did not load any factor. In contrast to the indexes, other variables loaded all but one of the factors. The main conclusions are that: (1) the coverage of the indexes is restricted; (2) the devices are not uniformly applicable to blacks; and (3) the complexity of this domain requires separate measures of each factor, achievable by combining standard indexes and new variables.     

The Construct Validity of Curriculum-Based Measurement of Reading: An Empirical Test of a Plausible Rival Hypothesis

Research has confirmed that curriculum-based measurement (CBM) of oral reading fluency and measures of reading comprehension are highly correlated, as predicted by developmental theories of reading. Research on CBM, however, has only begun to rule out plausible alternative explanations of this relationship—an important aspect of a strong program of construct validation (e.g., Messick, 1989). This study investigated one such rival hypothesis by examining the relative roles of general cognitive ability, speed and efficiency of elemental cognitive processing, and oral reading fluency in the prediction of reading comprehension. Results of simultaneous multiple regression analyses substantiate the construct validity of CBM oral reading fluency. These findings indicate that the significant relationship between oral reading fluency and reading comprehension cannot be explained by general cognitive ability or by processing speed and efficiency. CBM oral reading fluency also did not correlate significantly with any of the processing speed and efficiency tasks. Interestingly, however, CBM oral reading fluency accounted for less variance in reading comprehension (r² = .17) than expected based on the results of previous research and less than that explained by general cognitive ability (r² = .24). When controlling for psychometric g and processing speed in the regression analyses, CBM oral reading explained 11% of the variance in reading comprehension. Implications of these results for further research on the construct validity of CBM are discussed.     

Episcopal Measure of Faith Tradition: A Context-Specific Approach to Measuring Religiousness

Precise measurement of religiousness remains a vexing problem. In addition to relying almost exclusively on self-report, existing measures of religiousness pay little attention to the specific context of religious belief, and this may override distinctive norms of particular faith traditions and potentially confound the conclusions drawn from such research. To address these limitations, the authors describe a modified form of narrative content analysis that could eventually sort respondents into distinct theological traditions. A pilot test among Episcopalians demonstrates encouraging reliability (kappa 0.74, 95% LCI 0.47, P < 0.0002), and tests for convergent and discriminate validity suggest that the context of religious belief is both relevant and insufficiently assessed by the existing paradigm of religious measurements. If validated in a religiously diverse sample, this approach could be combined with existing, context-free measures of religiousness to generate more meaningful findings.     

The Measurement and Interpretation of Organizational Climate

This study describes the development and application of a new organizational climate instrument, the Survey of Organizational Characteristics (SOC). The participants in the development phase were 94 master's of business administration students, and the participants in the validation phase were 77 medical professionals: 24 medical faculty, 17 postdoctoral students, and 36 medical technicians. The final instrument consisted of 8 relatively independent scales with alpha coefficients based on the combined sample ranging from .77 to .98, and with 6 coefficients that were higher than .90. Validity hypotheses based on a priori knowledge about the 3 medical groups were for the most part well substantiated, although the small sample of postdoctoral students was a limitation. In relation to the findings, an attempt was made to clarify and perhaps narrow several of the conceptual and methodological gaps that persist between organizational climate and organizational culture.     

It-Scale for Children and Draw-A-Person Test: 30 Feminine vs. 25 Masculine Boys

Thirty feminine and twenty-five masculine boys, aged 4-10, were administered the It-Scale for Children and the Draw-a-Person Test. A feminine boy was defined by his preference for the dress, games, toys, role, and companionship of girls, and the stated wish to be a girl.

Results on both tests differed significantly for the two groups. Feminine boys scored similarly to published norms for girls, while masculine boys scored similarly to published norms for boys.     

A General Method of Empirical Q-matrix Validation

In contrast to unidimensional item response models that postulate a single underlying proficiency, cognitive diagnosis models (CDMs) posit multiple, discrete skills or attributes, thus allowing CDMs to provide a finer-grained assessment of examinees’ test performance. A common component of CDMs for specifying the attributes required for each item is the Q-matrix. Although construction of Q-matrix is typically performed by domain experts, it nonetheless, to a large extent, remains a subjective process, and misspecifications in the Q-matrix, if left unchecked, can have important practical implications. To address this concern, this paper proposes a discrimination index that can be used with a wide class of CDM subsumed by the generalized deterministic input, noisy “and” gate model to empirically validate the Q-matrix specifications by identifying and replacing misspecified entries in the Q-matrix. The rationale for using the index as the basis for a proposed validation method is provided in the form of mathematical proofs to several relevant lemmas and a theorem. The feasibility of the proposed method was examined using simulated data generated under various conditions. The proposed method is illustrated using fraction subtraction data.     

Group Leaders as Gatekeepers: Testing Safety Climate Variations across Levels of Analysis

This paper tests the moderating effect of transformational supervisory leadership on the relationship between organisational and group climates, using safety climate in risky operations as an exemplar. Results indicated that under low or poor organisational climate, indicative of limited organisational commitment to employee safety, transformational leaders promoted a higher group climate as compared to the organisational climate. Similarly, under a weak organisational climate, indicative of limited consensus among company employees regarding the priority of safety, transformational leaders promoted a stronger group climate, reflecting greater consensus among group members. This pattern suggests that supervisory leaders can act as gatekeepers, with transformational leaders offering better protection against potentially harmful organisation‐level priorities. Furthermore, transformational supervisors better informed their members of the organisational priorities as they perceived them, resulting in a stronger relationship between individual supervisors' perceptions and members' organisational climate perceptions. Implications for climate and leadership research are discussed. On s'intéresse dans cet article à l'impact régulateur du leadership hiérarchique de transformation sur la relation entre les climats organisationnels et de groupe en utilisant comme exemple la sécurité dans les opérations à risque. Les résultats montrent qu'en cas de climat organisationnel médiocre signifiant une implication organisationnelle minimale à propos de la sécurité du personnel, les leaders de transformation favorisent l'émergence d'un climat de groupe plus satisfaisant que le climat organisationnel. De même, si le climat organisationnel est fragile, traduisant un accord approximatif des salariés sur la prioritéà accorder à la sécurité, les leaders de transformation sont à l'origine d'un climat de groupe renforcé, expression d'un meilleur consensus parmi les membres du groupe. Ce schéma laisse penser que les leaders hiérarchiques peuvent intervenir comme gardiens, les leaders de transformation offrant une meilleure protection contre les priorités organisationnelles potentiellement nocives. De plus, les managers de transformation informent mieux leur équipe des priorités organisationnelles telles qu'ils les perçoivent, ce qui provoque une relation plus étroite entre les perceptions personnelles du manager et celles que son groupe a du climat organisationnel. On réfléchit aux retombées de ce travail sur les recherches portant sur le climat et le leadership.     

Psychometric Properties and Administration Measurement Invariance of Social Phobia Symptom Measures: Paper-Pencil vs. Internet Administrations

The psychometric properties of the online versions of assessment instruments for phobias and related symptoms were compared to those of paper-pencil versions. The examined measures were the Social Interaction Anxiety Scale (SIAS) and Social Phobia Scale (SPS). Students (n = 514) completed the instruments either in a paper-pencil in lab condition or an online at home condition. Means of the measures were largely equivalent between the two groups. Factor structures of the measures support construct validity of the two versions of the questionnaires. The coefficient alphas for the measures were reasonably high, ranging between .91 and .93 for both groups. Correlations between the measures were similar across groups. Overall, adequate psychometric properties of the online versions of the instruments were demonstrated, however, neither of the social phobia measures demonstrated measurement invariance between the two assessment modalities.     

Internet-Based,Unproctored Assessments on Mobile and Non-Mobile Devices: Usage,Measurement Equivalence,and Outcomes

Purpose

The purpose of this study was to determine the usage rates, measurement equivalence, and potential outcome differences between mobile and non-mobile device-based deliveries of an unproctored, non-cognitive assessment.

Design/Methodology/Approach

This study utilized a quasi-experimental design based on archival data obtained from applicants who completed a non-cognitive assessment on a mobile (n = 7,743; e.g., smartphones, tablet computers) or non-mobile (n = 929,341; e.g., desktop computers) device as part of an operational, high-stakes pre-employment selection process.

Findings

One percent of applicants used mobile devices to complete the assessment. Multiple-group confirmatory factor analysis indicated the assessment was equivalent across mobile and non-mobile devices at the configural, metric, scalar, and latent mean levels. A comparison of observed score means using one-way and factorial ANOVAs demonstrated that the use of mobile and non-mobile devices did not produce any practically significant score differences on the assessment across devices or applicant demographic subgroups.

Implications

Industry and technological trends suggest mobile device usage will only increase. Thus, demonstrating that mobile device functionality and hardware characteristics do not change the psychometric functioning or applicant outcomes for a non-cognitive, text-based selection assessment is critical to talent assessment.

Originality/Value

This study provides the first empirical examination of the usage of mobile devices to complete talent assessments and their impact on assessment properties and applicant outcomes, and serves as the foundation for future research and application of this growing technological trend in pre-employment assessment.

     

The MOS Short-Form General Health Survey: single item vs multiple measures of health-related quality of life: some nuances.

Researchers concluded that Item 2 of the Short-form General Health Survey of the Medical Outcomes Study can be used as a single-item measure of health-related quality of life. This conclusion is tested again on a random sample of elderly people in the Netherlands. Although it can be concluded that the psychometric properties of the short form are satisfactory, the use of the single item as a measure of health-related quality of life is discussed.     

Discrete vs. continuous processing: The fate of an incompletely processed perceptual dimension

A relevant issue in the debate on continuous vs. discrete processing of information is whether incompletely processed information does or does not affect a subsequent reaction. Two stimuli (SL, SR) were presented on eye level, subtending a visual angle of 45 degrees. SL was always inspected first, followed by a saccade to SR and finally by a same/different response. The fixation time of SL (TL), the saccadic time (TM) and the time from fixating SR to the response (TR) were separately measured. SL and SR consisted of two-dimensional stimuli (size and letter shape) constructed in such a way that encoding size took longer for one group and encoding shape took longer for another group of 10 subjects. All subjects were tested in three conditions: Shape was relevant in one, size in the second, and both dimensions were relevant in the third condition.

TL was less when encoding the relevant dimension was fast. When both dimensions were relevant, TL was about as long as when only the slow dimension was relevant, suggesting parallel and interference-free processing during TL. When only the slow dimension was relevant, TR (same) was much longer when the fast dimension differed. When the fast dimension was relevant, TR (same) was slightly longer when the slow dimension differed, which can be handled by either model. The experiment was repeated with three well-practiced and less variable subjects who carried out sufficient trials to measure TR as a function of TL. The results of this study were in line with the discrete model: A different slow and irrelevant dimension did not affect the same response regardless of the duration of TL. Interestingly, subjects were capable of retrieving the slow dimension, suggesting a code which can be used for retrieval but which does not affect the same/different response.     

On the Measurement of Meaning: Logotherapy's Empirical Contributions to Humanistic Psychology

Meaningful living is a central focus of several humanistic theories and therapies. Measurement of life meaning meets many obstacles, including pragmatic concerns, such as measuring subjective experiences, and theoretical objections often offered by humanistic psychologists. The purpose of this article is to summarize empirical efforts related to logotherapy, a humanistic–existential paradigm, to illustrate the utility of assessment within the larger context of humanistic psychology. An overview of five logotherapeutic measures of meaning is provided. These measures include the Purpose in Life test (PIL), the Life Purpose Questionnaire (LPQ), the Seeking of Noetic Goals test (SONG), the Meaning in Suffering Test (MIST), and the Life Attitude Profile Revised (LAP–R). Directions for use of such measures in future research are also offered.     

Information,Experience, and Divergent Thinking: An Empirical Test

ABSTRACT: Divergent thinking tests are often used to estimate the potential for creative problem solving. Scores on these tests may, however, reflect a kind of experiential bias. Similar biases once plagued IQ tests, the idea being that scores reflect the individual's background and information in long-term memory as much as ability per se. The investigation reported here attempted to assess the role of experience, knowledge, and memory in divergent thinking by comparing 2 kinds of tasks. One was a standard divergent thinking task (e.g., "list uses for a shoe," "uses for a brick," "uses for a newspaper"). The other allowed a number of responses but required that the examinee produce factual, knowledge- based responses. A second objective here was to compare standard- and knowledge-based ideation with tasks that shared 1 domain (e.g., transportation) with tasks that did not share a domain. Results indicated that there was a statistically significant correlation between the 2 types of tasks but only when they shared 1 domain. This was confirmed with product–moment correlations (r = .37, p = .025) and a canonical correlation (Rc = .69, p = 008). The correlations were not significant when the tasks represented different domains. It is interesting to note that both the knowledge-based and the standard divergent thinking tasks were unrelated to grade point average, which supports their discriminant validity. Limitations and directions for future research are suggested.     

The Cognitive Interview for Use with Adults: An Empirical Test of an Alternative Mnemonic and of a Partial Protocol

Research showed that the four Cognitive Interview (CI) mnemonics used individually are unequally effective. We propose to test (i) their benefit when used within the same free recall phase and (ii) an original instruction, ‘guided peripheral focus’ (GPF). In two studies, 84 and 42 students were interviewed with Structured Interviews (SI), CIs or CI variations about a film viewed 1 week before. Results indicated that (i) if a CI variation with the GPF instead of the ‘perspective’ elicits more correct information than an SI or a CI, variations replacing the ‘perspective’ or the ‘order’ and ‘perspective’ with control instruction(s) do not; (ii) a partial CI integrating the ‘everything’, ‘context’ and GPF increases correct information compared with an SI, whereas the CI does not. We will discuss in which extent these results demonstrate the effectiveness of the ‘everything’, ‘context’ and ‘GPF’ and a potential lack of benefit of the ‘order’ and ‘perspective’. Copyright © 2011 John Wiley & Sons, Ltd.