Evaluating Item-Sort Task Methods: The Presentation of a New Statistical Significance Formula and Methodological Best Practices

Purpose

An item-sort task is a common method to reduce over-representative item lists during the scale-creation process. The current article delineates the limitations and misapplications of the accepted statistical significance formula for item-sort tasks and proposes a new statistical significance formula with greater utility across a wider range of item-sort tasks.

Design

First, a simulation study compares the two formulas in an array of conditions that vary on sample size and number of assignment choices. Second, an empirical study compares the results of three separate item-sort tasks across the two formulas for statistical significance.

Findings

In the empirical study, the proposed formula produces more correct retention decisions than the existing formula across all three item-sort tasks. In the simulation study, the proposed formula is more appropriate than the existing formula under most conditions. The two formulas function identically in item-sort tasks with only two assignment choices.

Implications

Researchers could obtain erroneous results when misapplying the existing item-sort task statistical significance formula to cases with more than two assignment choices. The proposed formula corrects this limitation, ultimately providing accurate results more often than the existing formula. Applying the proposed formula could help future research and practice throughout the scale development process.

Originality

Despite widespread use, few attempts have been made to improve scale-creation pretest methods, particularly item-sort tasks. The current study demonstrates that even conventional statistical methods are susceptible to misuse and misapplication, and future research could benefit from the reexamination of other common methods.

     

Stability and change in religiousness during emerging adulthood

Understanding the development of religiousness is an important endeavor because religiousness has been shown to be related to positive outcomes. The current study examined mean-level, rank-order, and individual-level change in females' religiousness during emerging adulthood. Genetic and environmental influences on religiousness and its change and stability were also investigated. Analyses were completed with an epidemiological study of 2 cohorts of twins: 1 assessed at ages 14 and 18 and a 2nd at 20 and 25. Mean levels of religiousness decreased significantly with age, while rank-order stability was high. Individual-level change was also evident. Analyses also supported the hypotheses that more change would occur in the younger cohort compared with the older cohort and that more change would occur in religious service attendance than the general index of religiousness. Twin analyses suggested that the heritability of religiousness increased with age, while the shared environmental influences decreased. For the younger cohort, change was genetic in origin, while stability was environmental. In the older cohort, change was influenced by nonshared environment and stability by both genes and family environment.     

Learning myopia: an adaptive recency effect in category learning

Recency effects (REs) have been well established in memory and probability learning paradigms but have received little attention in category earning research. Extant categorization models predict REs to be unaffected by learning, whereas a functional interpretation of REs, suggested by results in other domains, predicts that people are able to learn sequential dependencies and incorporate this information into their responses. These contrasting predictions were tested in 2 experiments involving a classification task in which outcome sequences were autocorrelated. Experiment 1 showed that reliance on recent outcomes adapts to the structure of the task, in contrast to models' predictions. Experiment 2 provided constraints on how sequential information is learned and suggested possible extensions to current models to account for this learning.     

The ‘thinking aloud’ process: a way forward in social work supervision

Within current neoliberal discourses, critical reflection provides opportunity for innovation in social work practice. This article describes a thinking aloud process used with supervisor–supervisee dyads in community-based child welfare social work to assist critical reflection via the use of open-ended questioning and inquiry. The strength of thinking aloud permitted a deeper appreciation of how the supervision session is utilised to reflect on practice and provided a learning tool within supervision. The feedback from the dyads concurred that thinking aloud assisted in stimulating reflection, developing solutions to the key issues discussed, and was transformative in recognising areas for future development. The process of thinking aloud supports the opportunity and space for the supervisor and supervisee to articulate meaning, critically develop insight, reconstruct, and transport this into future practice. Thinking aloud offers an example of how knowledge can be co-constructed by practitioners within practice and critical reflection captured within qualitative research approaches.     

Addressing Relationship Health Needs in Primary Care: Adapting the Marriage Checkup for Use in Medical Settings with Military Couples

The overall objective of this study was to pilot the Marriage Checkup (MC), a brief intervention for enhancing marital resiliency tailored to a military population, for use by internal behavioral health consultants (IBHCs) working in an integrated primary care clinic. The MC was revised to fit into the fast-paced environment of primary care (e.g., streamlined to fit within three 30-min appointments), and military-relevant material was added to the content. IBHCs working in primary care were then trained to offer the intervention. Thirty participants were enrolled in the study and completed a relationship checkup and one-month follow-up questionnaires. Analysis of post-test and one-month follow-up data showed statistically significant improvements in participants’ marital health compared to pre-treatment. The MC intervention appeared to be well received by both couples and IBHCs.     

Two Russellian Arguments for Acquaintance

Bertrand Russell [1912] argued that we are acquainted with our experiences. Although this conclusion has generated a lot of discussion, very little has been said about Russell's actual arguments for it. This paper aims to remedy that. I start by spelling out two Russellian arguments for acquaintance. Then I show that these arguments cannot both succeed. For if one is sound, the other isn't. Finally, I weigh our options with respect to these arguments, and defend one option in particular. I argue that we have good reason to believe that we can be, and sometimes are, acquainted with our experiences.