When is best-worst best? A comparison of best-worst scaling,numeric estimation,and rating scales for collection of semantic norms

Large-scale semantic norms have become both prevalent and influential in recent psycholinguistic research. However, little attention has been directed towards understanding the methodological best practices of such norm collection efforts. We compared the quality of semantic norms obtained through rating scales, numeric estimation, and a less commonly used judgment format called best-worst scaling. We found that best-worst scaling usually produces norms with higher predictive validities than other response formats, and does so requiring less data to be collected overall. We also found evidence that the various response formats may be producing qualitatively, rather than just quantitatively, different data. This raises the issue of potential response format bias, which has not been addressed by previous efforts to collect semantic norms, likely because of previous reliance on a single type of response format for a single type of semantic judgment. We have made available software for creating best-worst stimuli and scoring best-worst data. We also made available new norms for age of acquisition, valence, arousal, and concreteness collected using best-worst scaling. These norms include entries for 1,040 words, of which 1,034 are also contained in the ANEW norms (Bradley & Lang, Affective norms for English words (ANEW): Instruction manual and affective ratings (pp. 1-45). Technical report C-1, the center for research in psychophysiology, University of Florida, 1999).     

Assessing and overcoming participant dishonesty in online data collection

Crowdsourcing services, such as MTurk, have opened a large pool of participants to researchers. Unfortunately, it can be difficult to confidently acquire a sample that matches a given demographic, psychographic, or behavioral dimension. This problem exists because little information is known about individual participants and because some participants are motivated to misrepresent their identity with the goal of financial reward. Despite the fact that online workers do not typically display a greater than average level of dishonesty, when researchers overtly request that only a certain population take part in an online study, a nontrivial portion misrepresent their identity. In this study, a proposed system is tested that researchers can use to quickly, fairly, and easily screen participants on any dimension. In contrast to an overt request, the reported system results in significantly fewer (near zero) instances of participant misrepresentation. Tests for misrepresentations were conducted by using a large database of past participant records (~45,000 unique workers). This research presents and tests an important tool for the increasingly prevalent practice of online data collection.     

Carers’ beliefs about counselling: a community participatory study in Wales

This interpretivist community participatory study explores carers’ beliefs about the potential usefulness of counselling in relation to the caring role. Twenty semi-structured interviews with carers were transcribed and analysed thematically. All participants thought counselling could potentially be helpful to carers, but their ideas about the ways in which it would help varied according to whether or not they had personal experience of counselling. Only carers with counselling experience had an awareness of counselling as focussing on changing feelings, thoughts, attitudes and behaviours. The implications of the findings for the provision of counselling services are discussed.     

The Sense and Sensibility of Equality

The idea of equality in political thought is often approached from a distributive perspective that entails a rethinking of institutional arrangements. In this paper I present an approach to conceived as a complement to the common institutional approach in liberal theory. The foundational claim is that blacks do not come into view for a wide range of people as worthy of full human recognition, that is, persons in possession of human vulnerabilities that require responses and in possession of warrants to rights that demand respect. The central argument is that this problem of diminished social value can be addressed by imagination and that imagination is crucially important to bridge the experiential divide between white and black Americans. I deploy two ideas I claim are essential for leveraging imagination: attentiveness and skill. When we are attentive we provide the imagination range to sense the more difficult aspects of others’ lives under inequality; when we are skilled we deploy a finer sensibility in taking the proper ethical stance towards others’ experiential difficulties under a regime of racial inequality. To achieve this, I provide a schematic of three pairs of concepts that provide a foundation though not full accounting of the necessary skills in response to ethical challenges: narrative/receptivity, reasons/responsibility, affect/compassion.     

Sailing for New Atlantis

Several academic traditions have addressed epistemological objectivity and/or partisanship in the study of technoscientific controversies. On the one hand, positivist and relativist scholars agree that the political commitments of the social researcher should not impinge on scientific enquiry, while on the other hand, feminist and Marxist scholars not only take stands in diverse technoscientific debates, but even claim their agendas to be more credible than those of orthodox scientists. Such perspectives stress that all research is partisan in one way or another because it involves questions of who controls, manipulates, and establishes decisions, facts, and knowledge. With this in mind, it is possible to identify different forms of partisan research including capture by participants, de facto and overt partisanship, and mercenary scholarship. These different forms of partisan scholarship are characterised by differences in the motives underlying epistemological choices of research topic and method, personal commitments to the fields studied, use of research findings in controversies, and positioning of results in wider debates. Two examples help to illustrate partisan scholarship: first, a study of new technologies for managing climate change (carbon dioxide capture and storage); and second, the construction of the new underground metro system in Athens and its accommodation of accessibility standards. Both cases entail partisan positions and raise similar concerns about the orthodox epistemological assumptions underpinning sociotechnical systems, especially when it comes to technoscientific controversies. Supporting STS partisan scholarship, therefore, enables greater social and democratic engagement with technoscientific development.     

Impact and Causes of Rater Severity/Leniency in Appraisals without Postevaluation Communication Between Raters and Ratees

In performance appraisals, some assessors are substantially more lenient than others. Research on this effect in appraisals involving communication and interaction between raters and ratees after the performance evaluation has taken place indicates that it may be at least partly caused by individual differences in assessor personality. However, little is known about the impact or causes of rater severity versus leniency in situations in which there is little or no contact between raters and ratees after the performance evaluation. In Study 1 (N = 174) the strength of the severity–leniency effect in this ‘no‐contact’ context is estimated and found to be similar to that reported for ‘with‐contact’ appraisals. No evidence of an association between assessor personality and assessor severity (vs. leniency) is found in the ‘no‐contact’ context. In Study 2 (N = 54) there is no evidence of an association between the fluid cognitive ability of assessors and the severity of their ratings in a no‐contact context. It is concluded that the severity versus leniency effect probably has a considerable impact on performance ratings in ‘no‐contact’ appraisal settings, but that neither rater personality nor rater cognitive ability appear to play a significant role in this.     

Perceptions of the roles of behaviour and genetics in disease risk: Are they associated with behaviour change attempts

Objective: We examined selected survivor characteristics to determine what factors might moderate the response to two psychosocial interventions.

Design: Seventy-one prostate cancer survivors (PCSs) were randomly assigned to either a telephone-delivered health education (THE) intervention or a telephone-delivered interpersonal counselling (TIP-C) intervention.

Measures: Psychological quality of life (QOL) outcomes included depression, negative and positive affect, and perceived stress.

Results: For three of the psychological outcomes (depression, negative affect and stress), there were distinct advantages from participating in THE. For example, more favourable depression outcomes occurred when men were older, had lower prostate specific functioning, were in active chemotherapy, had lower social support from friends and lower cancer knowledge. Participating in the TIP-C provided a more favourable outcome for positive affect when men had higher education, prostate specific functioning, social support from friends and cancer knowledge.

Conclusion: Unique survivor characteristics must be considered when recommending interventions that might improve psychological QOL in PCSs. Future research must examine who benefits most and from what components of psychosocial interventions to enable clinicians to recommend appropriate psychosocial care.     

How much behaviour change should we expect from health promotion campaigns targeting cognitions? An approach to pre-intervention assessment

For those planning interventions based on social cognition models, it is usually not clear what impact on behaviour will follow from attempts to change the cognitions specified in these models. We describe a statistical simulation technique to assess the likely impact of health promotion targeting Theory of Reasoned Action (TRA)-based predictors of condom use. We apply regression-based simulation techniques to data from the SHARE project (n?=?756 Scottish adolescents) to assess the potential impact of changes in cognitions on condom use. Results support the predictive utility of TRA-based models of psychological antecedents of condom use but also provide a cautionary warning about the magnitude of behaviour change likely to be achieved by interventions based on such models.