Web-based image norming: How do object familiarity and visual complexity ratings compare when collected in-lab versus online?

It has been demonstrated previously that, for some experimental paradigms, Web-based research can reliably replicate lab-based results. Yet questions remain as to what types of research can be reproduced, and where differences arise when they cannot be. The present article examines the effect of research location (laboratory vs. online) on normative data collection tasks. Specifically, participants were randomly assigned to a laboratory or online condition and were asked to rate 593 photorealistic images on the basis of object familiarity (N=5 103) and object visual complexity (N=5 98). Dependent measures were compared across location conditions, including response latencies and image rating agreement. Our results suggest that norming data collected online are reliable, but an interesting interplay between task type and research location was observed. Specifically, we found that participating online (i.e., a more familiar environment) leads to systematically higher familiarity ratings than in the lab (i.e., an unfamiliar environment). These differences are not found when the alternate complexity rating task is used.     

Social Interaction and Internet-Based Surveys: Examining the Effects of Virtual and In-Person Proctors on Careless Response

A lack of human interaction and environmental control in Internet-based data collection have been suggested as possible antecedents of careless responding, which occurs when participants respond to survey items without regard for item content. To address these possible antecedents, this study investigated whether survey proctoring deterred careless response in an undergraduate sample by reducing environmental distractions. The study randomly assigned respondents to one of three proctoring conditions: remote online un-proctored, remote online virtually proctored, and in-person classroom proctored. Data quality was examined via nine careless response indicators. Analyses indicated that proctor presence had effects on a small number of careless response indicators. Virtually proctored participants performed better than un-proctored participants on one of nine careless response indicators, and in-person proctored participants performed better on two careless response indicators compared to un-proctored participants. Environmental distraction fully mediated the relationship between in-person proctor presence and self-reported diligence. Implications for survey administration are discussed.     

Online data collection from video game players: methodological issues.

The paper outlines the advantages and disadvantages of using the Internet to collect data concerning both online and offline gamers. Drawing from experience of a number of studies carried out online by the authors and by reviewing the available literature, the authors discuss the main issues concerning data collected from video game players. The paper examines a number of areas, including recruiting and utilizing participants, validity, suitable methods of data collection (i.e., questionnaire studies, online tests, participant observation, online interviews), and ethical issues. It is concluded that online research methods can be a useful way of examining the psychosocial aspects of video game playing.     

Practical advice for conducting ethical online experiments and questionnaires for United States psychologists

It is increasingly easy and, therefore, increasingly common to conduct experiments and questionnaire studies in online environments. However, the online environment is not a data collection medium that is familiar to many researchers or to many research methods instructors. Because of this, researchers have received little information about how to address ethical issues when conducting online research. Researchers need practical suggestions on how to translate federal and professional ethics codes into this new data collection medium. This article assists United States psychologists in designing online studies that meet accepted standards for informed consent, deception, debriefing, the right to withdraw, security of test materials, copyright of participants’ materials, confidentiality and anonymity, and avoiding harm.     

Dynamic Interviewing Program (DIP): automatic online interviews via the instant messenger ICQ.

A Dynamic Interviewing Program (DIP) is presented for automated online interviewing via the instant messaging service ICQ. DIP can conduct parallel structured interviews with several ICQ users at a time, 24 hours a day. The interview itself can be dynamic (branched testing), which means that each question depends on the answer given to the previous one. An online ICQ address book offers the possibility to validate demographic data in order to evaluate data quality. Experiences with DIP and a first empirical study open up new perspectives for Internet-based automatic interviewing.     

Measurement invariance tests of the Impression Management sub-scale of the Balanced Inventory of Desirable Responding

The Impression Management (IM) subscale of the Balanced Inventory of Desirable Responding (Paulhus, 1988) has been used as a proxy for common method variance in anonymous settings and as a cause of faking on personality testing in confidential settings. This study uses confirmatory factor analysis to conduct measurement invariance/equivalence tests (also known as multiple group analysis) on the IM subscale in a quasi-experiment in anonymous and confidential data collection settings. Using Brown’s (2006) bottom-up approach to Cheung and Rensvold’s (1999) model testing steps and Cheung and Rensvold’s (2002) statistical tests, the IM sub-scale was determined to have equal form, equal factor loadings, equal indicator error variances, equal factor variance, and an equal factor covariance in both 1-factor and 2-factor models in both data collection settings. Mean scale scores were significantly higher in the confidential group than in the anonymous group. These results suggest that using the IM sub-scale as a cause of faking and as a proxy for common method variance (CMV) is likely to be acceptable because the psychometric properties of the instrument are invariant across testing conditions.     

Honest and Deceptive Impression Management in the Employment Interview: Can It Be Detected and How Does It Impact Evaluations?

Applicants use honest and deceptive impression management (IM) in employment interviews. Deceptive IM is especially problematic because it can lead organizations to hire less competent but deceptive applicants if interviewers are not able to identify the deception. We investigated interviewers’ capacity to detect IM in 5 experimental studies using real‐time video coding of IM (N = 246 professional interviewers and 270 novice interviewers). Interviewers’ attempts to detect applicants’ IM were often unsuccessful. Interviewers were better at detecting honest than deceptive IM. Interview question type affected IM detection, but interviewers’ experience did not. Finally, interviewers’ perceptions of IM use by applicants were related to their evaluations of applicants’ performance in the interview. Interviewers’ attempts to adjust their evaluations of applicants they perceive to use deceptive IM may fail because they cannot correctly identify when applicants actually engage in various IM tactics. Helping interviewers to better identify deceptive IM tactics used by applicants may increase the validity of employment interviews.     

Convergent methods in psychomusical research based on integrated,interactive computer control

Much of what we learn from an experiment depends on what we ask of the subjects and how we ask it. Most music perception and cognition research is implemented using a single methodology. Recently, questions have been raised about the validity and generalizability of such a limited approach, particularly regarding such complicated theoretical constructs as musical tonality. We propose that a useful technique is to converge on the answer to experimental questions by applying multiple methods, in essence, simultaneously investigating the central research question as well as ancillary questions of method. Such an approach is facilitated by a computer hardware and software system that integrates experimental design, data collection, data analysis, and data dissemination, linking off-the-shelf components through standard file formats and data-exchange protocols. The list-based system, using an IBM-type PC running Windows 3.0, handles real-time natural sound sampling and playback, signal synthesis and analysis, MIDI input and output, digitized picture display, and various subject-response mechanisms. On the basis of graphical icons, the integrated system can be used to build custom experimental designs. Examples of convergent psychomusical experiments that were designed, implemented, and analyzed using the system are discussed.     

Timepieces: Components of Survey Question Response Latencies

Political scientists have increasingly found that the time respondents require to answer survey questions is a useful measure of the strength of political attitudes. However, questions remain about the collection, use, and interpretation of response time data— particularly latencies collected during telephone interviews, far removed from the controlled environment of a laboratory setting. This paper uses a theoretical model of question response to decompose survey response time data into three hypothesized elements—components of response time attributable to baseline individual differences, systematic question effects, and the accessibility of the attitudes being targeted. These findings have implications for the study of political attitudes and other mental constructs, as well as practical implications for public opinion survey researchers using response time data.     

A Systematic Review of Online Response Rates in Four Counseling Journals

This systematic review of articles published in 4 counseling journals found 207 articles that used online recruitment, of which 98 (47.3%) reported a response rate. These 98 studies collectively contained 22,466 individual participants. Results showed that nearly one third of all empirical articles used online recruitment, and the average response rate was 34.2% (SD = 22.6). Response rates are further described by journal, population, research methodology, and recruitment strategy. Researcher‐reported issues with online recruitment and data collection were also analyzed qualitatively, and recommendations for researchers are discussed.     

Expectation creates something out of nothing: The role of attention in iconic memory reconsidered

Conscious experience is modulated by attention and expectation, yet is believed to be independent of attention. The experiments on iconic memory (IM) are usually taken as support for this claim. However, a recent experiment demonstrated that when attention is diverted away from the IM letter display subjects fail to see the absence of IM letters. Here we contribute to the ongoing debate by overcoming experimental shortcomings of this previous experiment, by measuring subjective visibility and by testing the effect of the post-cue. We were able to replicate these earlier findings and extend them by demonstrating that subjects who do not realize the absence of letters perceive illusory letters. This result means that there is still phenomenal consciousness, even when attention is diverted. Expectation creates illusory content that overwrites valid IM content. Taken together these findings suggest that the present experimental paradigm is not appropriate to make claims about IM content.     

IS THERE A METHOD TO OUR MADNESS? THE IMPACT OF DATA COLLECTION METHODOLOGY ON ORGANIZATIONAL SURVEY RESULTS

Results from 3 organizational survey efforts were used to explore the impact of data collection method on 5 outcomes: response preference, item completion rate, use of don't know responses, item mean and variability. The first study compared 977 online with 711 opscan responses in a U.S. government agency. The second study compared a Time 2 administration of 893 online with 732 opscan responses. The third study examined 4,654 automated phone (IVR) with 3,587 paper responses in a global pharmaceuticals firm. Although method utilization varied considerably by country, the more technological approaches were preferred in the U.S., and younger employees were significantly more likely to respond online. Overall, survey method accounted for a relatively small percentage of unique variance in the data (0% to 4%). Practitioners may be better off choosing an administration method based on factors such as cultural fit and ease of implementation rather than issues of data quality.     

Online implementation of an event history calendar with formr: A tutorial

Event history calendars (EHCs) are popular tools for retrospective data collection. Originally conceptualized as face‐to‐face interviews, EHCs contain various questions about the respondents' autobiography in order to use their experiences as cues to facilitate remembering. For relationship researchers, EHCs are particularly valuable when trying to reconstruct the relational past of individuals. However, although many studies are conducted online nowadays, no freely available online adaptation of the EHC is available yet. In this tutorial, detailed instructions are provided on how to implement an online EHC for the reconstruction of romantic relationship histories within the open‐source framework formr. Ways to customize the online EHC and provide a template for researchers to adapt the tool for their own purposes are showcased.     

Mediating Access to Headsprout<Superscript>®</Superscript> Early Reading for Children with Autism Spectrum Disorders

Many children with autism spectrum disorder (ASD) require instructional programs that simultaneously address academic content, interfering behavior, and potential gaps in prerequisite skills for a given curriculum. This study examined the impact of an intervention package consisting of contingent reinforcement and match-to-sample training on the independent progression of 3 children with ASD through Headsprout^® Early Reading (HER), a behavior analytic online reading program. A multiple probe across participants design was used to examine the effects of the intervention package on participants’ correct interactions per minute and total episodes completed. Transfer of training to oral reading of companion print books was also assessed. All participants demonstrated an increase in correct interactions per minute. Two participants learned to independently complete a HER episode each day and could orally read companion print books. The results suggest individualized behavior analytic programs can support the use of HER as a computer-based reading program for some children with ASD.     

Religion and the Human Rights Idea

Three recent books focus, in different ways, on the idea of human rights and its relation to religion and religious ethics. All three books discussed here address criticisms of the human rights idea and seek to establish the relationship of religion and human rights with regard to the field of policy. The present discussion begins with an overview that places these three books in the larger context of the development of the human rights idea and its historical relationship with religion. It then turns to Little's book, next to the collection of essays edited by Twiss, Simion, and Petersen, which is described internally as a Festschrift for Little, and then to Hogan's book, and in the final section it explores comparisons among the books.     

Interpersonal Assessment of Borderline Personality Disorder: Preliminary Findings

We examined the reliability and validity of scores on an interpersonal measure of borderline personality disorder (BPD). Ratings on the Interpersonal Measure of Borderline Personality Disorder (IM–B) were based on nonverbal behaviors and interpersonal interactions occurring during clinical interviews with 276 adults. Scores on the IM–B exhibited good reliability. IM–B scores also displayed expected patterns of associations with scores on other measures of BPD, as well as with scores on measures of affective dysfunction, interpersonal pathology, and behavioral impairment associated with BPD, including indexes of maladaptive emotion regulation, interpersonal sensitivity, and self-harm. The pattern of associations for IM–B scores was quite similar to what would be expected for a dimensional measure of BPD symptoms. Scores on the IM–B were also associated with symptoms of disorders generally comorbid with BPD. Finally, IM–B scores contributed incrementally to the prediction of interpersonal dysfunction and suicidal ideation and behavior. Discussion focuses on implications for the assessment of BPD.     

Data Collection via Online Platforms: Challenges and Recommendations for Future Research

Online platforms such as Amazon's Mechanical Turk (MTurk) are increasingly used by researchers to collect survey and experimental data. Yet, such platforms often represent a tumultuous terrain for both researchers and reviewers. Researchers have to navigate the complexities of obtaining representative samples from online participant cohorts, ensuring data quality, ethically incentivizing participant engagement, and maintaining transparency. Reviewers, on the other hand, have to navigate the complexities of evaluating the efficacy of such data collection and execution efforts in answering important research questions. In order to provide clarity to these issues, this article provides researchers and reviewers with a series of recommendations for effectively executing and evaluating data collection via online platforms, respectively.     

Philosophy and the care of the self: A literature survey

This article reviews a number of recent books and practices that address a renewed interest in the role that philosophy might play in the living of a rich and fulfilling life. The review looks at books addressed to the general public as well as books which discuss such classical and Hellenistic philosophers as took their task to be helping people achieve happiness in life. It then turns to contemporary studies of the self and of wisdom and turns finally to some newly emerging philosophical practices such as philosophical counselling and philosophical discussion groups of various kinds in order to explore whether philosophy can still be a source of consolation or guidance in contemporary life.