Comparing Paper-and-Pencil and Internet Survey Methods Conducted in a Combat-Deployed Environment

The Unit Behavioral Health Needs Assessment Survey (UBHNAS) is a tool used to aid behavioral health providers in assessing the needs of service members of a military unit. Traditionally, the survey has been administered by paper-and-pencil; however, implementing the survey via the Internet may be a faster and easier way to administer the survey. There are, however, potential drawbacks to Internet survey administration, and previous studies comparing the 2 methods have yielded inconsistent findings. Using a between-subjects design with a combat-deployed military sample, the present study compared the 2 survey administration methods. Specifically, we assessed comparability for (a) psychological constructs, (b) auxiliary measures (such as completion time, frequency of missing data, and length of written comments), and (c) ratings and perceptions of the UBHNAS, and preference in methodology. The method of survey administration did not predict differences in the psychological outcome variables. However, participants in the paper-and-pencil condition reported higher rates of comfort answering questions, belief that responses would remain anonymous, and frankness and honesty in answering questions. Nevertheless, participants in both conditions preferred to take the survey via the Internet as opposed to paper-and-pencil in the future. Finally, participants in the Internet condition completed the survey faster and tended to write longer comments, although in terms of the comments being informative, there was no difference between the groups. Based on our findings, we recommend the use of the Internet for administering the UBHNAS; however, we caution that there may be tradeoffs regarding concerns about comfort, honesty, and anonymity.     

Personality research on the Internet: A comparison of Web-based and traditional instruments in take-home and in-class settings

Students, faculty, and researchers have become increasingly comfortable with the Internet, and many of them are interestedin using the Web to collectdata. Few published studies have investigated the differences between Web-based data and data collected with more traditional methods. In order to investigate these potential differences, two important factors were crossed in this study: whether the data were collected on line or not and whether the data were collected in a group setting at a fixed time or individually at a time of the respondent’s choosing. The Visions of Morality scale (Shelton & McAdams, 1990) was used, and the participants were assigned to one of four conditions: in-class Web survey, in-class paper-and-pencil survey; take-home Web survey, and take-home paper-and-pencil survey. No significant differences in scores were found for any condition; however, response rates were affected by the type of survey administered, with the take-home Web-based instrument having the lowest response rate. Therefore, researchers need to be aware that different modes of administration may affect subject attrition and may, therefore, confound investigations of other independent variables.     

The Web Experimental Psychology Lab: Five years of data collection on the Internet

In fall 1995, the worldwide-accessible Web Experimental Psychology Lab (http://www.genpsylab. unizh.ch) opened its doors to Web surfers and Web experimenters. It offers a frequently visited place at which to conduct true experiments over the Internet. Data from 5 years of laboratory running time are presented, along with recommendations for setting up and maintaining a virtual laboratory, including sections on the history of the Web laboratory and of Web experimenting, the laboratory’s structure and design, visitor demographics, the Kids’ Experimental Psychology Lab, access statistics, administration, software and hardware, marketing, other Web laboratories, data security, and data quality. It is concluded that experimental data collection via the Internet has proven to be an enrichment to science. Consequently, the Web Experimental Psychology Lab will continue and extend its services to the scientific community.     

A validation of well-being and happiness surveys for administration via the Internet

Internet research is appealing because it is a cost- and time-efficient way to access a large number of participants; however, the validity of Internet research for important subjective well-being (SWB) surveys has not been adequately assessed. The goal of the present study was to validate the Satisfaction With Life Scale (SWLS; Diener, Emmons, Larsen, & Griffin, 1985), the Positive and Negative Affect Schedule (PANAS-X; Watson & Clark, 1994), and the Subjective Happiness Scale (SHS; Lyubomirsky & Lepper, 1999) for use on the Internet. This study compared the quality of data collected using paper-based (paper-and-pencil version in a lab setting), computer-based (Web-based version in a lab setting), and Internet (Web-based version on a computer of the participant’s choosing) surveys for these three measures of SWB. The paper-based and computer-based experiment recruited two college student samples; the Internet experiments recruited a college student sample and an adult sample responding to ads on different social-networking Web sites. This study provides support for the reliability, validity, and generalizability of the Internet format of the SWLS, PANAS-X, and SHS. Across the three experiments, the results indicate that the computer-based and Internet surveys had means, standard deviations, reliabilities, and factor structures that were similar to those of the paper-based versions. The discussion examines the difficulty of higher attrition for the Internet version, the need to examine reverse-coded items in the future, and the possibility that unhappy individuals are more likely to participate in Internet surveys of SWB.     

Measuring health beliefs on the Internet: A comparison of paper and Internet administrations of the Multidimensional Health Locus of Control Scale

A growing number of studies have supported the use of unidimensional psychometric test instruments administered via the Internet; however, support for the use of multidimensional scales is weak. The present study compares paper and Internet administrations of the Multidimensional Health Locus of Control (MHLC) Scale (Wallston & Wallston, 1981). In terms of reliabilities and factor structures, the Internet data were found to be at least as good as the paper data. MHLC scores were comparable for paper and Internet administrations, although the Internet sample scored significantly lower on the Powerful Others subscale. Overall, the results show that administration of the MHLC Scale via the Internet can produce data comparable to that obtained by pen-and-paper methods. However, it is concluded that generalization of these findings beyond the psychometric test instrument and sampling procedures used here is not warranted.     

Gender and the Internet: Women Communicating and Men Searching

This research examined gender differences in Internet use and factors responsible for these differences. A sample of 630 Anglo American undergraduates completed the Student Computer and Internet Survey that contained questions about e-mail and Web use, and about potential affective and cognitive mediators of use. Based on a general model of Internet use, we predicted and found that females used e-mail more than did males, males used the Web more than did females, and females reported more computer anxiety, less computer self-efficacy, and less favorable and less stereotypic computer attitudes. Path analysis to identify mediators of gender differences in Internet use revealed that computer self-efficacy, loneliness, and depression accounted in part for gender differences, but that gender continued to have a direct effect on use after these factors were considered. Implications for realizing the democratizing potential and benefits of Internet use are discussed.     

Prevalence of Internet addiction in Latino adolescents with psychiatric diagnosis

Internet addiction (IA) is particularly relevant in the adolescent population. The aim of this study was to describe the prevalence of IA in a clinical sample of Latino adolescents receiving ambulatory psychiatric treatment. The correlation between their pattern of Internet use and their respective psychiatric diagnosis was also studied. Adolescent patients from the Psychiatric Ambulatory Clinic at the Pediatric University Hospital (N=71) completed the Internet Addiction Test (IAT) and a questionnaire about Internet use. Information regarding demographic and diagnostic data was retrieved from their clinical records. None of the subjects presented severe IA. A total of 71.8% (n=51) of the adolescents obtained scores reflecting no problem related to IA. Only 11.6% (n=5) of subjects have discussed Internet use with their therapist. Mood disorders showed a statistically significant (p=0.044) correlation with a higher score on the IAT. Mental health care practitioners must consider questions on Internet use as an essential part of the patients' evaluation given its significant correlation with diagnosis of a mood disorder.     

Emotional Stroop via Internet among individuals with eating disorders

The present study investigated whether Stroop interference in eating disordered individuals for food - and body-related words, as repeatedly found previously using standard emotional Stroop tasks, would also be demonstrated when using an Internet based emotional Stroop task. Participants were anorexic women (n= 13), bulimic women (n= 20), non-clinically eating disordered women nevertheless over-concerned about eating and body appearance (n= 27) and normal control women (n= 31). Bulimic individuals showed Stroop interference for body-related words whereas anorexic individuals showed Stroop interference for food-related words. The present results thus suggest that administration of the emotional Stroop task is possible via the Internet. Furthermore, it is possible that the time consuming response delivery, relative to previous studies, could lead to Stroop interference only for the most emotionally significant information, in turn, differentiating between the core concerns of anorexic and bulimic individuals.     

网络成瘾倾向者在视觉工作记忆引导下的注意捕获

采用工作记忆任务和视觉搜索任务相结合的双任务范式, 探讨网络成瘾倾向者在视觉工作记忆引导下的注意捕获。实验1考察了单一分心刺激视场中分心刺激的性质对网络成瘾倾向者选择性注意的影响, 实验2通过控制匹配试验出现的概率来诱发不同的抑制动机, 探讨多分心刺激视场中两种抑制动机下网络成瘾倾向者的注意表现。结果发现：(1)无论在单一分心刺激还是多分心刺激视场中, 网络成瘾倾向被试的目标搜索反应时均显著短于正常组被试, 且两组的搜索正确率没有差异。(2)在单一分心刺激视场中, 无论是与工作记忆项目匹配还是不匹配的分心刺激都会捕获正常组被试的注意, 但不会捕获网络成瘾倾向被试的注意。(3)在多分心刺激视场中, 当抑制动机水平较低时, 两组被试均对匹配分心物产生注意捕获效应, 且网络成瘾倾向被试受工作记忆引导的注意捕获效应小于正常组被试; 当抑制动机较高时, 两组被试均对匹配分心物产生注意抑制效应, 且没有差异。研究结果表明, 面对非网络相关视觉刺激时, 网络成瘾倾向者受工作记忆引导的注意捕获效应小于正常组, 并表现出了知觉加工上的优势。     

Selection Testing via the Internet: Practical Considerations and Exploratory Empirical Findings*

Despite a growing body of applied research on using the Internet for some human resource management practices, few studies have provided equivalence information or practical lessons concerning selection testing via the Internet. We identify several issues associated with measurement and validity, the role of several individual characteristics, respondents' reactions and behaviors, and other considerations concerning Internet test administration. We also report results from an exploratory study of the correlation between paper‐and‐pencil and Internet‐administered cognitively oriented selection tests (including timed and untimed, proctored tests). Our empirical results suggest modest degrees of cross‐mode equivalence for an untimed situational judgment test (r= .84) and for a timed cognitive ability test (r= .60). Further, some types of items (math, verbal, spatial) in the timed cognitive ability test seem to play a differential role in the reduced cross‐mode equivalence. New issues regarding the perception of, and reaction to, items presented via the Internet are presented, and a variety of practical issues are derived and discussed.     

Internet administration of paper-and-pencil questionnaires used in couple research: assessing psychometric equivalence

This study examined the psychometric equivalence of paper-and-pencil and Internet formats of key questionnaires used in couple research. Self-report questionnaires assessing interpersonal constructs (relationship satisfaction, communication/conflict management, partner support, emotional intimacy) and intrapersonal constructs (individual traits, psychological symptoms, contextual influences) were administered to young adults in committed dating relationships. The same measures were administered twice via paper-and-pencil and/or Internet methods over a 2-week period. Method order was counterbalanced among participants, and temporal stability was controlled. Intrapersonal and interpersonal measures generally remained reliable when administered online and demonstrated quantitative and qualitative equivalence across methods. The implications of online administration of questionnaires are discussed, and specific recommendations are made for researchers who wish to transition to online data collection.     

Internet research: Self-monitoring and judgments of attractiveness

Two studies examined the relationship between self-monitoring and factors influencing romantic attraction to others. In Study 1, participants completed an Internet-mediated version of the Self-Monitoring Scale (Gangestad & Snyder, 1985) and indicated which of two people (one physically attractive, one with a more desirable personality) they found most attractive. Results matched previous findings (Snyder, Berscheid, & Glick, 1985), but the effect was smaller. Study 2, a paper-and-pencil replication of Study 1, examined whether the weaker effect was due to Internet mediation and found no differences in the choices made by high and low self-monitors. Results suggested that while determinants of attraction may vary for different populations, Internet research methods can tap the same phenomena as traditional laboratory studies.     

Inadvertent exposure to pornography on the Internet: Implications of peer-to-peer file-sharing networks for child development and families

A request from the National Academies to prepare a presentation for a Workshop on Non-Technical Strategies to Protect Youth from Inappropriate Material on the Internet occurred before much was known about children, youth, and the Internet. The author's strategy was to investigate websites that cater to children and adolescents. The developmental issue of consumer socialization was raised by a visit to the Disney website. In contrast, the developmental issues of sexuality, aggression, and intergroup relations were raised by visits to chat rooms hosted by two different Internet Web portals. Examination of existing research literature, in conjunction with visits to the websites, led to the following conclusions: (1) Many parents are aware of the problems of making children the targets of commerce; however, they are quite unaware of the kind of social and cultural worlds young people are creating online. (2) Children and adolescents are not simply the targets of adult Internet creations; they are active participants in creating their own cybercultures, for example, in teen chat rooms. (3) The nature and norms of these cultures can be very much influenced by adult rules, regulations, and participatory monitoring. (4) The important developmental issues raised by this new medium are not unique to the Internet. For example, each psychosocial phenomenon from consumerism to sexuality to aggression has important manifestations in the culture at large.     

Relationship between Facebook use and problematic Internet use among college students

The popularity of Facebook and other online social-networking sites has led to research on the potential risks of use, including Internet addiction. Previous studies have reported that between 8 percent and 50 percent of college students report problems consistent with Internet addiction. The current study assessed a range of variables related to Facebook use, and sought to determine how the use of Facebook relates to problematic Internet use. Undergraduate participants (N=281, 72 percent women) completed a battery of self-report measures, including the Internet Addiction Test, via an online interface. The results of the current study suggest that a sizable minority of students experience problems related to Internet use and that the use of Facebook may contribute to the severity of symptoms associated with Internet addiction.     

Statistical power for the two-factor repeated measures ANOVA

Determining a priori power for univariate repeated measures (RM) ANOVA designs with two or more within-subjects factors that have different correlational patterns between the factors is currently difficult due to the unavailability of accurate methods to estimate the error variances used in power calculations. The main objective of this study was to determine the effect of the correlation between the levels in one RM factor on the power of the other RM factor. Monte Carlo simulation procedures were used to estimate power for the A, B, and AB tests of a 2×3, a 2×6, a 2×9, a 3×3, a 3×6, and a 3×9 design under varying experimental conditions of effect size (small, medium, and large), average correlation (.4 and .8), alpha (.01 and .05), and sample size (n = 5, 10 ,15, 20, 25, and 30). Results indicated that the greater the magnitude of the differences between the average correlation among the levels of Factor A and the average correlation in the AB matrix, the lower the power for Factor B (and vice versa). Equations for estimating the error variance of each test of the two-way model were constructed by examining power and mean square error trends across different correlation matrices. Support for the accuracy of these formulae is given, thus allowing for direct analytic power calculations in future studies.     

The three-factor model of Internet addiction: The development of the Problematic Internet Use Questionnaire

Despite the fact that more and more clinical case studies and research reports have been published on the increasing problem of Internet addiction, no generally accepted standardized tool is available to measure problematic Internet use or Internet addiction. The aim of our study was to create such a questionnaire. On the basis of earlier studies and our previous experience withYoung's (1998a) Internet Addiction Test, initially, we created a 30-item questionnaire, which was assessed together with other questions regarding participants' Internet use. Data were collected online from 1,037 persons (54.1% of them male; mean age, 23.3 years; SD, 9.1). As a result of reliability analysis and factor analysis, we reduced the number of items to 18 and created the Problematic Internet Use Questionnaire (PIUQ) containing three subscales: obsession, neglect, and control disorder. Cronbach's alpha of the PIUQ is .87 (Cronbach's alpha of the subscales is .85, .74, and .76, respectively). The test-retest correlation of the PIUQ is .90. The PIUQ proved to be a reliable measurement for assessing the extent of problems caused by the "misuse" of the Internet; however, further analysis is needed.     

Measuring implicit alcohol associations via the Internet: Validation of Web-based implicit association tests

Given the potential advantages of online assessment of implicit alcohol-related cognitive processes, the goal of this study was to empirically validate the online administration of the implicit association test (IAT). First, we examined whether an Internet-delivered IAT programmed in Flash can be as effectively used to assess implicit alcohol-related associations as equivalent IAT versions that are programmed in local lab software, such as Inquisit. Second, participants performed the IAT versions once in the controlled laboratory setting and once on their home computers via the Internet. Findings with the alcohol IAT versions were robust and did not vary systematically with respect to setting (home or lab) or assessment software (Flash or Inquisit). Importantly, there were also indications that IAT versions performed at home were more strongly related to explicit measures and drinking behavior than were lab-based IAT versions. Together, these findings demonstrate that the alcohol IAT can be validly administered online via participants’ home computers.     

网络论坛情境中的人际知觉准确性及影响因素

本研究彩网上自然实验法,以MBTI价格类型量表作为价格评价工具,探讨了网络论坛情境人际知觉准确性最高,J-P维度的知觉准确性次之,E-I维度的知觉准确性最低。（3）相关分析与回归分析的结果表明,影响人际知觉准确性的因素主要来自四个方面：知觉者、知觉对象的判断、以及两者的相似性。