Web-Based Surveys: Best Practices Based on the Research Literature

ABSTRACT

Over the last 10 years, Web-based surveys have developed a reputation as an easy and economical method of collecting data. However, as with all research methods, there is a need for careful design and planning. This article reviews the literature on the use and effectiveness of Web surveys. It explores the similarities and differences between Web surveys and postal or telephone surveys, the opportunities and limitations of Web surveys, and the use of Web surveys in visitor studies research. Design tips are provided to assist users to make the best possible use of this important new tool.     

A human-centered approach for designing World-Wide Web browsers

Because the World-Wide Web has features that are not found in traditional information systems, it has some unique problems pertaining to human use, such as the difficulty of searching for specific information. This paper describes human cognitive limitations pertinent to the design of Web browsers and provides a conceptual model for human information processing on the Web. It also compares user strategies and use of browser functions for two tasks that differ in the specificity of the information that is to be retrieved. General principles and specific recommendations for the design of Web browsers are derived from these considerations.     

Operate your own World-Wide Web server

Although many researchers wishing to use the World-Wide Web for academic purposes rely on centralized Web services, they should be aware that it is neither expensive nor difficult to operate their own server. Doing so provides research-related benefits such as complete control over their host name and documents provided, the guaranteed ability to execute common gateway interface and server-side include programs, immediate access to their collected data, and the ability to better control who participates in their experiments. This paper surveys Web-server software features likely to be of interest to psychologists and conceptually summarizes their operation and use. The basic steps required to set up a Web server on popular microcomputers are reviewed, and security issues concerning Web-server operation are discussed. An accompanying resource Web page can assist users in setting up their own servers.     

Supporting Grass Roots Implementation of an Evidence-Based Psychotherapy Through a Virtual Community of Practice: A Case Example in the Department of Veterans Affairs

In contrast to a top-down dissemination approach, the present approach identified existing clinicians using an evidence-based psychotherapy (EBP), dialectical behavior therapy, throughout the Department of Veterans Affairs (VA) system, and created a virtual community of practice (VCoP) to facilitate sharing of materials, resources, information, and clinical expertise. This “bottom-up” or grassroots approach to building a VCoP has potential to support the adoption of EBPs in settings without the resources required to support a large-scale training and consultation initiative. The objective of the current paper is to describe creation, dissemination, and web analytics data of this VCoP to connect providers using an EBP in a national health care system. The VCoP was created using SharePoint and development included user acceptance testing. Dissemination was achieved through a variety of outlets, including use of e-mail listservs, promotion on internal Web sites, and presentations. Monthly Web analytics data were collected on number of unique users, location of users, and number of requests or visits. Monthly Web analytics data indicated that the site was visited every workday in the first year and consistently across 5 years. Number of unique users and number of requests was relatively stable with increases corresponding to efforts to increase site engagement. Location of users demonstrates a wide reach across the VA system. Web analytics data indicate sustained use of the site by providers and methods to facilitate and increase use and interaction appeared to have a positive impact. Future research should investigate the effectiveness of such a strategy on provider behavior.     

A Web-based digitized video image system for the study of motor coordination

A Web-based instructional project for the study of motor coordination is presented. The project includes two components: a tutorial on motor coordination and an interactive digitized video image system developed for the analysis of motion, including topics in biomechanics, motor coordination, motor control, and motor skill. This interactive system allows for the quantitative analysis of stored video images that are accessible from the World-Wide Web. The use of this digitizing system for the measurement and analysis of human locomotion coordination patterns is presented.     

A perception experiment with time-critical graphics animation on the World-Wide Web

The World-Wide Web offers a potentially interesting tool to collect data from a large and heterogeneous audience. While questionnaires have become rather common on the Internet, its potential reaches far beyond text processing. In principle, it is possible not only to perform interactive, dynamic experiments on the Web, but also to include graphical animation and time-critical responses, such as reaction times. We implemented a visual motion extrapolation task on the Web using the programming language Java, which can be interpreted by standard Web browsers such as Netscape or Internet Explorer. The data collected with this method turned out to be reliable and differed little from data obtained in a controlled laboratory setting, with the exception of conditions with fixation instruction. Thus, the Web can, generally speaking, be used for data collection of large sample sizes. The strengths and weaknesses of dynamic visual simulation experiments on the Internet are discussed.     

Codes of online sexuality: Celebrity,gender and marketing on the web

Marketers use the combination of sex and celebrity to sell movies, music, and more. this qualitative project uses content analysis to track sexualized images on the official Web sites and fan sites of 41 celebrities. An official Web site may construct a celebrity’s sexuality as part of overall marketing, with all images as tightly controlled as trademarked images. Over time, these celebrity images become a public sexual iconography. While such images have been studied in other mass media, few studies have scrutinized sexuality on mainstream Web sites. This analysis finds that female musicians are more likely than other celebrities to be represented by sexualized images, and that all types of female celebrities are sexualized more than males. In general, a female celebrity who uses sexuality to promote her public image wears this sexuality like a uniform on both official and fan Web sites.     

Human use regulatory affairs advisor (HURAA): learning about research ethics with intelligent learning modules.

The Human Use Regulatory Affairs Advisor (HURAA) is a Web-based facility that provides help and training on the ethical use of human subjects in research, based on documents and regulations in United States federal agencies. HURAA has a number of standard features of conventional Web facilities and computer-based training, such as hypertext, multimedia, help modules, glossaries, archives, links to other sites, and page-turning didactic instruction. HURAA also has these intelligent features: (1) an animated conversational agent that serves as a navigational guide for the Web facility, (2) lessons with case-based and explanation-based reasoning, (3) document retrieval through natural language queries, and (4) a context-sensitive Frequently Asked Questions segment, called Point & Query. This article describes the functional learning components of HURAA, specifies its computational architecture, and summarizes empirical tests of the facility on learners.     

A web-based family problem-solving intervention for families of children with traumatic brain injury.

We developed a Web-based intervention for pediatric traumatic brain injury (TBI) and examined its feasibility for participants with limited computer experience. Six families, including parents, siblings, and children with TBI, were given computers, Web cameras, and high-speed Internet access. Weekly videoconferences with the therapist were conducted after participants completed on-line interactive experiences on problem solving, communication, and TBI-specific behavior management. Families were assigned to videoconference with NetMeeting (iBOT cameras) or ViaVideo. Participants ranked the Web site and videoconferences as moderately to very easy to use. ViaVideo participants rated videoconferencing significantly more favorably relative to face-to-face meetings than did NetMeeting participants. Both the Web site and videoconferencing were rated as very helpful. All families demonstrated improved outcomes on one or more target behaviors, including increased understanding of the injury and improved parent-child relationships. All parents and siblings and all but 1 child with TBI said they would recommend the program to others. We conclude that a face-to-face intervention can be successfully adapted to the Web for families with varied computer experience.     

Interval-level measurement with visual analogue scales in Internet-based research: VAS Generator

The present article describes VAS Generator (www.vasgenerator.net), a free Web service for creating a wide range of visual analogue scales that can be used as measurement devices in Web surveys and Web experimentation, as well as for local computerized assessment. A step-by-step example for creating and implementing a visual analogue scale with visual feedback is given. VAS Generator and the scales it generates work independently of platforms and use the underlying languages HTML and JavaScript. Results from a validation study with 355 participants are reported and show that the scales generated with VAS Generator approximate an interval-scale level. In light of previous research on visual analogue versus categorical (e.g., radio button) scales in Internet-based research, we conclude that categorical scales only reach ordinal-scale level, and thus visual analogue scales are to be preferred whenever possible.     

Virtually Jewish

SUMMARY

The authors created a Web-based tutorial for novice-to-intermediate users of the World Wide Web using primarily Jewish content in order to encourage potential Jewish users of the World Wide Web. The lessons contain information on Jewish-themed Web sites and progressively complex information about how to use the Web. Tips and practice exercises are included.     

A Web-based family problem-solving intervention for families of children with traumatic brain injury

We developed a Web-based intervention for pediatric traumatic brain injury (TBI) and examined its feasibility for participants with limited computer experience. Six families, including parents, siblings, and children with TBI, were given computers, Web cameras, and high-speed Internet access. Weekly videoconferences with the therapist were conducted after participants completed on-line interactive experiences on problem solving, communication, and TBI-specific behavior management. Families were assigned to videoconference with NetMeeting (iBOT cameras) or ViaVideo. Participants ranked the Web site and videoconferences as moderately to very easy to use. ViaVideo participants rated videoconferencing significantly more favorably relative to face-to-face meetings than did NetMeeting participants. Both the Web site and videoconferencing were rated as very helpful. All families demonstrated improved outcomes on one or more target behaviors, including increased understanding of the injury and improved parent-child relationships. All parents and siblings and all but 1 child with TBI said they would recommend the program to others. We conclude that a face-to-face intervention can be successfully adapted to the Web for families with varied computer experience.     

Socially desirable responding on the web: investigating the candor hypothesis

The investigation presented here explores the hypothesis that participants are less likely to respond in a socially desirable fashion on self-report questionnaires completed on the Web relative to those completed in the laboratory--the candor hypothesis. A battery of social desirability questionnaires (i.e., Balanced Inventory of Desirable Responding [Paulhaus, 1984], Marlowe-Crowne Social Desirability Scale [Crowne & Marlowe, 1964], Eysenck Personality Questionnaire-Revised Lie Scale [Eysenck & Eysenck, 1994]) was administered to 3 groups: 2 groups consisted of undergraduate participants who were randomly assigned to complete the measures either in the laboratory (n = 60) or on the Web (n = 60), and 1 group consisted of self-selected participants who visited our experimental Web page and completed the measures online (n = 284). This design allowed us to assess the role of Web administration while controlling for differences in sample type, an oft-neglected issue in the Web literature. Results do not support the claim that administering self-report measures over the Web results in a decrease in socially desirable responding. Furthermore, these findings highlight the problems associated with confounding sample and medium. Implications for the use of Web as a research tool are discussed.     

The international personality item pool and the future of public-domain personality measures

Seven experts on personality measurement here discuss the viability of public-domain personality measures, focusing on the International Personality Item Pool (IPIP) as a prototype. Since its inception in 1996, the use of items and scales from the IPIP has increased dramatically. Items from the IPIP have been translated from English into more than 25 other languages. Currently over 80 publications using IPIP scales are listed at the IPIP Web site (http://ipip.ori.org), and the rate of IPIP-related publications has been increasing rapidly. The growing popularity of the IPIP can be attributed to five factors: (1) It is cost free; (2) its items can be obtained instantaneously via the Internet; (3) it includes over 2000 items, all easily available for inspection; (4) scoring keys for IPIP scales are provided; and (5) its items can be presented in any order, interspersed with other items, reworded, translated into other languages, and administered on the World Wide Web without asking permission of anyone. The unrestricted availability of the IPIP raises concerns about possible misuse by unqualified persons, and the freedom of researchers to use the IPIP in idiosyncratic ways raises the possibility of fragmentation rather than scientific unification in personality research.     

Human use regulatory affairs advisor (HURAA): Learning about research ethics with intelligent learning modules

The Human Use Regulatory Affairs Advisor (HURAA) is a Web-based facility that provides help and training on the ethical use of human subjects in research, based on documents and regulations in United States federal agencies. HURAA has a number of standard features of conventional Web facilities and computer-based training, such as hypertext, multimedia, help modules, glossaries, archives, links to other sites, and page-turning didactic instruction. HURAA also has these intelligent features: (1) an animated conversational agent that serves as a navigational guide for the Web facility, (2) lessons with case-based and explanation-based reasoning, (3) document retrieval through natural language queries, and (4) a context-sensitive Frequently Asked Questions segment, calledPoint & Query. This article describes the functional learning components of HURAA, specifies its computational architecture, and summarizes empirical tests of the facility on learners.