Design of randomized controlled trials: Principles and methods applied to a treatment for early stuttering

The “gold standard” methodology for treatment efficacy research is the randomized controlled trial (RCT), which is used extensively in medical research and in other areas such as psychology. Results from a well designed and conducted RCT, which show a new treatment to be clinically and statistically superior to current standard practice, can lead to a change in clinical practice. This paper presents a tutorial on RCTs, presenting and discussing the following principles and methods: the trial protocol, choice of control treatment, eligibility criteria, random allocation, outcomes and hypotheses, sample size, subject recruitment, analysis by intention to treat, interim analysis, stopping rules, safety data monitoring/trial management, and study documentation. RCTs are a complicated and logistically involved methodology. Hence, prior to the commitment of resources to such a trial in the development of a new treatment, the safety and clinical promise of the new treatment needs to be demonstrated with preliminary studies. Such preliminary studies have been completed for the Lidcombe Program (LP) of early stuttering intervention, and an RCT of that treatment is being conducted at the time of writing. The principles and methods of RCTs are illustrated with reference to that study.

Educational objectives: The reader will understand the design principles and methods of RCTs. The reader will understand the logistics of conducting a RCT of a treatment for early stuttering.     

Methods for Practising Ethics in Research and Innovation: A Literature Review,Critical Analysis and Recommendations

This paper provides a systematic literature review, analysis and discussion of methods that are proposed to practise ethics in research and innovation (R&I). Ethical considerations concerning the impacts of R&I are increasingly important, due to the quickening pace of technological innovation and the ubiquitous use of the outcomes of R&I processes in society. For this reason, several methods for practising ethics have been developed in different fields of R&I. The paper first of all presents a systematic search of academic sources that present and discuss such methods. Secondly, it provides a categorisation of these methods according to three main kinds: (1) ex ante methods, dealing with emerging technologies, (2) intra methods, dealing with technology design, and (3) ex post methods, dealing with ethical analysis of existing technologies. Thirdly, it discusses the methods by considering problems in the way they deal with the uncertainty of technological change, ethical technology design, the identification, analysis and resolving of ethical impacts of technologies and stakeholder participation. The results and discussion of our literature review are valuable for gaining an overview of the state of the art and serve as an outline of a future research agenda of methods for practising ethics in R&I.     

How Do You Say Nature?: Opening the Design Space with a Knowledge Environment

In this paper, we describe a design study on how families in Los Angeles experience nature, with a particular focus on how using alternative formats for research analysis and presentation can enrich a design research inquiry. Conducted in Pasadena, CA as a part of Super Studio, the year-long design research class emphasized knowledge building and sharing through design methods of analysis, exhibition, and concept prototypes. After conducting interviews and engaging participants with probes, the class built the results into an open-ended knowledge environment. The main observation from the exhibition – that Angelenos have a limited vocabulary for describing nature and their interaction with it – helped to define the opportunity for design intervention that created connection points between Angelenos and nature. There are few attempts to describe how probe returns evolve into design concepts. Showcased is a process that depends on the unique affordances of design as the vehicle for discovery and invention.     

A Comparison of the Common‐Item and Random‐Groups Equating Designs Using Empirical Data

We designed this study to evaluate several data collection and equating designs in the context of item response theory (IRT) equating. The random‐groups design and the common‐item design have been widely used for collecting data for IRT equating. In this study, we investigated four equating methods based upon these two data collection designs, using empirical data from a number of different testing programs. When the randomly equivalent group assumption was reasonably met, the four equating methods tended to produce highly comparable results. On the other hand, equating methods based upon either of the equating designs produced dissimilar results. Sample size can have differential effects on the equating results produced by the different equating methods. In practice, a common‐item equivalent‐groups design often produces unacceptably large differences in the group mean due to various anomalies such as context effects, poor quality of common items, or a very small number of common items. In such cases, a random‐groups design would produce more stable equating results.     

Clutching at straws: Is it time to abandon the moral judgment deficit explanation for delinquency?

We respond to the commentary by Brusten, Stams, and Gibbs (2007) on the research by Tarry and Emler (2007) , arguing for the appropriateness of key design decisions – studying delinquency as assessed by self‐report in a sample of 12‐ to 15‐year‐old males. We argue that ‘known‐group’ methods for assessing involvement in delinquency, the major alternatives to self‐report, produce divergent results with respect to moral judgment level because these methods are confounded with other influences on moral judgment level, in particular education.     

@Preacher_Bot: an experiment in evangelical speech making

This article is an experiment in digital methodology. It applies tools associated with the digital humanities to the study of Protestant evangelicalism in order to ask what digital research methods can offer the study of religion. Beginning with the development of a social media computer program called a Twitter bot designed to mimic evangelical ministers on twitter, the article considers how digital scholarship allows the work scholars produce to mirror more closely the religious worlds they study. Next, the article analyzes the results of the program using text analysis software. In documenting trends in the bot's output, the article highlights how digital research methods can help answer long-standing questions in the field as well as potentially ask new ones. It concludes, The article, then, is both an exploration of contemporary religious cultures and a consideration of new media methods for analyzing them.     

Integrating clients' moment‐to‐moment ratings into psychotherapy research: A novel approach

Counselling and psychotherapy researchers have considerably advanced the field's understanding of psychotherapy processes and how they relate to treatment outcomes. Despite these advances, little is known about the client's perspective of changes in psychotherapy processes that occur throughout a given session (i.e. micro‐processes). To address this gap, this article describes the novel application of methods that assess participants' moment‐to‐moment ratings to psychotherapy research. This method entails recording psychotherapy session content that clients and other potential raters (e.g. therapists, researchers) later review while simultaneously providing continuous ratings of psychotherapy processes (e.g. helpfulness, alliance). In addition, moment‐to‐moment ratings can facilitate significant events research by prompting researchers to elicit client feedback about the moments that are rated the most and least positively. However, few studies have used these methods in the context of psychotherapy research. Studies incorporating these methods may yield findings that advance psychotherapy research, training efforts and clinical practice. For example, studies may examine how the magnitude and timing of clients' moment‐to‐moment ratings of psychotherapy processes are associated with treatment outcomes, therapist ratings and physiological processes (e.g. heart rate variability). Trainee therapists and their supervisors may also use clients' moment‐to‐moment ratings to facilitate attunement to verbal and non‐verbal indicators of moments perceived more positively and negatively. Last, these methods can produce findings that are highly relevant to clinical practice, where therapists routinely navigate fluctuations in psychotherapy processes (e.g. alliance ruptures) that can be assessed using moment‐to‐moment ratings.     

A systems analysis of a failure to test the effectiveness of waiting-list group therapy.

This nonexperimental effectiveness study attempted to evaluate the utility of a brief waiting-list group. The setting was a university clinic providing treatment for an inner-city population. Health delivery and staff dynamics made it difficult to conduct clinical research in this treatment-oriented setting. The nonrandom design allowed for patient choice, with few clients attending more than two group sessions, thus decreasing its impact. Managed-care pressures decreased staff cooperation with our research objectives, resulting in very low return rates in testing and follow-up data. A social systems analysis, highlighting staff and institutional ambivalence, is used to understand the failure to adequately test the effectiveness of waiting-list group therapy. Recommendations are offered to investigators who contemplate conducting clinical research with limited resources.     

On the complexity of quantifying construct validity

To validate a measure of a construct is to validate a theory: The process includes theory specification, theory elaboration, choice of methods, choice of research design, reliance on necessary auxiliary theories, and empirical observation. Evaluating the success of the process is likely to be enhanced through quantification. To date, scientists have quantified the final step of empirical observation, but earlier steps in the validation process appear more difficult to measure. In this issue, D. Westen and R. Rosenthal (2005) emphasize that threats to validity are often methodological ones that precede empirical observation. Yet, their earlier attempt to quantify construct validity focused only on the final, empirical observation step of the process. For that reason, it can produce positive results despite real theoretical and methodological threats to validity. Nonetheless, the laudable attempt by these distinguished authors to quantify construct validity reminds us of the complexity of the construct validation process.     

服务型领导影响工作繁荣的动态双向机制

本研究基于资源保存理论, 采用动态追踪设计与双向影响设计, 通过文献研究和追踪问卷调查研究等方法, 揭示服务型领导对个人及团队工作繁荣影响的跨层作用机制、边界条件以及服务型领导与下属的双向影响过程, 从而增进对服务型领导作用机制的理解, 深化对领导本身动态变化本质的理解, 拓展工作繁荣前因变量的研究, 也为各类组织改善领导方式、提高领导效能、促进员工工作繁荣提供指导。     

Modern robust statistical methods: an easy way to maximize the accuracy and power of your research

Classic parametric statistical significance tests, such as analysis of variance and least squares regression, are widely used by researchers in many disciplines, including psychology. For classic parametric tests to produce accurate results, the assumptions underlying them (e.g., normality and homoscedasticity) must be satisfied. These assumptions are rarely met when analyzing real data. The use of classic parametric methods with violated assumptions can result in the inaccurate computation of p values, effect sizes, and confidence intervals. This may lead to substantive errors in the interpretation of data. Many modern robust statistical methods alleviate the problems inherent in using parametric methods with violated assumptions, yet modern methods are rarely used by researchers. The authors examine why this is the case, arguing that most researchers are unaware of the serious limitations of classic methods and are unfamiliar with modern alternatives. A range of modern robust and rank-based significance tests suitable for analyzing a wide range of designs is introduced. Practical advice on conducting modern analyses using software such as SPSS, SAS, and R is provided. The authors conclude by discussing robust effect size indices.     

Using dual-criteria methods to supplement visual inspection: Replication and extension

The dual-criteria and conservative dual-criteria methods effectively supplement visual analysis with both simulated and published datasets. However, extant research evaluating the probability of observing false positive outcomes with published data may be affected by case selection bias and publication bias. Thus, the probability of obtaining false positive outcomes using these methods with data collected in the course of clinical care is unknown. We extracted baseline data from clinical datasets using a consecutive controlled case-series design and calculated the proportion of false positive outcomes for baseline phases of various lengths. Results replicated previous findings from Lanovaz, Huxley, and Dufour (2017), as the proportion of false positive outcomes generally decreased as the number of points in Phase B (but not Phase A) increased using both methods. Extending these findings, results also revealed differences in the rate of false positive outcomes across different types of baselines.     

Structuring information interfaces for procedural learning

Interface design should be informed by the application of top-down cognitive principles derived from basic theory and research. Cognitive design principles from 2 domains, event cognition and media, were applied to the design of interfaces for teaching procedures. According to theories of event cognition, procedures should be presented hierarchically, organized by objects or large object parts and actions on objects. According to research on effects of media, adding appropriate graphics to text instructions can facilitate learning and memory. These principles were partially supported in 2 tasks: assembling a musical instrument and building a model. Although both top-down principles were effective in guiding interface design, they were not sufficient. They can be combined with iterative bottom-up methods to produce usable interfaces.     

Sidman or statistics?

Murray Sidman's statements regarding variability, experimental control, and generality are interwoven with examples from the literature on conditional discrimination. Sidman's position was that statistical inferences from group studies produce no information about the behavior of individual subjects and that statistical treatment of individual subject data masks variability which may represent conditions that are not controlled. Sidman's work on conditional discrimination provides excellent examples of how complex discriminations should be examined in detail with accuracy levels obtained for each type of discrimination within an experiment. Sidman made important contributions to the foundation of behavior analysis with extensive basic research as well as applications of methods and principles to clinical and educational settings.     

冥想对注意能力的影响

所有冥想方法都要对注意进行管理, 注意是冥想的核心机制。近期研究发现, 冥想显著提高了持续性注意、执行注意和选择性注意分配等多种注意能力。认知神经科学研究发现, 冥想使注意资源分配更有效; 长期冥想者表现出更高的失匹配负波波幅和更低的β波幅; 冥想增强了中央执行网络有关脑区的功能联结, 提高了突出网络有关脑区的活动; 注意聚焦冥想降低了默认网络有关脑区的活动。另外, 作为一种提高注意能力的方法, 冥想不仅适用于注意缺陷多动障碍患者和已康复的抑郁症患者等有注意相关障碍的临床群体, 也适用于儿童和老年人等多种年龄阶段的健康群体。未来应通过追踪研究探讨冥想的长期效果; 探索冥想中注意与情绪的交互作用; 根据群体特征设计出更有针对性的冥想。     

Gender‐of‐author differences in study design of older adult friendship surveys

This research extends literature documenting gender differences in choice of quantitative or qualitative methods by examining whether men and women make different decisions regarding study design within one type of quantitative method—survey research. The population consists of 117 surveys (1960–1999) of older adults about friendship. We find that the use of communal methods, especially by women scholars, increased over time, and researchers who are women are more likely to use “communal” survey methods than researchers who are men. The gender difference in use of communal methods is not necessarily explained by essential differences between men and women or even by gender differences in commitment to feminist research as previous scholars have argued; rather, it is possibly explained by structured differences in access to resources to do agentic research.     

Clinical Aspects of Transgender Speech Feminization and Masculinization

SUMMARY

Societal norms of speech, voice, and non-verbal communication are often strongly gendered. For transgender individuals who experience a mismatch between existing communication behaviours and felt sense of self, changes to the gendered aspects of communication can help reduce gender dysphoria, improving mental health and quality of life. While peer resources are often beneficial in changing overall appearance and presentation, speech and voice modification is best facilitated by a trans-competent speech professional. In this article we review clinical research relating to transgender speech and voice change and discuss clinical protocols for trans-specific assessment, treatment, and outcome evaluation.     

The Relevance of Counseling Outcome Research: Current Trends and Practical Implications

This systematic review investigated current trends in the publication sources, research focus, and methods of 365 counseling outcome studies. The results suggest that outcome research can provide a considerable knowledge base regarding numerous client concerns and various clinical interventions in individual, career, and school counseling. Counseling practitioners with a thorough knowledge of outcome research methods who use the findings appropriately will find this literature a useful adjunct to clinical decision making. Suggestions for improving the use of counseling research are offered for both practitioners and researchers.     

Decision resource consultancy: a convergence of disciplines that aid consumer marketing

When companies recognise the limitations of seat‐of‐pants decision making, it is not always immediately clear where to turn in order to procure the information and advice that can result in better business decisions. The chosen resource might be a research agency (assumed to be fast‐acting); a management consultancy (likely to be reassuringly expensive); or sources in the academic world (popularly seen as ‘leading‐edge’). What is delivered will depend on the type of supplier chosen: so far, each of these resources has developed in isolation, with its own assumptions, methods and codes of conduct. Nowhere is this more starkly seen than among providers of qualitative research, the discipline that gave birth to focus groups and marketing‐oriented depth interviews. Depending on the supplier, the client may be provided with research that is process‐based, skills‐based or knowledge‐based. The design, practice and analysis of the research will differ accordingly. In this paper, it is argued that the time has come to establish a new form of business consultancy which understands the strengths and weaknesses of these three traditional approaches, which can offer research brokerage that makes available the most appropriate techniques, and which can bring together the distinctive offerings of market‐research agencies, universities and management consultancies as a total resource in helping clients to make optimal decisions. An acceptance of new kinds of professional alliances will be required, as ‘meta‐consultants’ look for cooperation, not competition, among those who are based in each of these three domains. Copyright © 2002 Henry Stewart Publications.