Qualitative tools and experimental philosophy

Experimental philosophy brings empirical methods to philosophy. These methods are used to probe how people think about philosophically interesting things such as knowledge, morality, and freedom. This paper explores the contribution that qualitative methods have to make in this enterprise. I argue that qualitative methods have the potential to make a much greater contribution than they have so far. Along the way, I acknowledge a few types of resistance that proponents of qualitative methods in experimental philosophy might encounter, and provide reasons to think they are ill-founded.     

Anti-Naturalism: The Role of Non-Empirical Methods in Philosophy

Some naturalistic conceptions of philosophical methodologies interpret the doctrine that philosophy is continuous with science to mean that philosophical investigations must implement empirical methods and must not depart from the experimental results that the scientific application of those methods reveal. In this paper, I argue that while our answers to philosophical questions are certainly constrained by empirical considerations, this does not imply that the methods by which these questions are correctly settled are wholly captured by empirical methods. Many historical cases of successful answers to philosophical questions involve strategies and methods that allow the divergence from empirically established results. I develop this idea and then illustrate it with Frege's methodology in his Foundations of Arithmetic.     

Sample Size Planning for the Squared Multiple Correlation Coefficient: Accuracy in Parameter Estimation via Narrow Confidence Intervals

Methods of sample size planning are developed from the accuracy in parameter approach in the multiple regression context in order to obtain a sufficiently narrow confidence interval for the population squared multiple correlation coefficient when regressors are random. Approximate and exact methods are developed that provide necessary sample size so that the expected width of the confidence interval will be sufficiently narrow. Modifications of these methods are then developed so that necessary sample size will lead to sufficiently narrow confidence intervals with no less than some desired degree of assurance. Computer routines have been developed and are included within the MBESS R package so that the methods discussed in the article can be implemented. The methods and computer routines are demonstrated using an empirical example linking innovation in the health services industry with previous innovation, personality factors, and group climate characteristics.     

In Defence of Entrapment in Journalism (and Beyond)

The use of 'proactive' methods of newsgathering in journalism is very frequently condemned, from within and without the media. I argue that such condemnation is too hasty. In the first half of the paper, I develop a test which distinguishes between legitimate and illegitimate uses of proactive methods by law enforcement agencies. This test combines the virtues of the standard objective and subjective tests usually used, while avoiding the defects of both. I argue that when proactive methods pass this test, they are always legitimate; but that in addition they are sometimes mandatory. In the second half, I apply this test to journalism. I show that actual uses of proactive methods by journalists pass the test, and are therefore (at least) permissible. There are other ethical considerations which are relevant to the use of such techniques by journalists, which ought to be taken into account before it is decided to employ such methods, but I show that they are rarely of sufficient weight to render proactive newsgathering impermissible.     

Meta-analytic interval estimation for bivariate correlations

The currently available meta-analytic methods for correlations have restrictive assumptions. The fixed-effects methods assume equal population correlations and exhibit poor performance under correlation heterogeneity. The random-effects methods do not assume correlation homogeneity but are based on an equally unrealistic assumption that the selected studies are a random sample from a well-defined superpopulation of study populations. The random-effects methods can accommodate correlation heterogeneity, but these methods do not perform properly in typical applications where the studies are nonrandomly selected. A new fixed-effects meta-analytic confidence interval for bivariate correlations is proposed that is easy to compute and performs well under correlation heterogeneity and nonrandomly selected studies.     

Language production: Methods and methodologies

Methodological problems have been a longstanding barrier to the systematic exploration of issues in language production. Recently, however, production research has broadened beyond traditional observational approaches to include a diverse set of experimental paradigms. This review surveys the observational and experimental methods that are used to study production, the questions to which the methods have been directed, and the theoretical assumptions that the methods embody. Although tailored to the investigation of language production, most of the methods are closely related to others that are widely employed in cognitive research. The common denominator of these procedures is verbal responding. Because the processing complexities of verbal responses are sometimes overlooked in research on memory, perception, attention, and language comprehension, the methodological assumptions of production research have implications for other experimental procedures that are used to elicit spoken words or sentences.     

Methods for describing ordinal data with cardinal models

Several methods, both new and old, for describing ordinal data with a cardinal model are discussed in a newly developed and general context which broadens their applicability beyond their traditional use in multidimensional scaling. The relationships between the methods is investigated. It is shown that the two most commonly used methods (Guttman's rank-image principle and Kruskal's least-square monotonic transformation) are the boundary conditions of a newly proposed single parameter family of methods. An additional method is proposed which is shown to yield an equal-density model space. The commonly made distinction between “transformation” methods and “transformation-free” methods is shown to be a pseudodistinction. It is observed that all the methods can be stated as matrix operations on the model space with the conclusion that they try to optimize a linear combination of the model space.     

An overview of applied robust methods

Data in social sciences are typically non-normally distributed and characterized by heavy tails. However, most widely used methods in social sciences are still based on the analyses of sample means and sample covariances. While these conventional methods continue to be used to address new substantive issues, conclusions reached can be inaccurate or misleading. Although there is no ‘best method’ in practice, robust methods that consider the distribution of the data can perform substantially better than the conventional methods. This article gives an overview of robust procedures, emphasizing a few that have been repeatedly shown to work well for models that are widely used in social and behavioural sciences. Real data examples show how to use the robust methods for latent variable models and for moderated mediation analysis when a regression model contains categorical covariates and product terms. Results and logical analyses indicate that robust methods yield more efficient parameter estimates, more reliable model evaluation, more reliable model/data diagnostics, and more trustworthy conclusions when conducting replication studies. R and SAS programs are provided for routine applications of the recommended robust method.     

Methodological issues in language acquisition research with very young children

It is suggested that the dominance of observational methods in child language research with very young children has limited progress in understanding the language acquisition process. Evidence is presented that, contrary to popular belief, naturalistic methods do not guarantee ecological validity, nor are they free of task biases. Furthermore, observational studies do not necessarily provide better measures of linguistic competence than do more structured methods. Instead, it is argued that new insights from cognitive developmental research are applicable in studying language production in children younger than 3 years. A number of structured techniques are reviewed that have been used or could be used to study language production. It is concluded that a combination of methods is necessary in order to disentangle and control the many variables that enter into the language acquisition process.     

The interpretation of discrepancies in multimethod research: A comparison of children's negotiations in two research contexts

Although multimethod research has a large number of advocates, problems arise in the interpretation of any discrepancies between results. This paper identifies three approaches to their interpretation: hierarchical validity, cross-validity and triangulation. The first assumes that methods can be ranked in order of validity so that lower ranking methods can be validated against higher ranking methods. The second assumes there is no hierarchy and that multiple methods are desirable to overcome the weaknesses in any one method and, further, that only results obtained from more than one method should be given credence. The third position, which differs from the others in that it arises from a constructionist rather than a positivist framework, holds that multimethods are valuable because of the different insights they produce and focuses attention on the need to examine methods in terms of their contribution in the context of a particular theory. These approaches are discussed in reference to results for children's negotiations from mothers' reports versus those from a quasi-experimental puppets task.     

Is the worldview of qualitative inquiry a proper guide for psychological research?

Qualitative methods are becoming increasingly popular in psychology. Although the distinction between qualitative and quantitative often is stated in terms of methods, the real distinction is between worldviews: that favored by most qualitative methodologists, which emphasizes subjective experience and multiple realities, and that commonly accepted in science. The worldview accepted by most adherents of qualitative inquiry suggests the exclusive use of methods that include verbal reports of lived experience. Qualitative methods serve an important function in psychology, but their use as recommended by their adherents is limited in 2 respects: The adherents use a narrow and unconventional approach to qualitative methods that differs from that normally understood, and they favor use of a restricted range of qualitative methods over other qualitative methods and quantitative methods. If qualitative inquiry is to make a greater contribution to psychology, researchers in that tradition must acquire a better understanding of contemporary science, correct their misunderstandings of the rationale for quantitative methods, and address the apparent limitations of their methods emphasizing reported experience.     

权力感的无意识启动:方法研究*

情境性启动法是对权力感进行内隐社会认知研究的重要方法。本研究首先对权力情境性启动的两种启动方法(概念启动和定势启动)进行介绍,接着通过两个实验对两种启动方法在中文背景下的适用性和有效性进行了验证,认为中文材料的概念启动和定势启动是适用和有效的,可以作为国内权力社会认知研究中启动权力的方法。最后对权力概念启动和定势启动的研究范式、方法差异和内在机制进行了讨论。     

The analysis of classroom talk: Methods and methodologies

This article describes methods for analysing classroom talk, comparing their strengths and weaknesses. Both quantitative and qualitative methods are described and assessed for their strengths and weaknesses, with a discussion of the mixed use of such methods. It is acknowledged that particular methods are often embedded in particular methodologies, which are based on specific theories of social action, research paradigms, and disciplines; and so a comparison is made of two contemporary methodologies, linguistic ethnography, and sociocultural research. The article concludes with some comments on the current state of development of this field of research and on ways that it might usefully progress.     

Look-ahead content balancing method in variable-length computerized classification testing

Content balancing is one of the most important issues in computerized classification testing. To adapt to variable-length forms, special treatments are needed to successfully control content constraints without knowledge of test length during the test. To this end, we propose the notions of ‘look-ahead’ and ‘step size’ to adaptively control content constraints in each item selection step. The step size gives a prediction of the number of items to be selected at the current stage, that is, how far we will look ahead. Two look-ahead content balancing (LA-CB) methods, one with a constant step size and another with an adaptive step size, are proposed as feasible solutions to balancing content areas in variable-length computerized classification testing. The proposed LA-CB methods are compared with conventional item selection methods in variable-length tests and are examined with different classification methods. Simulation results show that, integrated with heuristic item selection methods, the proposed LA-CB methods result in fewer constraint violations and can maintain higher classification accuracy. In addition, the LA-CB method with an adaptive step size outperforms that with a constant step size in content management. Furthermore, the LA-CB methods generate higher test efficiency while using the sequential probability ratio test classification method.     

THE EFFECTIVENESS AND TRANSFER VALUE OF TWO METHODS OF CONCEPT LEARNING

WERDELIN, I., BONDESSON, E. & LARSSON, G.-B. The effectiveness and transfer value of two methods of concept learning. Scand. J. Psychol. , 1971, 12, 198–204.–Two methods of learning are compared as to their effectiveness and transfer value, the "direction method", where principles are presented and exemplified, and the "discovery method", where the learner discovers the principles from examples. In an experiment with 12a pupils in grade five it was found that the direction method was superior to the discovery method in these learning situations, that the methods are differently effective with different materials, and that experience with one of the methods may transfer to the ability to learn new materials with either method. The connections between concept, method and previous experience are very complex, however.     

Statistical inference in behavior analysis: Experimental control is better

Statistical inference promises automatic, objective, reliable assessments of data, independent of the skills or biases of the investigator, whereas the single-subject methods favored by behavior analysts often are said to rely too much on the investigator's subjective impressions, particularly in the visual analysis of data. In fact, conventional statistical methods are difficult to apply correctly, even by experts, and the underlying logic of null-hypothesis testing has drawn criticism since its inception. By comparison, single-subject methods foster direct, continuous interaction between investigator and subject and development of strong forms of experimental control that obviate the need for statistical inference. Treatment effects are demonstrated in experimental designs that incorporate replication within and between subjects, and the visual analysis of data is adequate when integrated into such designs. Thus, single-subject methods are ideal for shaping-and maintaining-the kind of experimental practices that will ensure the continued success of behavior analysis.     

Measuring Clients' Cognitive Competence in Research on Counseling

Counseling is defined as an activity concerned with the development of cognitive competence in clients. As part of the overall movement in research on counseling toward methodological pluralism and triangulation, it is suggested that researchers of counseling effects adopt methods of processing and representing clients' cognitive structures. Such methods can help to capture some of the uniquely rich personal and subjective aspects of the change process in counseling. A variety of methods for accessing and representing cognitive structures are presented, some of which are illustrated in a brief case report. The advantages and limitations of the methods presented are discussed.     

Parametric and nonparametric bootstrap methods for meta-analysis

In a meta-analysis, the unknown parameters are often estimated using maximum likelihood, and inferences are based on asymptotic theory. It is assumed that, conditional on study characteristics included in the model, the between-study distribution and the sampling distributions of the effect sizes are normal. In practice, however, samples are finite, and the normality assumption may be violated, possibly resulting in biased estimates and inappropriate standard errors. In this article, we propose two parametric and two nonparametric bootstrap methods that can be used to adjust the results of maximum likelihood estimation in meta-analysis and illustrate them with empirical data. A simulation study, with raw data drawn from normal distributions, reveals that the parametric bootstrap methods and one of the nonparametric methods are generally superior to the ordinary maximum likelihood approach but suffer from a bias/precision tradeoff. We recommend using one of these bootstrap methods, but without applying the bias correction.     

A comparison of interview and self-report methods for the assessment of borderline personality disorder criteria

Interview methods are widely regarded as the standard for the diagnosis of borderline personality disorder (BPD), whereas self-report methods are considered a time-efficient alternative. However, the relative validity of these methods has not been sufficiently tested. The current study used data from the Collaborative Longitudinal Personality disorder Study to compare diagnostic base rates and the relative validity of interview and self-report methods for assessing functional outcome in BPD. Although self-report yielded higher base rates of criteria endorsement, results did not support the common assumption that diagnostic interviews are more valid than self-reports, but instead indicated the combined use of these methods optimally identifies BPD criteria.     

Expanding the aperture of psychological assessment: introduction to the special section on innovative clinical assessment technologies and methods

Contemporary psychological assessment is dominated by tried-and-true methods like clinical interviewing, self-report questionnaires, intellectual assessment, and behavioral observation. These approaches have served as the mainstays of psychological assessment for decades. To be sure, these methods have survived over the years because clinicians have found the information gleaned from these improves our understanding of the client, helps formulate a treatment plan, or aids in assessing outcome. However, are there any recent technological or methodological developments that might improve the clinical assessment enterprise? Some believe so. The purpose of this special section is to provide a snapshot of relatively newer clinical assessment technologies and methods that have been underutilized. Each article in this special section presents one or more assessment methods or techniques that are currently available and are showing some promise in clinical assessment.