A comparison of sources of baseline data for treatments of problem behavior following a functional analysis

It is common practice in research on the treatment of problem behavior to compare levels of targeted behaviors during treatment to levels when treatment is not in place. Some researchers use data collected as part of a multielement functional analysis as the initial baseline, whereas others collect new baseline data following completion of the functional analysis. We evaluated whether the source of baseline data influences the reliability and efficiency of decision-making. Results suggest that similar decisions are made in regard to treatment efficacy using the different sources of baseline data, but using data from a multielement functional analysis as baseline may save time. Interrater agreement was adequate, but lower for some graphs than has been observed in past studies. Several potential explanations for this discrepancy are discussed.     

Adjusting for information inflation due to local dependency in moderately large item clusters

When multiple items are clustered around a reading passage, the local independence assumption in item response theory is often violated. The amount of information contained in an item cluster is usually overestimated if violation of local independence is ignored and items are treated as locally independent when in fact they are not. In this article we provide a general method that adjusts for the inflation of information associated with a test containing item clusters. A computational scheme was presented for the evaluation of the factor of adjustment for clusters in the restrictive case of two items per cluster, and the general case of more than two items per cluster. The methodology was motivated by a study of the NAEP Reading Assessment. We present a simulated study along with an analysis of a NAEP data set.The research was supported under the National Assessment of Educational Progress (Grant No. R999G30002) as administered by the Office of Educational Research and Improvement, U.S. Department of Education. This work was started when the author was at the Division of Statistics and Psychometrics at the Educational Testing Service. The author thanks Juliet Shaffer, Bob Mislevy, Eric Bradlow, three reviewers and an associate editor for their helpful comments on the paper.     

An estimate of the covariance between variables which are not jointly observed

Situations sometimes arise in which variables collected in a study are not jointly observed. This typically occurs because of study design. An example is an equating study where distinct groups of subjects are administered different sections of a test. In the normal maximum likelihood function to estimate the covariance matrix among all variables, elements corresponding to those that are not jointly observed are unidentified. If a factor analysis model holds for the variables, however, then all sections of the matrix can be accurately estimated, using the fact that the covariances are a function of the factor loadings. Standard errors of the estimated covariances can be obtained by the delta method. In addition to estimating the covariance matrix in this design, the method can be applied to other problems such as regression factor analysis. Two examples are presented to illustrate the method. This research was partially supported by NIMH grant MH5-4576     

Models for ordinal hierarchical classes analysis

This paper proposes an ordinal generalization of the hierarchical classes model originally proposed by De Boeck and Rosenberg (1998). Any hierarchical classes model implies a decomposition of a two-way two-mode binary arrayM into two component matrices, called bundle matrices, which represent the association relation and the set-theoretical relations among the elements of both modes inM. Whereas the original model restricts the bundle matrices to be binary, the ordinal hierarchical classes model assumes that the bundles are ordinal variables with a prespecified number of values. This generalization results in a classification model with classes ordered along ordinal dimensions. The ordinal hierarchical classes model is shown to subsume Coombs and Kao's (1955) model for nonmetric factor analysis. An algorithm is described to fit the model to a given data set and is subsequently evaluated in an extensive simulation study. An application of the model to student housing data is discussed.     

“Sherry’s Secret”

The case and commentaries below were developed as part of a project, Graduate Research Ethics Education, undertaken by the Association for Practical and Professional Ethics with funding from the National Science Foundation (NSF Grant No. SBR 9421897 and NSF Grant No. 9817880). The project aims at training graduate students in research ethics and building a community of scientists and engineers who are interested in and capable of teaching research ethics. As part of the project, each graduate student participant develops a case for use in teaching and writes a commentary to go with the case, and then a staff member is asked to write additional commentary on the case. The case below was written in the second year of the project and was published in Research Ethics: Cases and Commentaries edited by B. Schrag, Association for Practical and Professional Ethics, Bloomington, Indiana, Vol. II (1998). Publication of these cases and commentaries will be a recurring feature of Science and Engineering Ethics.     

职业评定量表(VIESA)的信度和效度研究

根据D.E.Super的职业发展理论,职业选择的成熟与发展一般分为五个阶段,中学生正处在从探索到成熟的重要转变期。本研究以美国大学测验中心的职业分类系统为基础,以职业评定量表VIESA为工具和对象,对北京市第六十五中学的179名高中三年级学生和北京师范大学心理学系的55名二年级学生进行了施测,检验了量表的信度和效度。量表的信度指标基本在.70以上,构想效度良好,以专业满意感为效标的效标效度达到显著水平。     

不同条件下拟合指数的表现及临界值的选择

在本模拟研究中设计了6种样本容量,6种因子载荷,和4种评分等级,并考察了正态和非正态分布两种情况。采用的错误模型为参数误置（真模型中每个因子各由5个题目来测量,错误模型中则是第一个因子由6个题测量,另两个因子各由4个和5个题来测量,即有一个因子载荷被误置）模型。结果发现（1）样本量、载荷量、评分等级数和分布形态都对GOF的取值确有影响。其中分布形态的影响最大。NNFI、IFI在不同条件下的平均值是最稳定的,其次是CFI、RMSEA和SRMR。它们都算是值得推荐的GOF,尤其是NNFI和IFI。（2）在正态分布中,当样本量≥1000时,根据NNFI、IFI、CFI、RMSEA、SRMR对模型是否拟合做出判断时有很低的两类错误率,在样本量<1000时则不理想。在偏态条件下无论选择哪个GOF两类错误率都很高。（3）采用2指数策略在很多情况下也不能显著降低两类错误率。（4）由于在数据分布非正态,或正态但样本量<1000时是难判断模型是否拟合的。因此我们提出了2界值策略。即为每个GOF确定上下两个界值。低于下界值时可判断模型是不正确的,而高于上界值时则可判断模型是正确的。GOF取值处于上下界值之间时难以判断模型是否拟合,只能说越高拟合的可能性越大。这时就要通过跨样本验证和增加样本量来确定模型是否正确     

Disclosure of Tests and Raw Test Data to the Courts: A Need for Reform

Psychologists' current practices with respect to the disclosure of tests and raw test data to courts are damaging their reputations as scientists, undermining their credibility as honest professionals acting in good faith, and contrary to the best interests of consumers. The profession is conducting itself in a contradictory fashion that deserves reform. There is need for an interdisciplinary panel drawn from the American Psychological Association and American Bar Association to develop reasonable procedures for disclosure of tests and test data in legal proceedings. These procedures should be developed with input from experienced psychological expert witnesses, attorneys, and judges from a variety of state and federal jurisdictions, diverse geographical areas, and different types of legal proceedings. This expert panel should also address the issue of attorney coaching of clients in preparation for assessment by psychologists.     

An analysis and synthesis of multiple correspondence analysis,optimal scaling,dual scaling,homogeneity analysis and other methods for quantifying categorical multivariate data

We discuss a variety of methods for quantifying categorical multivariate data. These methods have been proposed in many different countries, by many different authors, under many different names. In the first major section of the paper we analyze the many different methods and show that they all lead to the same equations for analyzing the same data. In the second major section of the paper we introduce the notion of a duality diagram, and use this diagram to synthesize the many superficially different methods into a single method.The ideas in this paper were worked out by the first author, with some suggestions provided by the second. The current version of this paper has evolved from three previous versions, the first two written by the first author.     

Testing equality of correlated proportions with incomplete data on both responses

Two test statistics are proposed for testing the equality of two correlated proportions when some observations are missing on both responses. The performance of these tests in terms of size and power is compared with other tests by means of Monte Carlo simulations. The proposed tests are easily computed and compare favorably with other tests.