How many parameters can a model have and still be testable?

A standard rule of thumb states that a model has too many parameters to be testable if and only if it has at least as many parameters as empirically observable quantities. We argue that when one asks whether a model has too many parameters to be testable, one implicitly refers to a particular type of testability, which we call quantitative testability. A model is defined to be quantitatively testable if the model's predictions have zero probability of being correct by chance. Next, we propose a new rule of thumb, based on the rank of the Jacobian matrix of a model (i.e., the matrix of partial derivatives of the function that maps the model's parameter values onto predicted experimental outcomes). According to this rule, a model is quantitatively testable if and only if the rank of the Jacobian matrix is less than the number of observables. (The rank of his matrix can be found with standard computer algorithms.) Using Sard's theorem, we prove that the proposed new rule of thumb is correct provided that certain “smoothness” conditions are satisfied. We also discuss the relation between quantitative testability and reparameterization, identifiability, and goodness-of-fit testing.     

On the investigation of local identifiability: A counterexample

An assertion that the parameters of a covariance structure are locally identified at a certain point only if the rank of the Jacobian matrix at that point equals the number of parameters, is shown to be false by means of a counterexample.     

Identifiability and statistical properties of two-stage learning with no successes in the initial stage

A model is considered for learning where there is an initial stage with only errors, an intermediate state with errors and correct responses, and an absorbing state with only correct responses. A model with observable states is constructed, and sufficient identifiable parameters are given for the original model. Distribution statistics and other properties of data are derived as functions of the identifiable parameters. Informal estimates and sufficient statistics are given for the identifiable parameters. A number of testable hypotheses about the theoretical parameters are described. Some identifying restrictions for the model are given, along with the estimates for the unrestricted free parameters which follow. An application is made to data from a pairedassociate learning experiment.This research was sponsored by NSF Grant GB2791 and by PHS Grant MH12717. Part of the work was done while the author held a visiting faculty appointment at Stanford University.     

Identifiability and Equivalence of GLLIRM Models

The generalized logit–linear item response model (GLLIRM) is a linearly constrained nominal categories model (NCM) that computes the scale and intercept parameters for categories as a weighted sum of basic parameters. This paper addresses the problems of the identifiability of the basic parameters and the equivalence between different GLLIRM models. It is shown that the identifiability of the basic parameters depends on the size and rank of the coefficient matrix of the linear functions. Moreover, two models are observationally equivalent if the product of the respective coefficient matrices has full column rank. Finally, the paper also explores the relations between the parameters of nested models. I would like to express my gratitude to the editor and three anonymous reviewers for their helpful suggestions on earlier versions of the paper. This work was supported by the Comunidad de Madrid (Spain) grant: CCG07-UAM/ESP-1615.     

Nonlinear submodels of orthogonal linear models

A model containing linear and nonlinear parameters (e. g., a spatial multidimensional scaling model) is viewed as a linear model with free and constrained parameters. Since the rank deficiency of the design matrix for the linear model determines the number of side conditions needed to identify its parameters, the design matrix acts as a guide in identifying the parameters of the nonlinear model. Moreover, if the design matrix and the uniqueness conditions constitute anorthogonal linear model, then the associated error sum of squares may be expressed in a form which separates the free and constrained parameters. This immediately provides least squares estimates of the free parameters, while simplifying the least squares problem for those which are constrained. When the least squares estimates for a nonlinear model are obtained in this way,i.e. by conceptualizing it as a submodel, the final error sum of squares for the nonlinear model will be arestricted minimum whenever the side conditions of the model become real restrictions upon its submodel. In this case the design matrix for the embracing orthogonal model serves as a guide in introducing parameters into the nonlinear model as well as in identifying these parameters. The method of overwriting a nonlinear model with an orthogonal linear model is illustrated with two different spatial analyses of a three-way preference table.     

An Introduction to Network Psychometrics: Relating Ising Network Models to Item Response Theory Models

In recent years, network models have been proposed as an alternative representation of psychometric constructs such as depression. In such models, the covariance between observables (e.g., symptoms like depressed mood, feelings of worthlessness, and guilt) is explained in terms of a pattern of causal interactions between these observables, which contrasts with classical interpretations in which the observables are conceptualized as the effects of a reflective latent variable. However, few investigations have been directed at the question how these different models relate to each other. To shed light on this issue, the current paper explores the relation between one of the most important network models—the Ising model from physics—and one of the most important latent variable models—the Item Response Theory (IRT) model from psychometrics. The Ising model describes the interaction between states of particles that are connected in a network, whereas the IRT model describes the probability distribution associated with item responses in a psychometric test as a function of a latent variable. Despite the divergent backgrounds of the models, we show a broad equivalence between them and also illustrate several opportunities that arise from this connection.     

Some local models for correlation experiments

This paper constructs two classes of models for the quantum correlation experiments used to test the Bell-type inequalities, synchronization models and prism models. Both classes employ deterministic hidden variables, satisfy the causal requirements of physical locality, and yield precisely the quantum mechanical statistics. In the synchronization models, the joint probabilities, for each emission, do not factor in the manner of stochastic independence, showing that such factorizability is not required for locality. In the prism models the observables are not random variables over a common space; hence these models throw into question the entire random variables idiom of the literature. Both classes of models appear to be testable.Work on this paper was supported, in part, by National Science Foundation Grant SES 79-25917.     

现代测量理论下四大认知诊断模型述评

该文介绍并比较了现代测量理论下四大认知诊断模型的思想方法、模型结构及各自的特点性能等。LLTM是一个较早的认知诊断模型,它实现了认知与测量的结合;规则空间模型实现了对认知结构的诊断,并创造性地提出了Q矩阵理论;统一模型与融合模型是同一类模型：两者均沿用了规则空间模型的Q矩阵方法,但克服了规则空间模型中的一些不足;融合模型被认为是二十一世纪初创立的一个很成功的认知诊断模型。     

On the law of Regular Minimality: Reply to Ennis

Ennis's critique touches on issues important for psychophysics, but the points he makes against the hypothesis that Regular Minimality is a basic property of sensory discrimination are not tenable.(1) Stimulus variability means that one and the same apparent stimulus value (as measured by experimenter) is a probabilistic mixture of true stimulus values. The notion of a true stimulus value is a logical necessity: variability and distribution presuppose the values that vary and are distributed (even if these values are represented by processes or sets rather than real numbers). Regular Minimality is formulated for true stimulus values. That a mixture of probabilities satisfying Regular Minimality does not satisfy this principle (unless it also satisfies Constant Self-Similarity) is an immediate consequence of my 2003 analysis. Stimulus variability can be controlled or estimated: the cases when observed violations of Regular Minimality can be accounted for by stimulus variability corroborate rather than falsify this principle. In this respect stimulus variability is no different from fatigue, perceptual learning, and other factors creating mixtures of discrimination probabilities in an experiment.(2) Could it be that well-behaved Thurstonian-type models are true models of discrimination but their parameters are so adjusted that the violations of Regular Minimality they lead to (due to my 2003 theorems) are too small to be detected experimentally? This is possible, but this amounts to admitting that Regular Minimality is a law after all, albeit only approximate: nothing in the logic of the Thurstonian-type representations per se prevents them from violating Regular Minimality grossly rather than slightly. Moreover, even very small violations predicted by a given class of Thurstonian-type models can be tested in specially designed experiments (perhaps under additional, independently testable assumptions). The results of one such experiment, in which observers were asked to alternately adjust to each other the values of stimuli in two observation areas, indicate that violations of Regular Minimality, if any, are far below limits of plausible interpretability.     

Does Smoking by Pregnant Women Influence IQ, Birth Weight, and Developmental Disabilities in Their Infants? A Methodological Review and Multivariate Analysis

Neuropsychologists are asked frequently to address the issue of the cause of a variety of central nervous system problems that may affect higher cortical function. One such issue is the relationship of maternal smoking to adverse reproductive outcomes involving neocortical insult including mental retardation, learning disabilities, attention-deficit hyperactivity disorder, and other insults that may be related to prolonged hypoxic states in utero. The instant paper develops the issue of causation as a scientific inquiry, reviews several traditional, applicable models, and critiques these models. An additional model of motility is proposed and discussed. The issue of the relationship of maternal smoking to adverse reproductive outcomes is then addressed from a review perspective along with new empirical analyses, the latter demonstrating that researchers tend to draw causal conclusions independent of whether the respective design of their studies would support conclusions about the causation of an event. Causal conclusions in the absence of causal designs have often lead to incomplete and incorrect conclusions. It is necessary to match conclusions not only to the outcomes of a research project but also to its design and accompanying limitations.     

Global model analysis by parameter space partitioning

To model behavior, scientists need to know how models behave. This means learning what other behaviors a model can produce besides the one generated by participants in an experiment. This is a difficult problem because of the complexity of psychological models (e.g., their many parameters) and because the behavioral precision of models (e.g., interval-scale performance) often mismatches their testable precision in experiments, where qualitative, ordinal predictions are the norm. Parameter space partitioning is a solution that evaluates model performance at a qualitative level. There exists a partition on the model's parameter space that divides it into regions that correspond to each data pattern. Three application examples demonstrate its potential and versatility for studying the global behavior of psychological models.     

Rank-reducibility of a symmetric matrix and sampling theory of minimum trace factor analysis

One of the intriguing questions of factor analysis is the extent to which one can reduce the rank of a symmetric matrix by only changing its diagonal entries. We show in this paper that the set of matrices, which can be reduced to rankr, has positive (Lebesgue) measure if and only ifr is greater or equal to the Ledermann bound. In other words the Ledermann bound is shown to bealmost surely the greatest lower bound to a reduced rank of the sample covariance matrix. Afterwards an asymptotic sampling theory of so-called minimum trace factor analysis (MTFA) is proposed. The theory is based on continuous and differential properties of functions involved in the MTFA. Convex analysis techniques are utilized to obtain conditions for differentiability of these functions.     

Prior sensitivity in theory testing: An apologia for the Bayes factor

A commonly voiced concern with the Bayes factor is that, unlike many other Bayesian and non-Bayesian quantitative measures of model evaluation, it is highly sensitive to the parameter prior. This paper argues that, when dealing with psychological models that are quantitatively instantiated theories, being sensitive to the prior is an attractive feature of a model evaluation measure. This assertion follows from the observation that in psychological models parameters are not completely unknown, but correspond to psychological variables about which theory often exists. This theory can be formally captured in the prior range and prior distribution of the parameters, indicating which parameter values are allowed, likely, unlikely and forbidden. Because the prior is a vehicle for expressing psychological theory, it should, like the model equation, be considered as an integral part of the model. It is argued that the combined practice of building models using informative priors, and evaluating models using prior sensitive measures advances knowledge.     

On the identifiability of Parameters in Thurstone's multiple factor analysis

In econometric literature a parameter in a theoretical model has been called identifiable if it can be uniquely determined in terms of the joint probability distribution of the observed variables. In this paper the identifiability of parameters in four different factor analysis models is considered. The last of these four models corresponds to Thurstone's factor analysis. In Sections 7 and 11, the possibility of a statistical testing of the models is discussed. Section 10 deals with the problem of actually determining the parameterr (the number of common factors) in terms of the probability distribution of the observed variables.This article will be included in Cowles Commission Paper, New Series, No. 39.     

多题多做测验模型及其应用

在IRT框架下,建立了0-1评分方式下单维双参数Logistic多题多做（MAMI）测验模型。与Spray给出的一题多做（MASI）模型相比,MAMI不仅模型更加精致,而且扩展了适用范围,参数估计方法也不同,采用EM算法求取项目参数。Monte Carlo模拟结果显示,应用MAMI测验模型与测验题量作相应增加的作法相比,两者给出的能力估计精度相同,但MAMI模型给出的项目参数估计精度更高。如果将MAMI测验模型与被试人数相应增加的作法相比,项目参数的估计精度相同,但MAMI给出的能力参数估计精度更高。这个发现表明,在一定条件下若允许修改答案,并采用累加式记分方式,纵使题量不变,也可使能力估计的精度相当于题量增加一倍的估计精度,而项目参数估计精度也会提高。这些发现不仅对技能评价和认知能力评价有参考价值,而且对数据的处理方式也有参考价值     

A note on the Tryon-Kaiser solution for the communalities

The Tryon-Kaiser solution for the communalities is reviewed. Numerical investigation suggests that the procedure is applicable if and only if the correlation matrix has unique minimum rank communalities. This implies that this approach to the communality problem is not general enough to be of practical use.     

高中英语阅读测验中题组模型的选择与应用

本研究通过高中英语阅读测验实测数据,对比分析双参数逻辑斯蒂克模型 (2PL-IRT)和加入不同数量题组的双参数逻辑斯蒂克模型 (2PL-TRT), 探究题组数量对参数估计及模型拟合的影响。结果表明：(1) 2PL-IRT模型对能力介于-1.50到0.50的被试,能力参数估计偏差较大;(2)将题组效应大于0.50的题组作为局部独立题目纳入模型,会导致部分题目区分度参数的低估和大部分题目难度参数的高估;(3)题组效应越大,将其当作局部独立题目纳入模型估计项目参数的偏差越大。     

Rule transition on the balance scale task: a case study in belief change

For various domains in proportional reasoning cognitive development is characterized as a progression through a series of increasingly complex rules. A multiplicative relationship between two task features, such as weight and distance information of blocks placed at both sides of the fulcrum of a balance scale, appears difficult to discover. During development, children change their beliefs about the balance scale several times: from a focus on the weight dimension (Rule I) to occasionally considering the distance dimension (Rule II), guessing (Rule III), and applying multiplication (Rule IV; Siegler, 1981). Because of the detailed empirical findings the balance scale task has become a benchmark task for computational models of proportional reasoning. In this article, we present a large empirical study (N = 420) of which the findings provide a challenge for computational models. The effect of feedback and the effect of individually adapted training items on rule transition were tested for children using Rule I or Rule II. Presenting adapted training items initiates belief revision for Rule I but not for Rule II. The experience of making mistakes (by providing feedback) induces a change for both Rule I and Rule II. However, a delayed posttest shows that these changes are preserved after 2 weeks only for children using Rule I. We conclude that the transition from Rule I to Rule II differs from the transition from Rule II to a more complex rule. Concerning these empirical findings, we will review performance of computational models and the implications for a future belief revision model.

It is one Thing, to show a Man that he is in an Error, and another, to put him in possession of Truth. John Locke

     

Population models and simulation methods: The case of the Spearman rank correlation

The purpose of this paper is to highlight the importance of a population model in guiding the design and interpretation of simulation studies used to investigate the Spearman rank correlation. The Spearman rank correlation has been known for over a hundred years to applied researchers and methodologists alike and is one of the most widely used non‐parametric statistics. Still, certain misconceptions can be found, either explicitly or implicitly, in the published literature because a population definition for this statistic is rarely discussed within the social and behavioural sciences. By relying on copula distribution theory, a population model is presented for the Spearman rank correlation, and its properties are explored both theoretically and in a simulation study. Through the use of the Iman–Conover algorithm (which allows the user to specify the rank correlation as a population parameter), simulation studies from previously published articles are explored, and it is found that many of the conclusions purported in them regarding the nature of the Spearman correlation would change if the data‐generation mechanism better matched the simulation design. More specifically, issues such as small sample bias and lack of power of the t‐test and r‐to‐z Fisher transformation disappear when the rank correlation is calculated from data sampled where the rank correlation is the population parameter. A proof for the consistency of the sample estimate of the rank correlation is shown as well as the flexibility of the copula model to encompass results previously published in the mathematical literature.