The Importance of Defining ‘Data’ in Data Management Policies

What comprises ‘data’ varies from one institution to another based on the information which is deemed important by individual institutions. To effectively and efficiently produce, collect, and retain data, an organization develops specific defining characteristics of data to meet its informational needs. Procedures to maintain and retain knowledge among laboratory members and principal investigators will allow for improved efficiency of data collection. Optimization of communication, maintenance of inventories, record keeping, and updating relevant training programs are all critical to supporting the quality and integrity of a particular organization’s data. Concurrent revisions to such procedures will ensure that the definition of data as well as the means by which it is collected and maintained remain appropriate to the needs of the individual organization.     

What Is Missing in Counseling Research? Reporting Missing Data

Missing data have long been problematic in quantitative research. Despite the statistical and methodological advances made over the past 3 decades, counseling researchers fail to provide adequate information on this phenomenon. Interpreting the complex statistical procedures and esoteric language seems to be a contributing factor. An overview of missing data and remediation procedures is provided. A systematic approach to addressing missing data along with reporting suggestions is presented.     

Model Conditioned Data Elasticity in Path Analysis: Assessing the “Confoundability” of Model/Data Characteristics

Much research has been directed at the validity of fit indices in Path Analysis and Structural Equation Modeling (e.g., Browne, MacCallum, Kim, Andersen, &; Glaser, 2002 Browne, M.W., MacCallum, R.C., Kim, C.T., Andersen, B.L., &; Glaser, R. (2002). When fit indices and residuals are incompatible. Psychological Methods, 7, 403–421.[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]; Heene, Hilbert, Draxler, Ziegler, &; Bühner, 2011 Heene, M., Hilbert, S., Draxler, C., Ziegler, M., &; Bühner, M. (2011). Masking misfit in confirmatory factor analysis by increasing unique variances: A cautionary note on the usefulness of cutoff values of fit indices. Psychological Methods, 16(3), 319–336.[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]; Hu &; Bentler, 1999 Hu, L., &; Bentler, P. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, J., 6(1), 1–55.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]; Marsh, Hau, &; Wen, 2004 Marsh, H.W., Hau, K.T., &; Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11(3), 320–341.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]). Recent developments (e.g., Preacher, 2006 Preacher, K.J. (2006). Quantifying parsimony in structural equation modeling. Multivariate Behavioral Research, 41(3), 227–259.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]; Roberts &; Pashler, 2000 Roberts, S., &; Pashler, H. (2000). How persuasive is a good fit? A comment on theory testing. Psychological Review, 107(2), 358–367.[Crossref], [PubMed], [Web of Science ®] , [Google Scholar], 2002 Roberts, S., &; Pashler, H. (2002). Reply to Rodgers and Rowe (2002). Psychological Review, 109(3), 605–607.[Crossref] , [Google Scholar]) have encouraged researchers to investigate other criteria for comparing models, including model complexity. What has not been investigated is the inherent ability of a particular data set to be fitted with a constrained set of randomly generated linear models, which we call Model Conditioned Data Elasticity (DE). In this article we show how DE can be compared with the problem of equivalent models and a more general problem of the “confoundability” of data/model combinations (see MacCallum, Wegener, Uchino, &; Fabrigar, 1993 MacCallum, R.C., Wegener, D.T., Uchino, B.N., &; Fabrigar, L.R. (1993). The problem of equivalent models in applications of covariance structure analysis. Psychological Bulletin, 114(1), 185–199.[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]). Using the DE package in R, we show how DE can be assessed through automated computer searches. Finally, we discuss how DE fits within the controversy surrounding the use of fit statistics.     

Precue effects in visual search: Data or resource limited?

Precuing an observer as to where a target is more likely to occur in a subsequent visual array can increase the detectability (d′) of a target at that location. This is often attributed to the observer’s increased allocation of some limited cognitive resource (“attention”) to the cued location. Two experiments are reported which are difficult to interpret in this way even though they involve similar cue effects. The first involves postcuing a location well after the array but before the observer responds, so that the cue can influence the response but not the observation. The second involves precuing, but with slow sequential presentation of array elements prior to the response, so the observer need not share any limited resource while processing each element in turn. Enhanced detectability similar to that produced with precues and simultaneous presentation of elements is shown to occur in each experiment. An alternative data-limited (rather than resource-limited) interpretation of these effects is provided by a mathematical model in which the observer integrates equally noisy or degraded internal representations of the array elements, but gives more weight to cued elements in selecting a response. Theoretical parameters of the model are shown to provide separate measures of both an observer’s overall sensitivity and precue effects in cost-benefit analyses of cuing data.     

Paper or plastic? Data equivalence in paper and electronic diaries

Concern has been raised about the lack of participant compliance in diary studies that use paper-and-pencil as opposed to electronic formats. Three studies explored the magnitude of compliance problems and their effects on data quality. Study 1 used random signals to elicit diary reports and found close matches to self-reported completion times, matches that could not plausibly have been fabricated. Studies 2 and 3 examined the psychometric and statistical equivalence of data obtained with paper versus electronic formats. With minor exceptions, both methods yielded data that were equivalent psychometrically and in patterns of findings. These results serve to at least partially mollify concern about the validity of paper diary methods.     

The Nonsingularity of Γ in Covariance Structure Analysis of Nonnormal Data

Covariance structure analysis of nonnormal data is important because in practice all data are nonnormal. When applying covariance structure analysis to nonnormal data, it is generally assumed that the asymptotic covariance matrix Γ for the nonredundant terms in the sample covariance matrix S is nonsingular. It is shown this need not be the case, which raises a question of how restrictive this assumption may be and how difficult it may be to verify it. It is shown that Γ is nonsingular whenever sampling is from a nonsingular distribution, including any distribution defined by a density function. In the discrete case necessary and sufficient conditions are given for the nonsingularity of Γ, and it is shown how to demonstrate Γ is nonsingular with high probability. Thus, the nonsingularity of Γ assumption is mild and one should feel comfortable about making it. These observations also apply to the asymptotic covariance matrix Γ that arises in structural equation modeling.     

Science as a Vocation in the Era of Big Data: the Philosophy of Science behind Big Data and humanity’s Continued Part in Science

We now live in the era of big data, and according to its proponents, big data is poised to change science as we know it. Claims of having no theory and no ideology are made, and there is an assumption that the results of big data are trustworthy because it is considered free from human judgement, which is often considered inextricably linked with human error. These two claims lead to the idea that big data is the source of better scientific knowledge, through more objectivity, more data, and better analysis. In this paper I analyse the philosophy of science behind big data and make the claim that the death of many traditional sciences, and the human scientist, is much exaggerated. The philosophy of science of big data means that there are certain things big data does very well, and some things that it cannot do. I argue that humans will still be needed for mediating and creating theory, and for providing the legitimacy and values science needs as a normative social enterprise.     

Editors’ Overview: Topics in the Responsible Management of Research Data

Responsible data management is a multifaceted topic involving standards within the research community regarding research design and the sharing of data as well as the collection, selection, analysis and interpretation of data. Transparency in the manipulation of images is increasingly important in order to avoid misrepresentation of research findings, and research oversight is also critical in helping to assure the integrity of the research process. Intellectual property issues both unite and divide academe and industry in their approaches to data management. Central to the realization and promulgation of responsible data management is clear and careful communication of standards and expectations within the research community to trainees as well as among colleagues. These topics are examined and explored in depth in a special issue of Science and Engineering Ethics on responsible data management.     

Detecting Duplication in Students’ Research Data: A Method and Illustration

Research integrity is core to the mission of higher education. In undergraduate student samples, self-reported rates of data fabrication have been troublingly high. Despite this, no research has investigated undergraduate data fabrication in a more systematic manner. We applied duplication screening techniques to 18 data sets submitted by psychology honors students for assessment. Although we did not identify any completely duplicated cases, there were numerous partial duplicates. Rather than indicating fabrication, however, these partial duplicates are likely a consequence of poor measure selection, insufficient data screening, and/or participant characteristics. Implications for the teaching and supervision of honors students are discussed.     

ROOTCLUS: Searching for “ROOT CLUSters” in Three-Way Proximity Data

In the context of three-way proximity data, an INDCLUS-type model is presented to address the issue of subject heterogeneity regarding the perception of object pairwise similarity. A model, termed ROOTCLUS, is presented that allows for the detection of a subset of objects whose similarities are described in terms of non-overlapping clusters (ROOT CLUSters) common across all subjects. For the other objects, Individual partitions, which are subject specific, are allowed where clusters are linked one-to-one to the Root clusters. A sound ALS-type algorithm to fit the model to data is presented. The novel method is evaluated in an extensive simulation study and illustrated with empirical data sets.

     

“Stay in the Game”: Collecting Data on Symptom Improvement in Naturalistic Settings in Order to Increase Reimbursement Rates

An industrious approach to increasing reimbursement rates from third-party payers is reviewed. Balancing fidelity to treatment with real-world demands at a local hospital is discussed, as well as ways we were able to increase reimbursement within our own department. Data in naturalistic settings, though not always able to adhere to randomized controlled trials, offers evidence of improvement in symptoms. This reaction paper encourages other clinical programs to "stay in the game" of data collection while balancing treatment fidelity.     

Data integration in legal evaluations: Can clinicians deliver on their premises?

Clinicians often claim to base their conclusions on configural analysis and integration of most or all of the data. However, research suggests that subjective beliefs about complex interpretive strategies and their resultant success are largely illusory. Studies show that: (1) access to more data does not necessarily improve judgmental accuracy, (2) clinical judgments, which purportedly depend on configural analyses, can usually be duplicated by simple linear models, (3) individuals often have difficulty deciphering interactions or patterns among just two or three variables, and (4) actuarial methods, even those that simply add variables together, almost always equal or exceed the accuracy of clinicians who purport to integrate large amounts of information. Corrective suggestions are discussed that may help the clinician better manage large data sets.     

No more ‘personal notes’? Data protection policy and practice in Higher Education counselling services in the UK

The introduction of the Data Protection Act 1998 has presented significant challenges to counselling recording, through its detailed requirements for the processing of personal data. The impact of these changes on recording in Higher Education counselling services in the UK is explored, using responses from an electronic survey of universities and colleges. The material is presented in terms of three themes, namely transparency, accountability and professional autonomy. The Act has encouraged more transparent and accountable recording practice by counsellors, while discouraging the use of 'personal notes' held in permanent form outside the remit of formal recording systems. Some key implications for redefining practitioner autonomy in relation to recording, consistent with the principles of transparency and accountability, are briefly explored.     

Is Semantic Information Meaningful Data?

There is no consensus yet on the definition of semantic information. This paper contributes to the current debate by criticising and revising the Standard Definition of semantic Information (SDI) as meaningful data, in favour of the Dretske-Grice approach: meaningful and well-formed data constitute semantic information only if they also qualify as contingently truthful. After a brief introduction, SDI is criticised for providing necessary but insufficient conditions for the definition of semantic information. SDI is incorrect because truth-values do not supervene on semantic information, and misinformation (that is, false semantic information) is not a type of semantic information, but pseudo-information, that is not semantic information at all. This is shown by arguing that none of the reasons for interpreting misinformation as a type of semantic information is convincing, whilst there are compelling reasons to treat it as pseudo-information. As a consequence, SDI is revised to include a necessary truth-condition. The last section summarises the main results of the paper and indicates some interesting areas of application of the revised definition.     

Model Evaluation and Multiple Strategies in Cognitive Diagnosis: An Analysis of Fraction Subtraction Data

This paper studies three models for cognitive diagnosis, each illustrated with an application to fraction subtraction data. The objective of each of these models is to classify examinees according to their mastery of skills assumed to be required for fraction subtraction. We consider the DINA model, the NIDA model, and a new model that extends the DINA model to allow for multiple strategies of problem solving. For each of these models the joint distribution of the indicators of skill mastery is modeled using a single continuous higher-order latent trait, to explain the dependence in the mastery of distinct skills. This approach stems from viewing the skills as the specific states of knowledge required for exam performance, and viewing these skills as arising from a broadly defined latent trait resembling the θ of item response models. We discuss several techniques for comparing models and assessing goodness of fit. We then implement these methods using the fraction subtraction data with the aim of selecting the best of the three models for this application. We employ Markov chain Monte Carlo algorithms to fit the models, and we present simulation results to examine the performance of these algorithms. The work reported here was performed under the auspices of the External Diagnostic Research Team funded by Educational Testing Service. Views expressed in this paper does not necessarily represent the views of Educational Testing Service.     

A Positive Relationship Between Religious Faith and Forgiveness: Faith in the Absence of Data?

Religious faith and beliefs appear to play an important role in the lives of many individuals and are the topic of much research. The present study investigated the relationship between religious faith and forgiveness in a sample (n = 196) of college students. Students were asked to complete the Heartland Forgiveness Scale and the Santa Clara Strength of Religious Faith Questionnaire. Analyses of scores on both measures revealed a positive, significant correlation between these constructs, suggesting that there is a meaningful relationship between religious faith and the tendency to forgive. Implications and directions for further research are discussed.     

Big Data,Big Waste? A Reflection on the Environmental Sustainability of Big Data Initiatives

Science and Engineering Ethics - This paper addresses a problem that has so far been neglected by scholars investigating the ethics of Big Data and policy makers: that is the ethical implications...     

National Identity in a United Germany: Nationalism or Patriotism? An Empirical Test With Representative Data

Nationalism and patriotism can be thought of as consequences of national identity that represent positive evaluations of one's own group but imply different social goals. This paper investigates the ways in which these concepts are related to attitudes toward minorities. The data analyzed were drawn from a representative sample of residents of the former East and West Germany who responded to items on the national identity of Germans in 1996 as part of a panel study. A model with multiple indicators was tested via a multiple-group analysis of a structural equations model followed by latent class analyses. Both East and West Germans displayed attitudinal patterns that link national identity with tolerance toward others; in both subsamples, nationalism and patriotism were respectively associated with greater intolerance and greater tolerance toward minorities.