Using Cognitive Interviewing for the Semantic Enhancement of Multilingual Versions of Personality Questionnaires

We discuss the use of cognitive interviewing with bilinguals as an integral part of cross-cultural adaptation of personality questionnaires. The aim is to maximize semantic equivalence to increase the likelihood of items maintaining the intended structure and meaning in the target language. We refer to this part of adaptation as semantic enhancement, and integrate cognitive interviewing within it as a tool for scrutinizing translations, the connotative meaning, and the psychological impact of items across languages. During the adaptation of a work-based personality questionnaire from English to Arabic, Chinese (Mandarin), and Spanish, we cognitively interviewed 12 bilingual participants about 136 items in different languages (17% of all items), of which 67 were changed. A content analysis categorizing the reasons for amending items elicited 11 errors that affect 2 identified forms of semantic equivalence. We provide the resultant coding scheme as a framework for designing cognitive interviewing protocols and propose a procedure for implementing them. We discuss implications for theory and practice.     

测验链接中的锚题代表性研究

本文旨在以“锚题代表性”这一研究命题切入,探索在非等组锚测验设计下,作为实现测验链接的重要载体,锚题和相关的测验试卷/水平之间究竟应该有什么关系。本文首先指出锚题代表性这一概念在等值和垂直量尺化领域中具有不同的含义,并给出其在垂直量尺化中的含义。通过考察测验链接中有关锚题代表性的既有研究,系统总结相关研究成果,本文概括出了当前锚题构建实践的可能优化方案,分析了锚题代表性研究的未来方向。     

Evaluating the Equivalence of an Employee Attitude Survey Across Languages,Cultures, and Administration Formats

Modern multinational organizations have found that testing or surveying their employees is difficult when the employees come from a variety of language backgrounds. In such situations, surveys and tests are often adapted for use across multiple languages. However, different language versions of an instrument are not necessarily equivalent, which may lead to misleading interpretations. In this study, we used weighted multidimensional scaling (MDS), analysis of covariance (ANCOVA), and ordinal logistic regression (LR) to evaluate the structural equivalence and differential item functioning (DIF) of an employee attitude survey from a large international corporation. Specifically, we evaluated the functioning of the survey items across 3 different languages, 8 different cultures, and 2 mediums of administration (paper-based and Web-based). MDS was used to evaluate structural equivalence, and ANCOVA and LR were used to evaluate DIF across selected employee groups and the 2 administration formats. The results indicated that the structure of the survey data was consistent and that the items functioned similarly across all groups. The results also illustrate the utility of MDS, ANCOVA, and LR for evaluating translated instruments. The implications of the results for future research in this area are discussed.     

我国一大型考试等值的铆题参数漂移检验

设置铆题来链接不同测验形式是一种常用的等值设计。但受到曝光等因素影响,铆题功能在不同施测时间会发生改变。本研究采用MH检验和logistic回归考察我国一大型考试等值的铆题质量,结果发现,有22个铆题发生参数漂移,铆题的难度参数和区分度参数可能发生漂移;这些铆题中大部分在二次使用时无法通过模型拟合检验;若不删除参数发生漂移的铆题导致较大的系统等值误差,应将铆题参数漂移检验作为等值中的一步必要工作。     

题目难度分布和样本容量对两种CTT等值结果的影响

基于经典测验理论(CTT)的等值方法主要有线性等值和等百分位等值两种。在不同情境下,不同的等值方法会产生不同的等值结果。本研究以真分数等值为依据,用蒙特卡洛模拟研究方法,综合比较了各种题目难度分布条件下和各种样本容量条件下两种CTT等值方法的等值结果。研究结果表明:(1)线性等值的误差受题目难度分布影响较大,等百分位等值的误差几乎不受题目难度分布影响。(2)线性等值的误差几乎不受样本容量的影响,等百分位等值的误差受样本容量影响较大。(3)不论题目难度分布如何,只要样本容量足够大,等百分位等值的效果都比线性等值更好。     

无铆题情况下测验分数等值方法探索——构造铆测验法

研究旨在探索无铆题情况下,使用构造铆测验法,实现测验分数等值。研究一和研究二分别探索题目难度排序错误、铆题难度差异对构造铆测验法的影响。结果表明：（1）等组条件下,随着错误铆题比例,难度排序错误程度,铆题难度差异增大,构造铆测验法的等值误差逐渐增大,随机等组法的等值误差较为稳定;不等组条件下,构造铆测验法的等值误差均小于随机等组法;（2）对于构造铆测验法,在不等组条件下,铆测验长度越短,等值误差越大。     

Interlingual lexical competition in a spoken sentence context: Evidence from the visual world paradigm

We used the visual world paradigm to examine interlingual lexical competition when Dutch–English bilinguals listened to low-constraining sentences in their nonnative (L2; Experiment 1) and native (L1; Experiment 2) languages. Additionally, we investigated the influence of the degree of cross-lingual phonological similarity. When listening in L2, participants fixated more on competitor pictures of which the onset of the name was phonologically related to the onset of the name of the target in the nontarget language (e.g., fles, “bottle”, given target flower) than on phonologically unrelated distractor pictures. Even when they listened in L1, this effect was also observed when the onsets of the names of the target picture (in L1) and the competitor picture (in L2) were phonologically very similar. These findings provide evidence for interlingual competition during the comprehension of spoken sentences, both in L2 and in L1.     

Bimodal bilinguals co-activate both languages during spoken comprehension

Bilinguals have been shown to activate their two languages in parallel, and this process can often be attributed to overlap in input between the two languages. The present study examines whether two languages that do not overlap in input structure, and that have distinct phonological systems, such as American Sign Language (ASL) and English, are also activated in parallel. Hearing ASL-English bimodal bilinguals' and English monolinguals' eye-movements were recorded during a visual world paradigm, in which participants were instructed, in English, to select objects from a display. In critical trials, the target item appeared with a competing item that overlapped with the target in ASL phonology. Bimodal bilinguals looked more at competing item than at phonologically unrelated items and looked more at competing items relative to monolinguals, indicating activation of the sign-language during spoken English comprehension. The findings suggest that language co-activation is not modality specific, and provide insight into the mechanisms that may underlie cross-modal language co-activation in bimodal bilinguals, including the role that top-down and lateral connections between levels of processing may play in language comprehension.     

大学英语四、六级考试分数等值研究

对现有的大学英语四、六级考试分数等值模式中存在的若干问题进行了深入的分析,并提出了新的解决方案——一个基于铆题设计和两参数IRT模型的解决方案。主要包括：（1）用两参数逻辑斯蒂模型替代原来的Rasch模型,以改进题目模型的适合性;（2）用共同题目的等值设计取代原来的共同被试等值设计,解决共同被试等值设计中,等值考生的动机水平难以控制的难题;（3）建立专用的等值用题库,并且一次性完成其中铆题的预测和参数标定工作,以解决原来等值模式中存在的误差累积问题。同时,由于铆题的保密工作难度较小,因此,等值专用题库对保证等值结果的可靠性也具有重大意义;（4）本文还对新的分数等值方案进行了真实的考试数据等值计算实验,并得到了一个令人满意的分数等值结果。     

Comprehending idioms cross-linguistically

Speakers of three different languages (English, Latvian, and Mandarin) rated sets of idioms from their language for the analyzability of the relationship between each phrase's literal and figurative meaning. For each language, subsets of idioms were selected based on these ratings. Latvian and Mandarin idioms were literally translated into English. Across three experiments, people classified idioms from the three languages according to their figurative meanings. Response times and error rates indicate that participants were able to interpret unfamiliar (e.g., other languages') idioms depending largely on the degree to which they were analyzable, and that different forms of processing were used both within and between languages depending on this analyzability. Results support arguments for a continuum of analyzability (Bortfeld & McGlone, 2001), along which figurative speech ranges from reflecting general conceptual structures to specific cultural and historical references.     

Interlinguistic conflict: Word–word Stroop with first and second language colour words

The congruency (or Stroop) effect is a standard observation of slower and less accurate colour identification to incongruent trials (e.g. “red” in green) relative to congruent trials (e.g. “red” in red). This effect has been observed in a word–word variant of the task, when both the distracter (e.g. “red”) and target (e.g. “green”) are colour words. The Stroop task has also been used to study the congruency effect between two languages in bilinguals. The typical finding is that the congruency effect for L1 words is larger than that for L2 words. For the first time, the present report aims to extend this finding to a word–word variant of the bilingual Stroop task. In two experiments, French monolinguals performed a bilingual word–word Stroop task in which target word language, language match, and congruency between the distracter and target were manipulated. The critical manipulation across two experiments concerned the target language. In Experiment 1, target language was manipulated between groups, with either French (L1) or English (L2) target colour words. In Experiment 2, target words from both languages were intermixed. In both experiments, the congruency effect was larger when the distracter and target were from the same language (language match) than when they were from different languages (language mismatch). Our findings suggested that this congruency effect mostly depends on the language match between the distracter and target, rather than on a target language. It also did not seem to matter whether the language-mismatching distracter was or was not a potential response alternative. Semantic activation of languages in bilinguals and its implications on target identification are discussed.

     

维-汉-英三语者三种语言语义通达模型的实验研究

以维吾尔族大学生为研究对象,采用跨语言启动条件下的真假词汇判断任务,变换启动词与目标词的关系类型和语言种类,考察了L2—L1、L3—L1、L3—L2等三种语言词汇之间的语义通达模式。结果发现,维-汉-英三语者的三种语言词汇之间的联结模式为熟练的第二语言词汇是直接通达语义;而非熟练的第三语言词汇既借助母语词汇又借助汉语词汇来通达语义。除此之外,被试的外语熟练水平和语言间的相似性是影响外语词汇通达语义的主要原因。     

Repeated retrieval practice and item difficulty: Does criterion learning eliminate item difficulty effects?

A wealth of previous research has established that retrieval practice promotes memory, particularly when retrieval is successful. Although successful retrieval promotes memory, it remains unclear whether successful retrieval promotes memory equally well for items of varying difficulty. Will easy items still outperform difficult items on a final test if all items have been correctly recalled equal numbers of times during practice? In two experiments, normatively difficult and easy Lithuanian–English word pairs were learned via test–restudy practice until each item had been correctly recalled a preassigned number of times (from 1 to 11 correct recalls). Despite equating the numbers of successful recalls during practice, performance on a delayed final cued-recall test was lower for difficult than for easy items. Experiment 2 was designed to diagnose whether the disadvantage for difficult items was due to deficits in cue memory, target memory, and/or associative memory. The results revealed a disadvantage for the difficult versus the easy items only on the associative recognition test, with no differences on cue recognition, and even an advantage on target recognition. Although successful retrieval enhanced memory for both difficult and easy items, equating retrieval success during practice did not eliminate normative item difficulty differences.     

Bilinguals Show Weaker Lexical Access During Spoken Sentence Comprehension

When bilinguals process written language, they show delays in accessing lexical items relative to monolinguals. The present study investigated whether this effect extended to spoken language comprehension, examining the processing of sentences with either low or high semantic constraint in both first and second languages. English-German bilinguals, German-English bilinguals and English monolinguals listened for target words in spoken English sentences while their eye-movements were recorded. Bilinguals’ eye-movements reflected weaker lexical access relative to monolinguals; furthermore, the effect of semantic constraint differed across first versus second language processing. Specifically, English-native bilinguals showed fewer overall looks to target items, regardless of sentence constraint; German-native bilinguals activated target items more slowly and maintained target activation over a longer period of time in the low-constraint condition compared with monolinguals. No eye movements to cross-linguistic competitors were observed, suggesting that these lexical access disadvantages were present during bilingual spoken sentence comprehension even in the absence of overt interlingual competition.     

Cross-lingual numerical distance priming with second-language number words in native- to third-language number word translation

An important question in research about bilingualism is whether translation from a second language (L2) to the native language (L1) is semantically mediated or whether it occurs through word associations at the lexical level. Recent research has shown that both L1 and L2 number word translation imply semantic access, suggesting strong L2 lexicosemantic mappings for number words (Duyck & Brysbaert, 2004). In the present study, this assumption was further explored by means of a cross-lingual number distance priming paradigm with Dutch–English–French trilinguals. We found that number word translation from L1 to the third language (L3), and vice versa, was facilitated by L2 number masked word primes that represented the same magnitude as the target (translation equivalents), relative to primes that are numerically less close. This confirms the existence of strong L2 lexicosemantic mappings for number words and generalizes previous semantic effects in L1–L2 translation to translation processes between L1 and L3.     

Inhibition of non-target languages in multilingual word production: Evidence from Uighur–Chinese–English trilinguals

The present study examined the hypothesis whether non-target languages are inhibited during multilingual language production by examining the n-2 language repetition cost. In two experiments, Uighur–Chinese–English trilinguals named Arabic digits in one of their three languages according to a visually presented cue. Significant n-2 repetition costs were obtained in both experiments, which indicate that inhibition exists during multilingual word production. In addition, in Experiment 1, it was also found that the n-2 repetition cost was reduced when cues were highly compatible with the task, which means non-target languages are less inhibited. In Experiment 2, the n-2 repetition cost was increased at a shorter CSI. Taken together, these results indicate that inhibition of non-target languages occurs during multilingual language production, and that efficiency of establishing the target language task schema has an effect on the inhibitory control process.     

Concept mediation in trilingual translation: Evidence from response time and repetition priming patterns

Translation responses to individual words were elicited from 48 English-Spanish-French trilinguals, who translated in six directions at study and two directions at test. Patterns of translation response times and error rates at study reflected the relative proficiency of the trilinguals in comprehension and production of their three languages. At test, repeated items were translated more quickly than new items, with the strongest priming effects occurring for identical repetitions. Repetition priming was also substantial when only the stimulus language or only the response language matched from study to test, implying that repeated comprehension and production processes contribute to priming in translation. Patterns of response times and repetition priming indicate that translation in all directions involved conceptual access. Additive patterns in response time asymmetries and repetition priming were consistent with the treatment of word comprehension and production processes of translation as independent.     

Observed-Score Equating: An Overview

In this paper, an overview of the observed-score equating (OSE) process is provided from the perspective of a unifying equating framework (von Davier in von Davier (Ed.), Statistical models for test equating, scaling, and linking, Springer, New York, pp. 1–17, 2011b). The framework includes all OSE approaches. Issues related to the test, common items, and sampling designs and their relationship to measurement and equating are discussed. Challenges to the equating process, model assumptions, and approaches to equating evaluation are also presented. The equating process is illustrated step-by-step with a real data example from a licensure test.     

Does Bilingualism Change Native-Language Reading? Cognate Effects in a Sentence Context

ABSTRACT— Becoming a bilingual can change a person's cognitive functioning and language processing in a number of ways. This study focused on how knowledge of a second language influences how people read sentences written in their native language. We used the cognate-facilitation effect as a marker of cross-lingual activations in both languages. Cognates (e.g., Dutch-English schip [ship]) and controls were presented in a sentence context, and eye movements were monitored. Results showed faster reading times for cognates than for controls. Thus, this study shows that one of people's most automated skills, reading in one's native language, is changed by the knowledge of a second language.