Stochastic pairwise comparative judgements and direct ratings of alternatives in the REMBRANDT system

The REMBRANDT system for multicriteria decision analysis consists of both the multiplicative variant of the AHP (which employs a method of pairwise comparative judgements by a decision maker to arrive at final impact scores for the alternatives under consideration) and SMART, the simple multiattribute rating technique (which utilizes direct rating of alternatives to achieve final impact scores). This paper examines the effect of imprecision or uncertainty in the decision maker's pairwise judgements or ratings of alternatives by expressing each pairwise judgement or rating as a probability distribution, and the structure of REMBRANDT's component models is exploited to derive interval judgements or interval ratings of the alternatives’ final impact scores. These interval judgements or interval ratings can be used to determine the probability of rank reversal amongst alternatives, i.e. to assess the stability of the final impact score vector. © 1998 John Wiley & Sons, Ltd.     

Power Relations and Group Aggregation in the Multiplicative AHP and SMART

The relative power of the members in a group of decision makers can be incorporated in the multiplicative AHP via power coefficients in the logarithmic least squares whereby we analyse the pairwise comparison matrices. When each decision maker judges every pair of alternatives under each of the criteria, aggregation over the criteria and over the decision makers proceeds via a sequence of geometric-mean calculations which can be carried out in any order, at least with predetermined criterion weights and power coefficients. Hence, since we preserve the rank order of the alternatives, we avoid a deficiency of the original AHP. We also consider SMART, an additive method which is logarithmically related to the multiplicative AHP so that power relations can easily be incorporated in it. Finally, in order to illustrate the proposed model, we analyse a generalized version of the well-known example of Belton and Gear as well as the power relations between member countries of the European Community. © 1997 by John Wiley & Sons, Ltd. J. Multi-Crit. Decis. Anal. 6 : 155–165 (1997) No. of Figures: 0. No. of Tables: 6. No. of References: 23.     

Validation and error in multiplicative utility functions

An approach to the concept of error in utility assessment is proposed. Four kinds of errors are considered and each kind is related to four separate elicitation methods - all in the context of a general multiplicative multiattribute utility model. The methods are a Keeney-Raiffa (1976) procedure; SMART, for Simple Multi-Attribute Rating Technique (Edwards 1977); SJT, for a Social Judgment Theory based regression model (Hammond et al. 1975); and HOPE, for Holistic Orthogonal Parameter Estimation (Barron and Person 1979).The individual judgments elicited are either holistic - in which the entity to be evaluated is considered as a whole - or decomposed - in which attention is directed to one or two aspects at a time.If a general multiplicative model can be assumed to be an appropriate representation of the decision maker's basic preference structure, error can occur in the direct estimation of the scaling constants and univariate utility functions for decomposition methods (Keeney-Raiffa and SMART), or in the holistic assessments for holistic methods (SJT and HOPE). Individual estimates may be merely subject to random noise or may be substantially incorrect. The utility model may be incorrectly specified; finally, all four methods may be subject to systematic error. The four assessment methods are considered in conjunction with errors of each kind.     

A multi-criterion experimental comparison of three multi-attribute weight measurement methods

The research compared three major multi-attribute weight measurement methods using multiple criteria: theoretical validity, predictive performance, and perceived performance. Major findings include (1) all three methods are theoretically valid using the EAM (Equal Weight Averaging Model) as the standard of comparison, (2) EAM outperforms SMART (Simple Multi-Attribute Rating Technique) and SMART outperforms AHP (Analytical Hierarchy Process) in terms of predictive performance, (3) in terms of perceived performance, EAM is considered more difficult and less preferred than AHP, but has equal trustworthiness. The research has demonstrated how self-estimation methods for multi-attribute analysis can be verified and improved theoretically and empirically by using weights and values from the EAM model as a theoretical validity standard. © 1998 John Wiley & Sons, Ltd.     

On the measurement of preferences in the analytic hierarchy process

In this paper we apply multiattribute value theory as a framework for examining the use of pairwise comparisons in the analytic hierarchy process (AHP). On one hand our analysis indicates that pairwise comparisons should be understood in terms of preference differences between pairs of alternatives. On the other hand it points out undesirable effects caused by the upper bound and the discretization of any given ratio scale. Both these observations apply equally well to the SMART procedure which also uses estimates of weight ratios. Furthermore, we demonstrate that the AHP can be modified so as to produce results similar to those of multiattribute value measurement; we also propose new balanced scales to improve the sensitivity of the AHP ratio scales. Finally we show that the so-called supermatrix technique does not eliminate the rank reversal phenomenon which can be attributed to the normalizations in the AHP. © 1997 John Wiley & Sons, Ltd.     

A solution for measuring accurate reaction time to visual stimuli realized with a programmable microcontroller

This article presents a new solution for measuring accurate reaction time (SMART) to visual stimuli. The SMART is a USB device realized with a Cypress Programmable System-on-Chip (PSoC) mixed-signal array programmable microcontroller. A brief overview of the hardware and firmware of the PSoC is provided, together with the results of three experiments. In Experiment 1, we investigated the timing accuracy of the SMART in measuring reaction time (RT) under different conditions of operating systems (OSs; Windows XP or Vista) and monitor displays (a CRT or an LCD). The results indicated that the timing error in measuring RT by the SMART was less than 2 msec, on average, under all combinations of OS and display and that the SMART was tolerant to jitter and noise. In Experiment 2, we tested the SMART with 8 participants. The results indicated that there was no significant difference among RTs obtained with the SMART under the different conditions of OS and display. In Experiment 3, we used Microsoft (MS) PowerPoint to present visual stimuli on the display. We found no significant difference in RTs obtained using MS DirectX technology versus using the PowerPoint file with the SMART. We are certain that the SMART is a simple and practical solution for measuring RTs accurately. Although there are some restrictions in using the SMART with RT paradigms, the SMART is capable of providing both researchers and health professionals working in clinical settings with new ways of using RT paradigms in their work.     

Think fast: The role of thin slices of behavior in employee selection decisions

This study integrates past research addressing decision‐making, employee selection, and influence processes in an attempt to provide both a theoretical and empirical foundation for future research addressing initial impressions throughout the interview process. Using data from a simulated hiring situation comprised of 28 recruiters and 229 applicants, the results suggest that initial impressions formed at the beginning of the interview make a substantive impact on final impressions. However, impressions formed at the career fair do not appear to impact final impressions without considering the interactive effects of decision confidence. Hypotheses proposing that decision confidence would moderate linkages between initial impressions formed at the beginning of the interview and both interview scores and final impressions were not supported. We discuss the theoretical and practical implications of these findings for selection.     

Can SMART Training Really Increase Intelligence? A Replication Study

A burgeoning research stream supports the efficacy of a novel behavior-analytic intervention, known as SMART training, in raising general intelligence by training a set of crucial cognitive skills, referred to as relational skills. A sample of Irish secondary school students (n?=?26) was divided into two IQ matched groups, with the experimental group receiving 12 weeks of SMART training delivered in bi-weekly 45-min sessions. WASI IQ assessments were administered at baseline and follow-up to all participants by blind testers. For each of the three WASI IQ indices and the four IQ subtests, significant follow-up rises were found for the experimental group only. Analyses of variance indicated a significant effect of training on Verbal IQ, Matrix Reasoning and Vocabulary scores. Results lend further support for the efficacy of the SMART training program in enhancing intellectual skills.     

On the Death of SMART and the Birth of GRAPA

Should SMART be dead—as Edwards and Barron affirmed in 1994? This article is a comparative study of the methods SMART and SMARTS, in relation to the eliciting of weights. It compares the weights and final rankings of alternatives. To these two techniques has been added a graphical version of the PA (GRAPA), which is simpler and more intuitive. My study is designed to orient nonprofessional decision-makers with regard to which technique to employ, when what is sought is simplicity and ease of use without loss of rigor.     

Two new cases of rank reversals when the AHP and some of its additive variants are used that do not occur with the multiplicative AHP

Many researchers have long observed some cases in which certain ranking irregularities can occur when the original analytic hierarchy process (AHP), or some of its variants, are used. This paper presents two new categories of ranking irregularities which defy common intuition. These ranking irregularities occur when one decomposes a decision problem into a set of smaller problems each defined on two alternatives and the same criteria as the original problem. These irregularities are possible when the original AHP, or some of its additive variants, are used. Computational experiments on random test problems and an examination of some real‐life case studies suggest that these ranking irregularities are dramatically likely to occur. This paper also proves that these ranking irregularities are not possible when a multiplicative variant of the AHP is used. Copyright © 2001 John Wiley & Sons, Ltd.     

Instructional development for teachers in higher education: Effects on students’ perceptions of the teaching–learning environment

Background. Although instructional development for teachers has become an important topic in higher education, little is known about its actual impact. In particular, evidence regarding the impact of teachers’ instructional development on students’ perceptions of the teaching–learning environment is scarce. Aims. The impact of an instructional development programme for beginning university teachers on students’ perceptions of the teaching and learning environment was investigated. We also explored whether this impact is dependent on class size and student level (first years vs. non‐first years). Sample. Quantitative data were gathered from more than 1,000 students at pre‐ and post‐tests, using a quasi‐experimental design. Method. A multi‐level analysis was conducted in which five models were estimated. Results. A basic model made clear that teachers did differ from each other with respect to the dependent variables concerned; however, differences in scale scores also resulted to a large extent from differences between students. A second model, in which the moderating impact by way of teacher characteristics, context, and student characteristics was not taken into account, reported no significant effect of training. A third model, examining the net impact of instructional development revealed some impact, which was, remarkably, negative. A first interaction model proved a differential impact of instructional development for teachers teaching first years and those teaching non‐first years. A second one showed that the impact of training depended on the number of students one teaches. Conclusions. Instructional development for teachers in higher education does not easily result in effects on students’ perceptions of the teaching and learning environment. Perspectives for further research into instructional development are discussed.     

Aging,Task Complexity,and Efficiency Modes: The Influence of Working Memory Involvement on Age Differences in Response Times for Verbal and Visuospatial Tasks

ABSTRACT

We examined the information-processing functions (response-time × load) of younger and older adults for two verbal and one visuo-spatial task; each task was implemented in a baseline and a high-complexity condition. Heightened complexity transformed the baseline functions in either an additive or a multiplicative fashion. The processing efficiency of older adults was defined as the old-young ratio of the slopes of the load functions. Three levels of efficiency could be distinguished. The first level, with an age-related slowing factor of about 1.2, consisted of low-complexity verbal processing and additive-complexity verbal processing. The second level, associated with a slowing factor of about 1.6, consisted of a mixture of verbal-high-multiplicative-complexity processing and visuo-spatial-low-complexity processing. The third level, with a slowing factor of about 4, consisted of visuo-spatial processing of high multiplicative complexity. The results go against any common factor theory of aging. Instead, they suggest that a shift from a higher to a lower mode of efficiency is triggered by a greater degree of working memory involvement.     

Aging, task complexity, and efficiency modes: the influence of working memory involvement on age differences in response times for verbal and visuospatial tasks

We examined the information-processing functions (response-time x load) of younger and older adults for two verbal and one visuo-spatial task; each task was implemented in a baseline and a high-complexity condition. Heightened complexity transformed the baseline functions in either an additive or a multiplicative fashion. The processing efficiency of older adults was defined as the old-young ratio of the slopes of the load functions. Three levels of efficiency could be distinguished. The first level, with an age-related slowing factor of about 1.2, consisted of low-complexity verbal processing and additive-complexity verbal processing. The second level, associated with a slowing factor of about 1.6, consisted of a mixture of verbal-high-multiplicative-complexity processing and visuo-spatial-low-complexity processing. The third level, with a slowing factor of about 4, consisted of visuo-spatial processing of high multiplicative complexity. The results go against any common factor theory of aging. Instead, they suggest that a shift from a higher to a lower mode of efficiency is triggered by a greater degree of working memory involvement.     

The Multiplicative AHP,SMART and ELECTRE in a Common Context

This paper presents a comparative study of three popular methods for multicriteria decision analysis based on a particular model of human preferential judgement. Since decisions are invariably made within a given context, we model relative preferences as ratios of increments or decrements in an interval on an axis of desirability. Next we sort the ratio magnitudes into a small number of categories, represented by numerical values on a geometric scale. We explain why the analytic hierarchy process (AHP) and the French collection of ELECTRE methods, typically based on pairwise comparison methods, are concerned with categories of ratio magnitudes, whereas the simple multiattribute rating technique (SMART) essentially uses orders of magnitude of these ratios. This phenomenon provides a common basis for the analysis of the methods in question and for a cross-validation of their results. We illustrate the approach via a well-known case study, the choice of a location for a nuclear power plant. We conclude by discussing the scope of the comparative study. © 1997 John Wiley & Sons, Ltd.     

归类优势与基本水平效应的再探讨

类别特异性分数是预测归类优势的有效指标。实验1考察该分数能否预测基本水平归类优势, 结果：分数较高的表现归类优势。实验2检验有归类优势的下位类别的分数, 结果：它们都有较高分数, 同时发现下位类别的特异性可以影响分数。实验3探讨相同的基本水平与特异性不同的下位组合的归类表现, 结果：下位特异性低就表现基本水平归类优势, 高则相反。据此提出“经验说”, 分数高低与人们在经验中对各层次类别形成的表征特异性程度相关。     

Personality in proportion: a bipolar proportional scale for personality assessments and its consequences for trait structure

Trait structures resulting from personality assessments on Likert scales are affected by the additive and multiplicative transformations implied in interval scaling and correlational analysis. The effect comes into view on selecting a plausible alternative scale. To this end, we propose a bipolar bounded scale ranging from -1 to +1 representing an underlying process in which the assessor would review and discount positive and negative behavioral instances of a trait. As an appropriate index of likeness between variables X and Y, we propose LXY = SigmaXY/N, the average of the raw scores cross products. Using this index, we carried out a raw scores principal component analysis on data consisting of 133 participants who had each been rated by 5 assessors including self on 914 items. Contrary to the Big-Five structure that was found in these data on standard analysis, the results showed a relatively large first principal component F1 and 2 very small ones, F2 and F3. The sizes LFF=SigmaF2/N, the averages of the squared component scores, were modest to small. It thus appears that the scale, bipolar proportional versus standard, has a profound impact on the size and structure of personality assessments. The dissimilarity remains on analyzing self-ratings rather than averaged (over the 5 assessors) ratings.     

Complexity in the relationships among the subdimensions of extraversion and job performance in managerial occupations

The present study investigated differential relationships, nonlinear relationships, and multiplicative relationships among the subdimensions of extraversion (agency and affiliation) and job performance (getting ahead and getting along) for a sample of 179 managers. We found that: (i) agency was positively related to getting ahead performance, whereas affiliation was unrelated to getting ahead; (ii) agency had an inverted‐U relationship with getting along performance; and (iii) there was a multiplicative effect of agency and affiliation on getting along. The findings provide insight into the nature of the relationship between extraversion and managerial performance, the level of breadth at which to represent these variables for optimal prediction and explanation, and the use of extraversion measures for selection purposes.     

学习率的心理现实性及其对项目选择的影响

项目选择一直是元认知控制研究中的热点问题。本研究以不同难度和分值的计算题为实验材料,通过两个实验探讨学习率的心理现实性及其对项目选择的影响。实验1中,在不限时条件下,被试者需要完成不同难度的计算题并赋予不同分值。实验2中,在限时条件下,通过变化计算题的难度和分值设计了三种不同学习率的项目,被试只能选择其中一种项目来计算以获得更高的分值。研究表明：第一,当项目所用时间增多时,被试对该项目所赋分值就增大,而时间（难度）与分值的比例是不变的,即学习率是相同的;第二,被试倾向于选择学习率更高的项目来完成计算任务,而当学习率相同时,被试倾向于优先选择高分困难项目。研究证实了学习率的心理现实性,并确定了学习率是项目选择的主要依据。     

An Alternate Method of Deriving Priorities and Related Inferences for Group Decision Making in Analytic Hierarchy Process

Some work has been carried out in the past on statistically deriving priorities in Analytic Hierarchy Process (AHP). In AHP, the aggregated worths of the alternatives, when compared with respect to several criteria, are estimated in a hierarchical comparisons model introduced by Saaty. In this setup, statistical models are used for Saaty's method of scaling in paired comparisons experiments in any level of the hierarchy. At the end, the final priority weights of the alternatives and related inferences are developed with appropriate statistical methods. Existing statistical methods in the literature assume independence of the entries of the paired comparison matrix. However, these entries are highly dependent among themselves. In this article, we propose a statistical method that allows for the dependence among the entries of the pairwise comparisons matrix. The proposed method is then illustrated with a numerical example. Copyright © 2011 John Wiley & Sons, Ltd.     

Writing SMARTER goals for professional learning and improving classroom practices

Writing SMART Goals has recently become an integral part of the landscape of teacher professional learning in Australian schools, including both teacher professional development and performance management in classroom practices. This paper discusses the origins of SMART Goals and the diverse terms and definitions associated with such goal setting. Anecdotal evidence suggests that for some teachers, writing personal, professional or teaching and learning goals may be a daunting task. For school leadership teams, these goals often are the basis for operationalising school improvement over time. We make a case for clarity with SMART Goals within the context of each school, as interpreted and supported by school principals and leadership teams working with their staff. This paper concludes with some tentative suggestions for school leaders intended to improve SMART Goals, and teacher professional learning, over time. We hope this clarification and our suggestions support writing more effective learning goals for both teachers and students over time.