A multi-criterion experimental comparison of three multi-attribute weight measurement methods

The research compared three major multi-attribute weight measurement methods using multiple criteria: theoretical validity, predictive performance, and perceived performance. Major findings include (1) all three methods are theoretically valid using the EAM (Equal Weight Averaging Model) as the standard of comparison, (2) EAM outperforms SMART (Simple Multi-Attribute Rating Technique) and SMART outperforms AHP (Analytical Hierarchy Process) in terms of predictive performance, (3) in terms of perceived performance, EAM is considered more difficult and less preferred than AHP, but has equal trustworthiness. The research has demonstrated how self-estimation methods for multi-attribute analysis can be verified and improved theoretically and empirically by using weights and values from the EAM model as a theoretical validity standard. © 1998 John Wiley & Sons, Ltd.     

Drug prevention in a community setting: A longitudinal study of the relative effectiveness of a three-year primary prevention program in boys & girls clubs across the nation

Tested the effectiveness of a youth drug prevention program in a community setting. Boys & Girls Clubs of America's Stay SMART program, adapted from a school-based personal and social competence drug prevention program, was offered, with and without a 2-year booster program, to 13-year-old members of Boys & Girls Clubs. Over 27 months, (a) 5 Boys & Girls Clubs offered the Stay SMART program, (b) 5 Boys & Girls Clubs offered the Stay SMART program with the booster programs, and (c) 4 Boys & Girls Clubs served as a control group. The Stay SMART program alone and the Stay SMART program with the booster programs showed effects for marijuana-related behavior, cigarette-related behavior, alcohol-related behavior, overall drug-related behavior, and knowledge concerning drug use. The Stay SMART program with the booster programs produced additional effects for alcohol attitudes and marijuana attitudes after each year of booster programs. Results suggest that a school-based personal and social competence program can be adapted effectively to a community setting and that booster programs might enhance program effects. Implications for alternative community models of prevention are discussed. At the time of the study, the parent organization was Boys Clubs of America.This research was supported by a grant from the Office for Substance Abuse Prevention, ADAMHA, Public Health Service, Department of Health and Human Services, Grant No. 1 H84 AD 00903-01.     

Comparing the effects of goal types in a walking session with healthy adults: Preliminary evidence for open goals in physical activity

ObjectivesGoal-setting is one of the most common strategies used to increase physical activity. Current practice is often based on specific, measurable, achievable, realistic, and time-bound (SMART) goals. However, theory and research suggests that this approach may be problematic. Open goals (e.g., “see how well you can do”) have emerged as a possible alternative, but are yet to be tested experimentally in physically active tasks. In a walking-based session, this study aimed to experimentally compare the effects of open, SMART and do-your-best goals with a control condition on distance walked and psychological variables related to engagement.DesignRepeated measures design (mixed model).MethodParticipants (N = 78; M_age = 55.88) were randomly assigned to one of four goal conditions: an open, SMART, or do-your-best goal, or a control condition (“walk at your normal pace”), before completing a baseline and two manipulated attempts of a 6-min walking test.ResultsOpen, SMART, and do-your-best goals achieved greater distance walked, and higher ratings of perceived exertion, than the control across both experimental attempts. Open and SMART goals led to greater enjoyment of the session. However, SMART goals led to higher pressure/tension, while open goals led to higher perceptions of performance and higher interest in repeating the session.ConclusionsThese findings provide preliminary evidence for the efficacy of setting open goals in physical activity, and suggest that they may be more psychologically adaptive to pursue than SMART or do-your-best goals. Implications are discussed, and recommendations are made for future goal-setting research in physical activity.     

SMARTS and SMARTER: Improved Simple Methods for Multiattribute Utility Measurement

This paper presents two approximate methods for multiattribute utility measurement, SMARTS and SMARTER, each based on an elicitation procedure for weights. Both correct an error in SMART, originally proposed by Edwards in 1977, and in addition SMARTER is simpler to use. SMARTS uses linear approximations to single-dimension utility functions, an additive aggregation model, and swing weights. The paper proposes tests for the usability of these approximations. SMARTER, based on a formally justifiable weighting procedure developed by Barron and Barrett, uses the same procedures as SMARTS except that it omits the second of two elicitation steps in swing weights, substituting calculations based on ranks. It can be shown to perform about 98% as well as SMARTS does, without requiring any difficult judgments from elicitees.     

A solution for measuring accurate reaction time to visual stimuli realized with a programmable microcontroller

This article presents a new solution for measuring accurate reaction time (SMART) to visual stimuli. The SMART is a USB device realized with a Cypress Programmable System-on-Chip (PSoC) mixed-signal array programmable microcontroller. A brief overview of the hardware and firmware of the PSoC is provided, together with the results of three experiments. In Experiment 1, we investigated the timing accuracy of the SMART in measuring reaction time (RT) under different conditions of operating systems (OSs; Windows XP or Vista) and monitor displays (a CRT or an LCD). The results indicated that the timing error in measuring RT by the SMART was less than 2 msec, on average, under all combinations of OS and display and that the SMART was tolerant to jitter and noise. In Experiment 2, we tested the SMART with 8 participants. The results indicated that there was no significant difference among RTs obtained with the SMART under the different conditions of OS and display. In Experiment 3, we used Microsoft (MS) PowerPoint to present visual stimuli on the display. We found no significant difference in RTs obtained using MS DirectX technology versus using the PowerPoint file with the SMART. We are certain that the SMART is a simple and practical solution for measuring RTs accurately. Although there are some restrictions in using the SMART with RT paradigms, the SMART is capable of providing both researchers and health professionals working in clinical settings with new ways of using RT paradigms in their work.     

Power Relations and Group Aggregation in the Multiplicative AHP and SMART

The relative power of the members in a group of decision makers can be incorporated in the multiplicative AHP via power coefficients in the logarithmic least squares whereby we analyse the pairwise comparison matrices. When each decision maker judges every pair of alternatives under each of the criteria, aggregation over the criteria and over the decision makers proceeds via a sequence of geometric-mean calculations which can be carried out in any order, at least with predetermined criterion weights and power coefficients. Hence, since we preserve the rank order of the alternatives, we avoid a deficiency of the original AHP. We also consider SMART, an additive method which is logarithmically related to the multiplicative AHP so that power relations can easily be incorporated in it. Finally, in order to illustrate the proposed model, we analyse a generalized version of the well-known example of Belton and Gear as well as the power relations between member countries of the European Community. © 1997 by John Wiley & Sons, Ltd. J. Multi-Crit. Decis. Anal. 6 : 155–165 (1997) No. of Figures: 0. No. of Tables: 6. No. of References: 23.     

The effects of goal types on psychological outcomes in active and insufficiently active adults in a walking task: Further evidence for open goals

ObjectivesThis study aimed to extend recent work on the effects of goal types in physical activity (PA; Swann, Hooper et al., 2020) by comparing the effects of SMART, open, and do-your-best (DYB) goals on performance and psychological responses in active and insufficiently active adults in a walking task.Design4 (goal condition) x 3 (attempt) x 2 (group) mixed design.MethodsActive (n = 18) and insufficiently active (n = 18) participants completed baseline and two experimental attempts of a 6-minute walking test in four conditions: SMART goal; open goal; DYB goal; and control. A range of measures were taken during and following each attempt, and after each session. A series of mixed ANOVA’s were conducted for all measures assessed between groups.ResultsInsufficiently active participants achieved greater distances in the open condition compared to the SMART condition (p < .001), whereas active participants achieved greater distances in the SMART condition compared to the open condition (p < .001). Additionally, exploratory analyses revealed that insufficiently active participants reported greater pleasure and enjoyment (p < .05) in the open condition compared to active participants, who conversely reported more pleasure and enjoyment (p < .05) in the SMART condition than insufficiently active participants.ConclusionsFindings provide initial evidence that PA and psychological responses differ between active and insufficiently active individuals depending on goal type. This work has potential implications for goal setting strategies in PA promotion and raises further questions about current practices of setting SMART goals for insufficiently active participants.     

Can SMART Training Really Increase Intelligence? A Replication Study

A burgeoning research stream supports the efficacy of a novel behavior-analytic intervention, known as SMART training, in raising general intelligence by training a set of crucial cognitive skills, referred to as relational skills. A sample of Irish secondary school students (n?=?26) was divided into two IQ matched groups, with the experimental group receiving 12 weeks of SMART training delivered in bi-weekly 45-min sessions. WASI IQ assessments were administered at baseline and follow-up to all participants by blind testers. For each of the three WASI IQ indices and the four IQ subtests, significant follow-up rises were found for the experimental group only. Analyses of variance indicated a significant effect of training on Verbal IQ, Matrix Reasoning and Vocabulary scores. Results lend further support for the efficacy of the SMART training program in enhancing intellectual skills.     

Scale sensitivity in the multiplicative AHP and SMART

We consider first a variant of the analytic hierarchy process (AHP) with a one-parametric class of geometric scales to quantify human comparative judgement and with a multiplicative structure: logarithmic regression to calculate the impact scores of the alternatives at the first evaluation level and a geometric-mean aggregation rule to calculate the final scores at the second level. We demonstrate that the rank order of the impact scores and final scores is scale-independent. Finally we show that the multiplicative AHP is an exponential version of the simple multi-attribute rating technique (SMART). In fact, the multiplicative AHP is concerned with ratios of intervals on the dimension of desirability, whereas SMART analyses differences in the corresponding orders of magnitude.     

Validation and error in multiplicative utility functions

An approach to the concept of error in utility assessment is proposed. Four kinds of errors are considered and each kind is related to four separate elicitation methods - all in the context of a general multiplicative multiattribute utility model. The methods are a Keeney-Raiffa (1976) procedure; SMART, for Simple Multi-Attribute Rating Technique (Edwards 1977); SJT, for a Social Judgment Theory based regression model (Hammond et al. 1975); and HOPE, for Holistic Orthogonal Parameter Estimation (Barron and Person 1979).The individual judgments elicited are either holistic - in which the entity to be evaluated is considered as a whole - or decomposed - in which attention is directed to one or two aspects at a time.If a general multiplicative model can be assumed to be an appropriate representation of the decision maker's basic preference structure, error can occur in the direct estimation of the scaling constants and univariate utility functions for decomposition methods (Keeney-Raiffa and SMART), or in the holistic assessments for holistic methods (SJT and HOPE). Individual estimates may be merely subject to random noise or may be substantially incorrect. The utility model may be incorrectly specified; finally, all four methods may be subject to systematic error. The four assessment methods are considered in conjunction with errors of each kind.     

Writing SMARTER goals for professional learning and improving classroom practices

Writing SMART Goals has recently become an integral part of the landscape of teacher professional learning in Australian schools, including both teacher professional development and performance management in classroom practices. This paper discusses the origins of SMART Goals and the diverse terms and definitions associated with such goal setting. Anecdotal evidence suggests that for some teachers, writing personal, professional or teaching and learning goals may be a daunting task. For school leadership teams, these goals often are the basis for operationalising school improvement over time. We make a case for clarity with SMART Goals within the context of each school, as interpreted and supported by school principals and leadership teams working with their staff. This paper concludes with some tentative suggestions for school leaders intended to improve SMART Goals, and teacher professional learning, over time. We hope this clarification and our suggestions support writing more effective learning goals for both teachers and students over time.     

On the measurement of preferences in the analytic hierarchy process

In this paper we apply multiattribute value theory as a framework for examining the use of pairwise comparisons in the analytic hierarchy process (AHP). On one hand our analysis indicates that pairwise comparisons should be understood in terms of preference differences between pairs of alternatives. On the other hand it points out undesirable effects caused by the upper bound and the discretization of any given ratio scale. Both these observations apply equally well to the SMART procedure which also uses estimates of weight ratios. Furthermore, we demonstrate that the AHP can be modified so as to produce results similar to those of multiattribute value measurement; we also propose new balanced scales to improve the sensitivity of the AHP ratio scales. Finally we show that the so-called supermatrix technique does not eliminate the rank reversal phenomenon which can be attributed to the normalizations in the AHP. © 1997 John Wiley & Sons, Ltd.     

Stochastic pairwise comparative judgements and direct ratings of alternatives in the REMBRANDT system

The REMBRANDT system for multicriteria decision analysis consists of both the multiplicative variant of the AHP (which employs a method of pairwise comparative judgements by a decision maker to arrive at final impact scores for the alternatives under consideration) and SMART, the simple multiattribute rating technique (which utilizes direct rating of alternatives to achieve final impact scores). This paper examines the effect of imprecision or uncertainty in the decision maker's pairwise judgements or ratings of alternatives by expressing each pairwise judgement or rating as a probability distribution, and the structure of REMBRANDT's component models is exploited to derive interval judgements or interval ratings of the alternatives’ final impact scores. These interval judgements or interval ratings can be used to determine the probability of rank reversal amongst alternatives, i.e. to assess the stability of the final impact score vector. © 1998 John Wiley & Sons, Ltd.     

Social desirability response bias as one source of the discrepancy between subjective weights and regression weights

This study of the judgment process investigated whether social desirability response bias is more closely associated with directly weighted subjective weights than less directly assessed regression weights. Graduating college students indicated their preferences for 11 job characteristics in four different tasks: (a) a statistical weighting task completed honestly, (b) a statistical weighting task completed to “look good” to a recruiter, (c) a subjective weighting task completed honestly, and (d) a subjective weighting task completed to “look good” to a recruiter. Predicted judgments were calculated from the weights obtained in each condition. Pearson correlation coefficients using these predicted judgments were computed between conditions; the relationship between the honest and positive impression predicted judgments derived from subjective weights was significantly greater than the relationship between honest and positive impression predicted judgments derived from regression weights for 19 of 23 participants. It was concluded that social desirability response bias was more closely related to subjective weights than to regression Weights.     

Weight approximations in multi‐attribute decision models

The use of surrogate weights based on rankings has been proposed as a method for avoiding difficulties associated with the elicitation of weights in multi‐attribute decision analysis. When the simple multiattribute rating technique using swings (SMARTS) method is being employed it has been suggested that rank order centroid (ROC) weights are the best surrogate weights to use. This study shows that ROC weights are appropriate to use as a substitute for original weights that are constrained to sum to a fixed total (usually 1 or 100) as used in the point allocation method. If, however, the original weights are determined without any initial restrictions, as in the direct rating method, and are then normalized, which is the common procedure in SMARTS analysis, then the ROC weights do not provide the best approximations to the original weights. This paper shows how to obtain rank order distribution (ROD) weights that provide a better approximation than the ROC approach to unrestricted original weights. The paper also shows that, as the number of attributes in a decision problem increases, the ROD weights approximate to the more easily calculated rank sum weights. Copyright © 2003 John Wiley & Sons, Ltd.     

Experimental consideration of preference in decision making under certainty

This paper describes an experiment in decision making under certainty with multiple, conflicting objectives and continuous decision variables. Two techniques for analysing such problems are considered: one taken from the paradigm of multicriteria decision making (MCDM), a non-directed approach called the NAIVE technique, and one from the paradigm of multiattribute decision analysis (D/A), the SMART technique. While the two techniques seek and are throught to arrive at the same end—a solution which is in some sense optimal for the decision maker (DM)—the former approach implicitly incorporates DM preferences while the latter approach considers preferences explicitly. The setting is a laboratory study using a sample of university students on a three-criteria problem which is designed to study the extent to which value functions implied/assessed by the techniques are consistent with DMs' holistic ranking of alternatives. Results show that (1) the two techniques of interest show significantly better rank order correlation with holistic judgement compared with other techniques, (2) DMs prefer the non-directed MCDM approach and (3) subjects break down into two groups: those that use assessable value functions when ranking and those that do not. This implies that for small-dimensioned problems DMs may first need to be classified as to the assessability of their value functions before a solution method is chosen.     

Identifying and prioritizing factors influencing the selection of the top suppliers of e‐procurement using FDEMATEL and FANP

The aim of this study is to identify, rank, and determine the importance of factors influencing the selection of the top supplier of electronic procurement in organizations using a hybrid approach. This is a case study of Irancell Telecommunication Service Company in which fuzzy decision‐making trial and evaluation (FDEMATEL) and fuzzy analytical network process (FANP) were used. FDEMATEL was employed to determine the interaction and interrelationships among the identified criteria and subcriteria, while utilizing FANP to calculate the weights of the criteria and their relevant subcriteria to determine their priorities. Therefore, the research literature was first reviewed to identify the factors influencing the selection of suppliers. For this purpose, the experts on communication networks were interviewed to select 16 final factors as the most important subcriteria that were then categorized as 5 groups (technology, environmental, services, organizational, and commercial) on which the study was based. After extracting the criteria for ranking the suppliers, codifying evaluation questionnaire, and converting the opinions in the Likert scale, 16 criteria were finally chosen with scores above the mean. FDEMATEL was used to determine the weights of subcriteria. In the final step, FANP was employed for prioritization. Results indicate that the organizational criterion is the most influential factor; however, the criterion services is the most permeable factor. The greatest weights were obtained from production capacity and provisions and geographical situation which were weighted 0.097 and 0.039, respectively. They are also regarded as the most and least important criteria for the selection of the top supplier of e‐procurement by the experts.     

A medication screening tool for cognitive aging researchers

A Web-based medication screening tool that provides researchers with information about side effects associated with medications commonly used by older participants is described. This tool can be used for research purposes to better separate the effects of normal, healthy aging processes from the deleterious effects often associated with medication usage. Researchers can use this tool, called SMART (Screening Medications: Aging Research Taxonomy), to (a) obtain information pertaining to the cognitive, sensory, and motor side effects associated with specific medications and (b) screen medications in their research for side effect severity. The Web address for the tool is www.psychology.gatech.edu/SMART     

Refusal Self‐Efficacy Among SMART Recovery Members by Affiliation Length and Meeting Frequency

Using the Brief Situational Confidence Questionnaire, the authors examined the refusal self‐efficacy of 105 Self‐Management and Recovery Training (SMART Recovery) members according to their affiliation length and meeting frequency. Results demonstrated that longer affiliation with SMART Recovery and higher meeting frequency significantly enhanced substance refusal self‐efficacy.     

高校教师教学水平评价多元概化理论权重效应分析

探讨高校教师教学水平评价多元概化理论合成信度的权重效应。使用"高校教师教学水平评价问卷",要求534名学生对16名教师进行评价,对收集到的数据作嵌套设计的多元概化理论分析。结果表明：（1）在高校教师教学水平评价中,多元概化理论合成信度估计权重、先验权重和效果权重产生不同效应;（2）结合高校教师教学水平评价,在充分分析三种权重对合成信度影响的基础上,提出了多元概化理论"合成信度三种权重效应分析模式图",能为正确使用多元概化理论权重提供科学参考。