Evaluation from the inside: Participatory evaluation of agricultural research in the Philippines

This paper discusses the experience of a Philippines-based agricultural research program, where participatory evaluation is embedded in a broader, user-centered participatory research approach. Three case projects illustrate and analyze participatory evaluation of agricultural research in a developing country context. Different evaluation types are identified and their use in different phases of the research process is discussed. These field experiences show how “evaluation from the inside” can contribute to effective research planning and implementation, particularly in enhancing sensitivity to user needs and situations. network for user participatory rootcrop R&D sponsored by the International Potato Center in Asia. Under his leadership, UPWARD has increasingly sought to build participatory monitoring and evaluation into the network’s research and development activities. He has a Ph.D. in communication and innovation studies from Wageningen Agricultural University in The Netherlands. Prior joining UPWARD, Campilan worked with the International Institute of Rural Reconstruction and the Philippine Root Crop Research and Training Center. His research interests include communication of innovations, institutional linkage development, participatory research methods and tools, and strengthening local knowledge systems. From 1991 until 1997 he was coordinator of Users’ Perspectives With Agricultural Research and Development (UPWARD) network. He was previously based in Latin America. His main research interests include the socioeconomic and cultural aspects of crop genetic diversity conservation and utilization involving ethnobotanical studies, on-farm conservation, and work on seed systems. He is also interested in research on rural enterprise development. He has been actively involved in capacity building initiatives among national agricultural researchers, especially in participatory research methods and planning techniques. He can be contacted CIP-ESEAP, Kebun Percobaan Muara, Jalan Raya Ciapus, Bogor 16610, Indonesia, fax (62 251) 316 264, e-mail: G.Prain@cgiar.org. Her major responsibilities include facilitating the network’s activities on sustainable crop management R&D and on capacity building in participatory approaches and methods. At the UPWARD coordinating office, she is in charge of training, publications, and information management. She has extensive training and hands on experience in the use of participatory methods and tools, particularly through a Philippines project on soil resource management for sweetpotato production. She has an MSc in family resource management and development communication from the University of the Philippines at Los Ba?os. Her research interests include sustainable crop management, strengthening local R&D capacity, and field testing participatory methods and tools.     

The WAIS Picture Arrangement Subtest as a Measure of Anticipation

Summary

Recent research and theory has stressed the importance of future extension as a parameter in psychological development and in psychopathology. One of the primary methods for assessing future extension has been through fantasy productions to TAT cards or to story stems. Both of these methods have been useful but they are limited because they are dependent in part on verbal fluency. It has been suggested that the Picture Arrangement (PA) subtest of the Wechsler Adult Intelligence Scale assesses anticipation and planning and, as such, it provides a non-verbal measure of future extension. PA scores were found in a pilot study to have a positive and significant relationship to future extension as measured in stories told to story roots, and in the present study to future extension as measured in TAT stories.     

Evaluation of agricultural research in Eastern,Central, and Southern Africa

Research and Development (R&D) evaluation within agriculture is becoming increasingly important as a planning tool in the research management process in eastern, central, and southern Africa. Evaluation of agricultural research in the region is performed at various levels for different purposes. This article traces the evolution of R&D activities, and looks at the current status of the agricultural research evaluation and the capacity to undertake such evaluations as a part of research management-both at the national and regional levels. It critically examines the experiences of the region in evaluating agricultural research and the efforts made to promote such activities, summarizing the lessons learned. Finally, the article analyzes the critical constraints impeding the successful adoption of the agricultural research evaluation process and offers suggestions, which could alleviate these constraints. From 1993 to 1997, he was an advisor on impact evaluation and policy analysis to the Southern Africa Center for Cooperation in Agricultural and Natural Resources Research and Training (SACCAR), in Botswana. Graduated in agricultural economics from the University of Guelph, Canada, he started his professional career in Africa, at the University of Dar-es-Salaam in 1979. From 1982 to 1991, he worked for CIMMYT, and was then a senior economist at the Australian Bureau of Agricultural and Resource Economics in Canberra. He has worked and published in a range of subjects, including agricultural policy and development, agricultural marketing, production economics, farming systems research, capacity building, and impact assessment. Anandajayasekeram is the current president of the Association for Farming Systems Research and Extension and the Southern African Association for Farming Systems Research and Extension. Dr. David R. Martella has, since 1991, been Regional Agricultural Advisor at the United State Agency for International Development, Regional Economic Development Support Office for East and Southern Africa, Nairobi, Kenya. He previously worked as an agricultural economist and advisor for USAID in Mozambique. Martella has fifteen years of experience in farming and related enterprises as owneroperator in California, in Mexico, and in Swaziland. He also has sixteen years of research and program management experience in Africa. His areas of specialization include economic theory, econometrics and quantitative methods, and experimental design. Martella has a graduate degree in agricultural economics from Purdue University.     

Whither (European) evaluation methodology?

Quasi-experimental evaluation methods promised to provide a way of testing and improving solutions to social problems. These methods have been found wanting. Alternative approaches, including “pragmatic evaluation,” “naturalistic evaluation,” and “pluralist evaluation” are also shown to be unsatisfactory. The initial promise of the evaluation movement has been disappointed. Realistic evaluation is advocated as an alternative to existing forms of evaluation. It is rooted in some European traditions in epistemology, ontology, and social theory. It offers a framework within which rigorous outcome evaluations can be undertaken with results that can inform policy and practice. He is president of the Research Committee on Methodology of the International Sociological Association. He is seconded part-time to the Home Office Police Research Group as a research consultant. An earlier version of this paper was presented at the founding conference of the European Evaluation Society, The Hague, 1–2 December, 1994.     

内隐自尊的研究趋势及测量方法

内隐自尊是社会认知研究的新领域。本文主要回顾了内隐自尊研究的趋势及测量方法的进展。内隐自尊的研究趋势主要表现为：内隐与外显自尊研究对照进行;应用性研究初步开展;研究方法的不断更新;与其它社会认知结构的关系;与行为的关系等。目前在研究方法上以反应时范式为主,最常用是内隐联想测验,还包括阈上语义启动、阈下语义启动、姓名字母与数字偏爱、Stroop颜色命名任务、内隐自我评价调查和外部情绪性Simon任务等。     

Evaluation of agricultural research in Latin America and the Caribbean

This article assesses the state of evaluation, and identifies priorities for improving evaluation, in agricultural research organizations in the region of Latin America and the Caribbean in the early 1990s. Based on thirteen case studies conducted in 1992, the article describes the institutional settings and regional patterns, and trends in evaluation practice. Illustrative cases from Argentina, Brazil, and Guatemala are presented. The organizations studied have extensive experience with evaluation; however, this experience has not been well documented or shared. Evaluation is generally the weakest phase in the management cycle. In the past, most evaluations have been extrinsically motivated, and as such, they have been of little use to local researchers and managers. Obstacles to improving evaluation include the centralization of administrative systems, weak program management, a lack of understanding of potential uses of evaluation in management, and limited knowledge of appropriate evaluation methods. Agricultural research managers feel that evaluation training should be provided as one component of a broader effort covering planning, monitoring, and evaluation. Since joining ISNAR in 1990, he has engaged in research, training, and advisory work on research management, with an emphasis on evaluation. Previously, for fifteen years Horton was head of the social science department of the International Potato Center in Peru. Horton received B.s. and M.S. degrees in agricultural economics from the University of Illinois and a Ph.D. in economics from Cornell University. His current interests include action research and learning, organizational assessment, and institutionalization of planning, monitoring, and evaluation. He worked for fourteen years at the Brazilian Corporation for Agricultural Research, conducting research and development activities in the areas of human resources and of strategic management. He is a full professor at the Department of Social and Work Psychology, Institute of Psychology, University of Brasilia, Brazil, where he teaches and advises at the undergraduate and graduate levels and does research and consulting. His current areas of interest are organizational behavior, training, organizational evaluation, and science and technology management.     

Self-efficacy,planning, and preparatory behaviours as joint predictors of physical activity: A conditional process analysis

Objective: Planning can bridge the gap between intentions and action, but what bridges the gap between planning and action? This study helps to answer the question by disentangling the interrelationships between self-efficacy, planning and preparatory behaviours in predicting physical activity. Preparatory behaviours are tested as a working mechanism of planning. Moreover, it is tested whether the utility of preparatory behaviours depends on an individual’s level of self-efficacy.

Methods: A survey assessed planning, self-efficacy and preparatory behaviours for physical activity. Adults (N = 166) provided data at two measurement points. In a longitudinal model, preparatory behaviours were specified as a mediator between planning and physical activity. Self-efficacy was specified as a possible moderator at two points in the model.

Results: Preparatory behaviours mediated the relationship between planning and physical activity. An interaction between self-efficacy and preparatory behaviours on physical activity was found, indicating that individuals with low self-efficacy beliefs were more active if they engaged more frequently in preparatory behaviours.

Conclusion: Planning seems to stimulate preparatory behaviours, which in turn make future physical activity more likely. Furthermore, as performing preparatory behaviours represent a step forward towards the enactment of behavioural goals, preparatory behaviours may be particular beneficial for individuals afflicted by self-doubts regarding physical activity.     

Crime problems of the future

Group process methods for problem solving and planning are now widely used in organizations in the United States. Such methods, which involve active participation by employees, are not often used in Russia. We believe these methods would help Russia move from a centrally planned, authoritarian style of management to a more participatory, information-sharing style of management. Accordingly, two training sessions were held with faculty members at universities in Irkutsk and Novosibirsk. This article describes how these meetings were arranged, the results of the planning activities, and the implications of participatory methods of decision making for organizations in Russia and in other transitional economies.     

MCDM methods in strategic planning of forestry on state‐owned lands in Finland: applications and experiences

The forests in Finland have been under intensive planning for decades. Currently, mathematical programming is widely used in planning of wood production. Today's multi‐functional forestry, however, calls for more flexible decision support methods. MCDM tools have been used in responding to fresh planning challenges. For example, the Finnish Forest and Park Service, entrusted with the care of the vast majority of state‐owned natural resources in Finland, endeavours to produce large‐scale natural resource plans satisfying the needs of both economic, social, and ecological sustainability. Participatory approach is applied in the process. Several forestry applications of MCDM methods, particularly those making use of the AHP or the HIPRE program, have been presented. Also, the outranking methods ELECTRE and PROMETHEE have been tested. Due to the nature of forestry applications, statistical techniques for analysing uncertainties in pairwise comparisons and for utilizing interval judgement data have been developed to improve the usability of the AHP. Recently, a hybrid method called A'WOT, making use of the AHP and SWOT, was also introduced into strategic forest planning. This paper summarizes the experiences gained in applying a MAVT and two outranking methods in connection with a participatory natural resource planning process in Finland. In addition, some results of the method development work related to application needs are briefly presented. The details of the planning cases reviewed here have previously been presented in forestry journals. The purpose of this paper is not only to show how MCDM methods have been applied in forestry, but also to discuss the usability and usefulness of MCDM methods from the viewpoint of supporting forestry decision making—and how they might further be improved. Also, some perspectives for the future development work of MCDM applications in the field of natural resource management are focused on. As a conclusion, the use of more than just one MCDM method in a single planning process is seen usually recommendable. In addition, developing hybrid MCDM methods is regarded as a potential direction for future research. Also, closer co‐operation between method developers and appliers is called for to produce more useful applications. Copyright © 2002 John Wiley & Sons, Ltd.     

Industrial Salesforce Motivation and Herzberg's Dual Factor Theory: A UK Perspective

Abstract

This study examines salespersons' views on the job-related factors that motivate and dissatisfy them. There is a discussion of literature pertaining to Herzberg's theory of motivation which is tested by the results of this survey and questioned as a basis for salesforce motivation planning. The findings of the research are explained and their implications for use in salesforce management are offered.     

Technological semantics and technological practice: Lessons from an enigmatic episode in twentieth-century technology studies

This paper is a review of words and their meanings in the field of technology studies, and an analysis the semantics of an idealistic international technology-related social movement that flourished briefly during the second half of the twentieth century. Sloppy nomenclature employed by proponents and observers of the movement led to people with opposite views appearing to agree (and vice versa), with the consequence that the movement’s valuable policy insights exerted only marginal influence on mainstream technology policy. I conclude that poor technological semantics may undermine effective technological practice. Suggestions for a constructive technological nomenclature are presented. His research, teaching, and consulting concentrate on strategic technology management, intellectual property management, technology-based entrepreneurship, technology-based industry development, and regional economic development planning. He holds doctoral degrees in both technology studies and strategic management. Kelvin has conducted research or directed programs in technology management at the State University of New York at Stony Brook, the University of Utah, and the University of California at Berkeley. He is also an active participant in the Entrepreneurial Technology Apprenticeship Program of the U.S. Minority Business Development Administration and the National Technology Transfer Center.     

Disciplinary roots and branches of evaluation: Some lessons from agricultural research

Since its origins in North America in the 1960s, the field of program evaluation has grown considerably, and its concerns have broadened from accountability to program improvement, decision support, and institutional learning. Program evaluation is now commonly practiced in governmental organizations not only in North America but also in many countries of Western Europe and Oceania. Although program evaluation is a relatively new field with many controversies and lively debates, a unifying body of evaluation theory, methods, and standards is gradually emerging. Evaluation has recently been described as a “transdiscipline,” as are statistics and measurement. This article is based largely on my personal experiences working in agricultural research organizations in developing regions. Here, a number of different types of evaluation are carried out, but program evaluation as defined by Patton (1997) and as practiced by social scientists to assess public programs is largely unknown. Distinct branches of agricultural research evaluation can be identified, with disciplinary roots in the natural sciences and in agricultural economics. The most rigorous agricultural research evaluations are economic studies. Systematic internal evaluation is notably lacking. Current pressures to improve performance, transparency, and accountability are creating demands for more systematic evaluation, and many program evaluation concepts and methods would seem to be of value in agricultural research organizations. However, in the current scenario of declining funding for agricultural research, managers are yet to be convinced to expand their evaluation activities and explore unfamiliar paradigms and methods. Moreover, they are not yet convinced that social-science-based program evaluation would produce useful results. Natural scientists and economists tend to view program evaluation as “soft-science” or no science at all. Douglas Horton works at the International Service for National Agricultural Research (ISNAR) in The Netherlands. Since joining ISNAR in early 1990, he has done research, training, and advisory work on agricultural research management, with an emphasis on evaluation. Previously, for fifteen years, Horton was head of the social science department of the International Potato Center (CIP) in Peru. With colleagues at CIP and in national agricultural research organizations, he documented patterns and trends in world potato production and use, engaged in participatory technology development and assessed the impact of CIP programs. Horton received B.S. and M.S. degrees in agricultural economics from the University of Illinois, and a Ph.D. in economics from Cornell University. While at Cornell, Horton worked with W.F. Whyte, pioneer in the fields of participant observation and participatory action research.     

Introducing ACRL Information Literacy Competency Standards to Graduate Schools of Theology

Abstract

This paper recommends that the Information Literacy Competency Standards for Higher Education of the Association of College and Research Libraries should be endorsed by the Association of Theological Schools' accreditation standards and implemented by theological schools to teach graduate students information literacy and to assess their learning outcomes. Several select theological research guides and the experience of a seminary's research methods course are closely examined in detailed comparison to the Information Literacy Competency Standards, their performance indicators and outcomes.     

Planning bridges the intention–behaviour gap: Age makes a difference and strategy use explains why

Objective: This study examines age-differential association patterns between intentions, planning and physical activity in young and middle-aged individuals. The effectiveness of planning to bridge the intention–behaviour gap is assumed to increase with advancing age. We explore the use of behaviour change strategies that include selection, optimisation and compensation (SOC) as underlying mechanisms for age differences. Methods: In N = 265 employees of a national railway company (aged 19–64 years), intentions, planning, SOC strategy use and physical activity were assessed at baseline (Time 1) and again 1 month later (Time 2). Hypotheses were tested in two different path models. Results: Age moderates the extent to which planning mediates the intention–behaviour relation due to an increasing strength of the planning–behaviour link. As a possible psychological mechanism for these age differences, we identified SOC strategy use as a mediator of the age by planning interaction effect on physical activity. Conclusion: These findings suggest differential mechanisms in behaviour regulation in young and middle-aged individuals.     

A Cognitive-Behavioural Approach to the Secondary Prophylaxis of Work-Related Back Pain

Abstract

A clear call has been made for the prevention of the development of chronic back pain. Indeed, chronic back pain is quite prevalent, causes great suffering, and is extremely costly. Although several attempts have been made in clinical settings to prevent the development of chronic pain, there are few reports on the techniques involved. Consequently, this paper aims to summarize some current initiatives in secondary prevention which are relevant for health care professionals. After reviewing some background principles, a guide to prevention is provided. This is based on a four-stage approach that includes screening, assessment, preventive intervention, and recycling of the program. Although research to date suggests that these methods are promising, there is a real need to conduct further research to determine the effectiveness of such programs as well as to develop the techniques further.     

Placebo: Its action and place in health research today

The place for the placebo in human clinical research is addressed in this paper. The World Medical Association which is comprised of some 80 National Medical Associations uses much of its resources to address medical ethics and human rights issues. It adopted the Declaration of Helsinki in June 1964 which addressed the protection of individuals in clinical trials. The use of placebos assumes an important role in this document. Five Revisions of the Declaration of Helsinki have occurred and the most recent was adopted in October 2000. The provisions on placebo are now in Paragraph 29 which reads as follows: “The benefit, risks, burdens and effectiveness of a new method should be tested against those of the best current prophylactic, diagnostic and therapeutic methods. This does not exclude the use of placebo, or no treatment, in studies where no proven prophylactic, diagnostic or therapeutic methods exists.” The reactions to the newly revised version of the Declaration of Helsinki were numerous and rapid, not the least of which was the paragraph I quoted above. At the direction of the WMA Council, a small group of experts together with the WMA workgroup studied the Paragraph 29 to ensure that no ethically sound research was being restricted by the revision. The outcome was approved by the Council and later the WMA General Assembly in October 2000. This gives a note of clarification as to the appropriate use of placebos. Numerous papers and statements over the last several years have described positions very much in line with the Note of Clarification cited above. This paper was presented at an international conference, “Placebo: Its Action and Place in Health Research Today,” held in Warsaw, Poland on 12–13 April, 2003.     

Within-person association of volitional factors and physical activity: Insights from an ecological momentary assessment study

ObjectiveCurrent health behavior models of physical activity (PA) suggest that not all PA intentions are translated into actual PA behavior, resulting in a significant intention-behavior gap (IBG) of almost 50%. These models further suggest that higher self-efficacy and specific planning can aid in decreasing this gap. However, as most evidence stems from between-person (trait level), questionnaire-based research, it is unclear how large short-term IBGs are, how self-efficacy and planning covary within-persons across time and whether they similarly predict smaller IBGs. It is likely that day-to-day changes in circumstances and barriers affect these variables thus the applicability of theoretical models is uncertain. Here, within-person prospective analyses of ecological momentary assessment (EMA) data can provide insights.Methods35 healthy participants (aged 23–67) completed four EMA-based questionnaires every day for three weeks. Each prompt assessed PA (retrospectively, “since the last EMA prompt”); PA intentions, planning specificity, self-efficacy, and intrinsic motivation (prospectively, “until the next EMA prompt”) and momentary affect. Generalized logistic mixed-effect modeling was used to test predictors of PA.ResultsAcross the 2341 answered EMA prompts, PA intentions were not enacted in 25% of the episodes (IBG). In episodes with given intentions, PA likelihood increased with higher levels of self-efficacy, planning specificity, and intrinsic motivation. The latter two also positively predicted PA duration and intensity.ConclusionsShort-term intention behavior gaps seem to be smaller than what is known from more long-term studies, most likely as individuals can anticipate the actual circumstances of PA. Further, current health behavior models show validity in explaining within-person dynamics in IBGs across time. Knowing the relevance of planning specificity, self-efficacy and intrinsic motivation for day-to-day variations in PA enactment can inform respective real-time mHealth interventions for facilitating PA.     

A Duality for the Algebras of a Łukasiewicz <Emphasis Type="Italic">n</Emphasis> + 1-valued Modal System

In this paper, we develop a duality for the varieties of a Łukasiewicz n + 1-valued modal system. This duality is an extension of Stone duality for modal algebras. Some logical consequences (such as completeness results, correspondence theory...) are then derived and we propose some ideas for future research. Presented by Daniele Mundici     

Psychosocial and Behavioural Assessment of Chronic Pain: Recommendations for Clinicians and Researchers

Abstract

A variety of methods have been developed for the assessment of chronic benign pain. In research or in clinical practise, choosing an adequate measurement technique can be a laborious task. This article has two main purposes: to report on some basic features to consider when selecting assessment instruments, and to present a review of some psychosocial and behavioural measurement methods for the assessment of chronic benign pain. The selected methods have, with one exception, been evaluated on Swedish chronic pain populations and are used in international pain research. In conclusion, we recommend the use of psychometrically sound instruments, and that the purposes for use of a measure have to be thoroughly considered in advance. We also emphasize that in clinical practise, each separate measure must be interpreted in a wider context, where clinical findings and judgements are considered as a whole. In addition, none of the single reviewed assessment techniques can replace the communication between the patient and the clinician.