Evaluating research institutions: Lessons from the CGIAR

Investing in research is a long-term, risky proposition. In agriculture, it could take fifteen years or more for a research finding to show an improvement in a farmer’s field. Yet, research institutions, like other organizations it needs to be evaluated. For more than twenty years, independent panels of outside experts have evaluated each of the international research centers that the Consultative Group of International Agricultural Research (CGIAR) supports. This paper examines the evolution of this review system, outlines the key methodological challenges faced, and draws lessons for others engaged in evaluating research institutions. It notes that the scope of the CGIAR reviews have been broadened over time in response to users’ concerns. Reviews now cover four dimensions of performance: research results, quality and relevance of research, vision and strategic directions, and management efficiency. The methodological challenges faced in measurement, valuation, and attribution are reviewed, along with practices found to be helpful in addressing these concerns. The paper concludes that the panel approach to institutional evaluation has served CGIAR’s needs well, and recommends it as an evaluation technique for integrating quantitative and qualitative dimensions of institutional performance. His recent work has concentrated on the governance and management of the CGIAR and the individual research centers it supports, including questions of evaluating management effectiveness. Prior to joining the World Bank in 1979, he was Assistant Professor of Political Science at Bogazici University (Istanbul) and Senior Research Director of Systems Research Incorporated (Lansing, Michigan).     

The experience of ‘healing stories’ in the life narrative: A grounded theory

‘healing stories' are part of an emergent narrative literature in psychology and counselling. This research explored the subjective experience of ‘healing stories’ within the context of individual's life narratives. A grounded theory (Strauss and Corbin, 1990) method of analysis was used to generate and analyse six participant's life narratives. Analysis of these accounts revealed three categories. The first is the context of struggle that preceded the healing process. participants told of struggle in their relationships with others as well as turmoil in their experience of themselves. The second category, the healing process reflected the participants' growing awareness of life's existential givens (Yalom, 1980). This represented a personal exploration that opened lparticipants to a ‘healing story’ that offered them an insight into choices and possibilities for liberation. The third category illustrated that in the ongoing struggle the ‘healing story’ has continued to provide inspiration in the participants' lived experience as an ‘evoked companion’ (Stern, 1985). This research posited a core dynamic whereby participants' experience of their ‘healing story’ provided not only an initial inspiration that was healing but also, in their ongoing struggle, the ‘healing story’ continued to inspire their lived experience.     

Knowledge level learning and the power law: A soar model of skill acquisition in scheduling

Zusammenfassung. Das hier vorgestellte Modell lernt graduell, Planungsaufgaben aus der Klasse der Maschinenbelegungsprobleme (job-shop-scheduling problems) zu lösen. Mit Hilfe des Chunking-Mechanismus von Soar wird episodisches Wissen über die Belegungsreihenfolge von Aufträgen auf Maschinen memoriert. Bei der Entwicklung des Modells wurden zahlreiche qualitative (z. B. Transfereffekte) und quantitative Befunde (z. B. Bearbeitungszeiten) aus einer früheren empirischen Untersuchung berücksichtigt. In einer Validierungsstudie wurden dieselben Aufgaben von 14 Probanden und dem Modell bearbeitet. Die Passung von Simulationsdaten und empirischen Ergebnissen fiel insgesamt gut aus. Allerdings löst das Modell die Aufgaben schneller und zeigt auch einen etwas besseren Lernverlauf als die Probanden. Das Modell liefert eine Erklärung für das Rauschen, das typischerweise bei Bearbeitungszeiten zu beobachten ist: es handelt sich um erworbenes Wissen, das mehr oder weniger gut und auch unterschiedlich häufig auf neue Situationen übertragen wird. Der Lernverlauf der Probanden entspricht nur für aggregierte Daten einer Potenzfunktion (power law). Der vorgestellte Mechanismus zeigt, wie ein symbolisches Modell der Informationsverarbeitung graduelle Verhaltensänderungen generiert und wie der offensichtliche Erwerb allgemeiner Prozeduren ohne explizites Lernen von deklarativen Regeln erfolgen kann. Es wird nahegelegt, daß es sich hier um die Modellierung einer Form impliziten Lernens handelt. Summary. The model presented here gradually learns how to perform a job-shop scheduling task. It uses Soar's chunking mechanism to acquire episodic memories about the order to schedule jobs. The model was based on many qualitative (e.g., transfer effects) and quantitative (e.g., solution time) regularities found in previously collected data. The model was tested with new data where scheduling tasks were given to the model and to 14 subjects. The model generally fit these data with the restrictions that the model performs the task (in simulated time) faster than the subjects, and its performance improves somewhat more quickly than the subjects' performance. The model provides an explanation of the noise typically found in problem solving times - it is the result of learning actual pieces of knowledge that transfer more or less to new situations but rarely by an average amount. Only when the data are averaged (i.e., over subjects) does the smooth power law appear. This mechanism demonstrates how symbolic models can exhibit a gradual change in behavior and how the apparent acquisition of general procedures can be performed without resorting to explicit declarative rule generation. We suggest that this may represent a type of implicit learning.     

The puzzle of language evolution

Linguistics must again concentrate on the evolutionary nature of language, so that language models are more realistic with respect to human natural languages and have a greater explanatory force. Multi-agent systems are proposed as a possible route to develop such evolutionary models and an example is given of a concrete experiment in the origins and evolution of word-meaning based on a multi-agent approach.     

Reinventing the University Through the Teaching of Diversity

University reform encompasses an increased emphasis on the social and emotional development of students. In particular, the development of caring, empathy, and community are emphasized. This team of psychology professors approaches the task of university reform by presenting both its philosophy of diversity inclusion and specific inclusion strategies as means of promoting enhanced development of our students. Although the discipline of psychology is used as an example, the philosophical argument and the strategic approaches may be applied to other disciplines.     

Age differences in accuracy and choosing in eyewitness identification and face recognition

Studies of aging and face recognition show age-related increases in false recognitions of new faces. To explore implications of this false alarm effect, we had young and senior adults perform (1) three eye-witness identification tasks, using both target present and target absent lineups, and (2) and old/new recognition task in which a study list of faces was followed by a test including old and new faces, along with conjunctions of old faces. Compared with the young, seniors had lower accuracy and higher choosing rates on the lineups, and they also falsely recognized more new faces on the recognition test. However, after screening for perceptual processing deficits, there was no age difference in false recognition of conjunctions, or in discriminating old faces from conjunctions. We conclude that the false alarm effect generalizes to lineup identification, but does not extend to conjunction faces. The findings are consistent with age-related deficits in recollection of context and relative age invariance in perceptual integrative processes underlying the experience of familiarity.     

Mathematical engineering and mathematical change 1

In this paper, I introduce and examine the notion of “mathematical engineering” and its impact on mathematical change. Mathematical engineering is an important part of contemporary mathematics and it roughly consists of the “construction” and development of various machines, probes and instruments used in numerous mathematical fields. As an example of such constructions, I briefly present the basic steps and properties of homology theory. I then try to show that this aspect of contemporary mathematics has important consequences on our conception of mathematical knowledge, in particular mathematical growth.