The combined use of the participatory dialogue method and survey methodology to evaluate development projects: A case study of a rural development project in Bangladesh

This is a case study that shows how the qualitative and participatory dialogue method was combined with survey methodology to evaluate the impact of a Canadian International Development Agency (CIDA) supported bilateral and credit-based rural development project (RD-12) for the landless poor in Bangladesh. The article argues that although the necessity and importance of statistical and quantitative evaluation is not denied, qualitative participatory evaluation has a complementary role to play in producing a body of unique and illuminative data on qualitative developmental impacts. The findings on the social and economic impacts of the project obtained using the dialogue method were integrated with the quantitative data obtained from the survey. The findings show how the qualitative details and insights enriched the bare-bone statistical information. A more comprehensive body of knowledge on the psychosocial changes among the beneficiaries and the development process at large was elicited from the survey questionnaire. More importantly, the developmental impacts were described and viewed from the beneficiaries’ own perceptions and perspectives and in their own words. The article lists a number of lessons that were learned while using the dialogue method to evaluate this project.     

School-Linked Services in Context

Summary

This paper describes a formative evaluation of a school-linked service delivery system for immigrant and low income families in a large suburban county in Maryland. Initially we discuss local needs that prompted the development of the program and we place problems with services for children and families with multiple needs in a broad context. Next we discuss the methods and results of the evaluation. Findings indicate the program provides a range of appropriate services to intended beneficiaries but that lack of infrastructure and articulated understandings among participating organizations present potential problems.     

"It's not my fault--but only I can change it": counterfactual and prefactual thoughts of managers

In testing for the self-serving bias in performance evaluation, the authors propose that comparing managers' counterfactual and prefactual thoughts about subordinates' performance is more conclusive than the attributional approach and also offers practical advantages. In a study with 120 managers, a 4-way interaction of subordinate performance, temporal perspective, direction, and reference confirmed the predicted pattern. Managers' thoughts about how a weak performance could have been enhanced had external references, but thoughts about how such a performance could be enhanced in the future focused on the leader. This asymmetry was only observed for weak performance. Results are discussed with regard to biases in leaders' performance evaluations and to how counter- and prefactual thoughts could be used for leadership research and practice.     

Surfacing embedded assumptions: Using repertory grid methodology to facilitate organizational change

This article outlines a 3-year research project that examines the role of performance evaluation and control systems in bringing about change in manufacturing organizations. A key aim of the project was to develop tools and techniques that managers can use to enable them to understand and enact change. As part of an indepth case study methodology, repertory grid interviews were used with both managers and operatives in three companies as a means of identifying commonly held assumptions about a range of work behaviours. The article discusses the use of the grid as a developmental, rather than exploratory technique, thereby focusing on its use as a process through which managers can assess their systems in use and highlight any necessary change issues. It is argued that repertory grid techniques have a useful part to play in diagnosing the key change issues that work and organizational psychologists face when planning organizational change and development interventions.     

Building capacity in planning,monitoring and evaluation: Lessons from the field

This paper reports on the author’s experiences as manager of a capacity-building project in Latin America. The project aimed to strengthen planning, monitoring, and evaluation (PM&E) in agricultural research. Nine lessons are drawn: (1) Project design is much more than a technical process; it is essentially one of negotiation. (2) In capacity-building projects, design activities cannot end when implementation begins. (3) Capacity-building efforts should prepare managers to deal with complexity, uncertainty and change. (4) In capacity-building efforts, it is essential to collaborate rather than patronize. (5) Organizational assessment is a complex social process, intertwined with organizational politics. (6) In designing capacity-building projects, it is essential to involve managers and staff members in assessing needs and opportunities. (7) Action-learning strategies offer great potential for capacity building. (8) In the context of strategic management and organizational learning, PM&E take on new meanings. (9) Training is most effective when it is designed to serve a purpose within an organizational change process. It is concluded that capacity building is more a process of social experimentation than of social engineering. Management systems cannot be imported, but need to be developed within organizations. Development agencies should play catalytic, facilitating roles, rather than take responsibility for organizational change. To support genuine capacity development, donors and funding agencies need to ensure that their planning and accountability procedures foster flexibility, innovation, and learning. Since joining ISNAR in 1990, he has engaged in research, training, and advisory work on research management, with an emphasis on evaluation. Previously, for fifteen years Horton was head of the social science department of the International Potato Center in Peru. Horton received B.S. and M.S. degrees in agricultural economics from the University of Illinois and a Ph.D. in economics from Cornell University. His current interests include action research and learning, organizational assessment, and institutionalization of planning, monitoring, and evaluation.     

Socializing Retail Employees in Ethical Values: The Effectiveness of the Formal versus Informal Methods

The purpose of this study was to test the effectiveness of an informal socialization process for reported employee attitudes towards unethical behavior within a large retail organization. This study compares the reported ethical attitudes of full-time employees (N = 203) who were socialized using an informal approach, to the reported ethical attitudes of their managers (N = 274) who were socialized using a formal approach. The results indicate that the informal approach to ethical socialization was not as effective as the formal approach to ethical socialization. Employees were more accepting of unethical behavior than were their managers. Implications are discussed and recommendations for ethical socialization techniques are provided.     

Faith-based development of World Vision Tanzania

ABSTRACT

This article examines how faith is intertwined in conceptions of development in the work of World Vision Tanzania, a faith-based organisation focusing on child welfare. Issues that are explored include the question how an apparently common faith setting constitutes and provides a source of social meanings and values for the assessment of human development as well as secular development concepts such as human agency and empowerment. The analysis shows how World Vision staff assign different values to Christian and non-Christian development when assessing the potential and achievements of the beneficiaries in terms of secular concepts. It is argued that a faith-based worldview in this context provides the foundation for the specific set of values and social meanings, which are examined in this article, and a source for these more generally when value judgements and assessments of agency, empowerment, freedom, and, ultimately, well-being are made. It is further suggested that, by employing ideas of freedom, agency, and empowerment and investing them with Christian meaning, World Vision is able to generate a portrait of itself and its beneficiaries that is positioned in a hierarchy of values where Christian ideals and secular development fuse and become paramount.     

An assessment of manager/subordinate perceptions of performance appraisal effectiveness

This study postulates that performance appraisal will be effective to the extent that managers and subordinates have a shared perception of its purpose and function and the degree to which it meets the needs of both groups. A two part research project was conducted to: 1) identify manager/subordinate perceptions of the purposes served by the formal performance appraisal process; 2) determine the extent to which these purposes are being satisfied in organizational practice; and 3) assess if managers and subordinates have a shared perception of the effectiveness of the appraisal process. The sample for the study included 36 managers and 143 subordinates from a division of a large Midwest service organization. The results were mixed with regard to the appraisal system effectively accomplishing its diverse goals. However, in general, there was support for the notion that both managers and subordinates find the appraisal process to be a worthwhile organizational practice. Approximately 70% of the respondents indicated that they would participate in the appraisal program whether they were required to do so or not.     

The Truth of the Matter

Feminist standpoint theory (FST) has a troubled history that has limited its use and development as a core feminist epistemological project. This article revisits debates from its past, and re‐examines an apparent central problem: that of the realism identifiable in FST. Looking closely at the criticism leveled against one particular standpoint theorist—Nancy Hartsock—I show the criticism not only to be unfounded, as has previously been argued, but also unnecessary. I demonstrate that the accusations of supposedly realist contradictions in Hartsock's work are easily resolvable by engagement with critical realism (CR). I argue that CR not only accommodates Hartsock's conception of realism, and so dissolves any contention, but that CR complements and shores up FST's central claim: that situated knowledge carries with it an epistemic privilege. Another contemporary conception of realism is being developed—New Materialism (NM)—that, it could be argued, would also be a suitable ontology with which to develop FST. I show how NM could present problems for FST as a fundamentally political project, and conclude that CR offers a more fruitful future collaboration for FST.     

Psychologie communautaire et recherche : l’exemple du projet EMILIA

The development of community psychology has also had significant implications in the area of evaluation and research methodology. The empowerment principle, that is, giving users the means to control their health and to make their own health decisions, has important implications for the choice of evaluation strategies for both individual therapeutic acts as well as for public health programmes. If the user's potential expertise in the health area is to be fully acknowledged, how is this to be done with regard to research? How can users become not only actors of their own health, but also play an active role in research and health evaluation? The article gives an overview of research strategies based on community psychology principles, with a particular emphasis on action research and participative-action research. The example of a multisite European project on empowering mental-health service users to access life-long learning and employment, currently underway in Paris, is presented to illustrate these community research methods.     

Toward a psychology of positive youth development

This article analyzes the development of initiative as an exemplar of one of many learning experiences that should be studied as part of positive youth development. The capacity for initiative is essential for adults in our society and will become more important in the 21st century, yet adolescents have few opportunities to learn it. Their typical experiences during schoolwork and unstructured leisure do not reflect conditions for learning initiative. The context best suited to the development of initiative appears to be that of structured voluntary activities, such as sports, arts, and participation in organizations, in which youths experience the rare combination of intrinsic motivation in combination with deep attention. An incomplete body of outcome research suggests that such activities are associated with positive development, but the developmental processes involved are only beginning to be understood. One promising approach has recorded language use and has found that adolescents participating in effective organizations acquire a new operating language that appears to correspond to the development of initiative.     

Developing clinical effectiveness in psychotherapy training: Action research

This study is an evaluation of a new approach to clinical training in Transactional Analysis using the methodology of action research. The evaluation focuses on the second year of training and placement‐based clinical practice at the Metanoia Institute, UK. The design of the training year was research‐based and used the concept of an ‘internship’ year. The design aimed to close the gaps between clinical practice, supervision and formal training; facilitate translation of theory into practice; and integrate findings from the ‘common factors’ research into effectiveness of psychotherapy. Training and supervision methods were linked and focused on emerging issues in clinical practice and the development of students’ capacity for critical reflection and a research attitude to practice. The evaluation was a naturalistic study which used quantitative and qualitative methodology within the action research framework to reflect on the impact of the training on the participants, the organisation and the clinical practice. The quantitative outcomes show that the project developed the effectiveness of students’ clinical practice and reflexivity. Qualitative analysis gives insight into the experiences of the participants and the emotional impact of the research process.     

The Effect of Professional Partnership on the Development of a Mutual-help Organization

The effects of partnership between Schizophrenics Anonymous (SA, a mutual-help organization) and the Mental Health Association in Michigan (MHAM, a professionally staffed advocacy organization) on SA's growth and development were explored. Following the initiation of a formal partnership, SA groups were more available throughout the state, more likely to be associated with formal mental health settings, and less likely to have leaders who had been participants in other SA groups. Groups with consumer leaders had significantly greater longevity than groups with professional leaders. Changes in the organizational structure and process of SA were also identified. SA leaders reported that SA moved from a collective to a more bureaucratic structure. As a result, there was greater consistency, administrative capacity, and response capacity. This enhanced capacity came with costs reported by SA leaders. The leadership role of SA members became less defined. SA members expressed concerns about the more hierarchical structure of SA's organization, decreased consumer control, increased professional involvement in SA, and an excessive focus on group development as opposed to group maintenance. Mental Health Association in Michigan staff reported that MHAM was also impacted by the partnership, both with regard to internal functioning and external perception. Implications for effective partnerships between mutual-help and professional organizations are discussed.     

Does method of acquisition affect the quality of expert judgment? A comparison of education with on‐the‐job learning

Expertise can be acquired through experience or through education, and many occupations demand a mixture of both. In this paper we evaluate the level of expertise acquired through these two routes by comparing credit decisions made by professional credit managers (who have learned through experience rather than education) and lecturers and students in management, (who have no experience but have learned the relevant concepts). We also tested a control group of laypeople with neither relevant formal education nor experience. Participants assumed the role of credit manager and made creditworthiness predictions and credit granting decisions for six companies with known performance. Lecturers and students generally outperformed credit managers on the prediction tasks, and the credit managers performed only slightly better than laypeople. In short, for at least one domain, it appears to be more efficient to learn from formal education than from experience. We argue that the prerequisites for successful learning are largely absent from the everyday experience of credit managers, but are present in an educational context.     

Owning evaluation: Revaluing clinical work in a university context

This paper draws on the experience of a year-long evaluation project in a university counselling service. The author considers some reasons for practitioners' reluctance to undertake formal evaluation, but suggests their confidence in clinical work may be enhanced if ambivalence can be explored. The paper discusses the impact of organizational culture on such ventures and considers what conditions might promote their success.     

Has the cataphasic form of schizophrenia increased in modern times?

According to our investigations the incidence of cataphasia, which is a subtype of schizophrenia, has become more common in the last years and on average, has started earlier in life. As the central disorder of this illness concerns thinking perhaps there exists a relation of our finding to the rapid modern development of techniques, insofar as the cognitive function is overstrained in childhood and adolescence, when the nervous system has not matured yet. Introverted individuals seem to be at a particular risk. However, it must not be disregarded that cataphasia is an inherited illness which cannot be caused by external influences, but can only be promoted in its manifestation.     

Evaluation of agricultural research in Latin America and the Caribbean

This article assesses the state of evaluation, and identifies priorities for improving evaluation, in agricultural research organizations in the region of Latin America and the Caribbean in the early 1990s. Based on thirteen case studies conducted in 1992, the article describes the institutional settings and regional patterns, and trends in evaluation practice. Illustrative cases from Argentina, Brazil, and Guatemala are presented. The organizations studied have extensive experience with evaluation; however, this experience has not been well documented or shared. Evaluation is generally the weakest phase in the management cycle. In the past, most evaluations have been extrinsically motivated, and as such, they have been of little use to local researchers and managers. Obstacles to improving evaluation include the centralization of administrative systems, weak program management, a lack of understanding of potential uses of evaluation in management, and limited knowledge of appropriate evaluation methods. Agricultural research managers feel that evaluation training should be provided as one component of a broader effort covering planning, monitoring, and evaluation. Since joining ISNAR in 1990, he has engaged in research, training, and advisory work on research management, with an emphasis on evaluation. Previously, for fifteen years Horton was head of the social science department of the International Potato Center in Peru. Horton received B.s. and M.S. degrees in agricultural economics from the University of Illinois and a Ph.D. in economics from Cornell University. His current interests include action research and learning, organizational assessment, and institutionalization of planning, monitoring, and evaluation. He worked for fourteen years at the Brazilian Corporation for Agricultural Research, conducting research and development activities in the areas of human resources and of strategic management. He is a full professor at the Department of Social and Work Psychology, Institute of Psychology, University of Brasilia, Brazil, where he teaches and advises at the undergraduate and graduate levels and does research and consulting. His current areas of interest are organizational behavior, training, organizational evaluation, and science and technology management.     

Facilitating Local Implementation of Special Education Program Evaluation

The results of a project assessing the applicability of four program evaluation models is described. Survey data from districts documented an interest in local-level special education evaluation models. A training institute for local staff was therefore established to provide staff with competency in four evaluation models. Subsequently, the models were replicated in eight local sites from which representatives had expressed an interest in using the models. All sites completed evaluation planning and implementation activities. Findings suggest that each of the four models in the project can be used with minimal levels of outside assistance and can be readily adapted to local needs. However, each model places somewhat different demands on local staff; and some models may be more appropriate for particular evaluation purposes.     

Projection, transitional phenomena, and the Rorschach

This article sheds light on the theoretical foundations of projective testing. Drawing on the psychoanalytic perspective, especially Winnicott's works, it elucidates the nature of projection and transitional relatedness, with particular reference to the Rorschach. This conceptualization of the Rorschach task is compared with Exner's more cognitive-behavioral orientation, which holds that the Rorschach is not a projective test. I suggest that Exner's Comprehensive System could become more truly comprehensive if it were integrated with a more complex understanding of the response process, allowing more importance to the role of stimuli from the inner world to counterbalance its focus on the processing of external stimulation. Clinical and research implications are discussed.     

Endorsement of Group Change and Organization Change Affirmative Action Programs for Male and Female Beneficiaries

To test the position that some people’s willingness to endorse affirmative action is influenced by the match between beneficiary group and the attributional underpinnings of the programs, endorsement of four programs that could be used to benefit either women or men were assessed among 95 undergraduate students in Ontario, Canada. Students preferred the use of a remedial program that suggested that beneficiaries are under-qualified and need to develop job relevant skills for female versus male beneficiaries. This was driven by perceptions of the effectiveness of the program. For male beneficiaries, participants were more supportive of a mentorship development program that implied the necessity of structural change than they were of the remedial program. These effects were independent of participant gender.