Why is Critical Realism and Realistic Evaluation Important in Public Health Programs for Prevention of Obesity?

Opinion

Public health is a field that is occupied with the notion of health and how health can be preserved, protected and promoted for populations. One way to do this is with wide interventions that aim to promote public health, aiming at certain aspects of life such as weight gain, physical activity, mental health, substance use etc.

Public health is a field that encompasses cooperation from various disciplines and perspectives. Public health is ‘the science and art of preventing disease, prolonging life, and promoting health through organized efforts of the society’ as defined by the Acheson Report in 1988 [1]. What is still missing is the ‘art’ component when we are referring to how we should evaluate an effort or an intervention that aims to improve public health.

Philosophy can possibly provide the answer to such efforts. At first, we might be intrigued to think that philosophy can be too abstract to inform everyday practice in public health in the complexity of today’s reality. Nothing is further from the truth than the previous statement.

The reasoning and causality in interventions that target social groups and communities is a complex field that is not straightforward to analyses. What element produced which effect cannot be answered by monolithically trying to capture it in one metric.

Established methods of evaluation provide a static view of the outcomes as a measure of how ‘effective’ interventions Opinion are. Rarely is evaluation focused on the mechanisms that operate in an intervention. Under such mechanisms, one should understand the factors which enable the changing effect of a program towards the desired outcome within an environment. In other words, with established methods of evaluation, researchers measure the outcome that a program/intervention produced, and regard this as the result of the whole interplay of components that this intervention introduced in any social environment. An intervention/ program does not operate within a social environment like a black box. Black box logic implies all the input that is fired by an intervention in an environment with the wish to produce a certain output. In practice, what is taken back is the fermentation of the intervention resources with the reasoning of the actors, all boiled down in the ‘stove’ of each context.

However, public health research evaluation very often attempts to evaluate programs, by employing black box logic: an outcome of an intervention is considered the difference in a chosen measurable variable, such as an increase in physical activity levels, weight loss, increase of awareness, etc. Usually when an intervention fails to showcase an incremental difference, it is deemed in the literature as not successful. This evaluation approach ignores the fact that other effects might have been produced that cannot be measured or that an effect might have been produced in what we wish to achieve with the intervention but it is not sufficiently captured in the chosen marker. The good news is that there is a type of evaluation that can deliver the promise of an appropriate approach to evaluating complex interventions in the community or in wider social systems. Realistic evaluation has the potential, when properly done, to enrich our knowledge base and our understanding on what works, in which circumstances and for whom [2].

Realistic evaluation has been applied in various studies and in the last ten years, its application grew substantially in various social, medical or public health programs. It is necessary to provide a short description of what realistic evaluation is and where it is based. Realistic evaluation draws its origins initially from the philosophical postulate of critical realism as described in the work of the British philosopher Roy Bhaskar (1944-2014). Realistic evaluation is the product of the work of Pawson and Tilley and has found applications in various disciplines [2]. Realistic evaluation represents the connection of philosophy and social practice, theory and applied research. Realistic evaluation aims to function as the continuation along the road of driving realism into research practice. Pawson and Tilley, provide to the researchers an operationalization formula which considers configurations in the form of context, mechanisms and outcomes (CMO) [2].

The aim of this commentary is to reflect on the nature of knowledge gained through the process of construction of context-mechanism-outcome (CMO) configurations. The realistic evaluation approach was applied on three public health programs for obesity prevention. These programs were chosen as case studies after a well-defined selection procedure and took place in England, Scotland and Germany [3]. The process of applying a realistic evaluation approach enabled the researcher of the interventions to create theories about the interplay among the mechanisms fired by an intervention, as well as the reasoning attached to those mechanisms by people involved in those interventions.

Reflecting on this process of constructing context- mechanism-outcome (CMO) configurations, the researcher possesses the ability to construct configurations through the data, micro-theories that explain the reality and the conditions that produced this reality. What came afterwards was the realization that the philosophy of critical realism allows the researcher to reflect further on this realistic evidence and link it to overarching themes which are associated with social reality. These overarching themes are the concepts of human agency and social structure and help the researcher to form a deeper understanding of the reasoning, the motivations and the intentions of people as agents in response to the ever- changing structures within society who either constrain or enable those capacities [3].

In the study of Kolovou [4], three interventions were analyzed in England, Scotland and Germany, respectively against their background and the existing context where they were embedded. All three contexts proved to possess a variety of micro-nutrients that were fertile components for the initiation of the interventions in the ground where they were embedded. The interventions ideals were probably customized to the context in which were embedded, or it could be said that the context gave birth to the ideals of the interventions. Those elements represented organizational structures that were exceptionally functional to accommodate the initiation of a project, social understanding and tradition, which was particularly hospitable and reasonable for the philosophy of the project, with an advanced and sophisticated understanding and background in public health. Thus, the structural conditions that promoted the embedding and the rooting of the intervention into the existing context included the micro level of the organization model, the operational rules, the hierarchy and the role allocation within the project. At the meson level, structures that enabled the introduction of the interventions included the local community, the direct exchange of the project with other services and authorities, other community actors and stakeholders and the local understanding of the topic introduced by the project. At the macro level, there were structural topics such as the political agenda of the era that could either promote and/or offer political windows for the initiation of a project. Often, there were situations in which a lack of political support or systemic downplaying of the targets of the project would make it a second priority and would locate the political interest elsewhere, and thus the corresponding funding as well. The cultural tradition of some contexts, the historical circumstances, or the background of other relevant endeavors that had rooted the soil with the acceptance of social integration projects or walking schemes [3] were also regarded as structural conditions, which promoted the initiation of the interventions. The structural conditions of each intervention however were not always fruitful and promoting. In many circumstances the structural conditions hindered and delayed the implementation of the project. There were conditions where the agents of the interventions encountered structural ‘walls’ and had to find alternative structural openings or postponed them for other time conditions, which were more favorable. It was identified that during many occasions, the organizational teams in all three interventions encountered situations, where the micro-structures of their organizational structures had to be reconciled or negotiated with the meso- or macro-structural conditions in order to produce a satisfactory circumstance for all stakeholders. Those were situations for which solutions had to be produced in response to challenging circumstances, to create the so-called ‘win-win’ arrangements for all actors involved. Those arrangements were necessary for the advancement of the projects. This should not be interpreted as bending the theory of an intervention or negotiating its targets but as necessary arrangements which favor the initiation of a new endeavor, such as avoiding sailing on the day of the turbulence and sailing on the day of the light winds. In particular, such arrangements represented necessary negotiations that allowed space and time so that the involved agents could realize what they could win from the new conditions that the intervention produced. It should be noted that the element of time is crucial. Patience, allowing time and flexibly going away and coming back at a better moment, was vital to the success of the interventions.

Such structural factors remain neutral in the absence of the human agency. Structural factors, in other words, represent ‘windows of opportunities’ or a ‘structural wall’ in the face of a human agent who represents a driving force initiating any form of change. It is the reasoning of people which can activate the causal powers of the structural conditions. A window of opportunity will never be realized as an ‘opportunity’ without the action of the agent but it will remain inactivated. Two case studies were interventions which were initiated and would have come into existence due to the driving force of some pioneer personalities. Before they become structural entities, they were the vision of one person. The third project was not initiated through the driving force of one sole person but the existence of local managers that were determined to make the project work within communities using a pathfinder approach. Being free to find alone what would work and what would better serve the needs of their well-defined community was demonstrated among the mechanisms that created CMOs with an enabling effect on the project and it demonstrates how human agency is increased in this approach as well.

Meaning and reasoning were the result of human agency and how human spark could motivate the dynamics of the interventions in order to increase the anticipated outcomes. One clear finding which was independently identified in all three interventions was the activation of non-health related motivations. Simply put, interventions did not work based on their capacity to attract people who wanted to do something for their health. The interventions worked because they managed to stimulate other reasoning for people to participate or perhaps managed to match the branding of the intervention’s delivery mode to the right motivations that people needed to participate. Participants in the setting of walking schemes did not walk to lose weight but were participating to find peers and get through loneliness and loss of beloved people. The target group in the German setting was motivated by their empowerment logic of the intervention which made participants feel that their needs were taken seriously, rather than increasing their aerobic capacity by gymnastics classes. Finally, the project in Scottish communities managed to reach its target group by avoiding stigmatizing language, putting forward positive and holistic messaging that was not weight-related and organizing community activities that were vibrant and community binding. In the study of Kolovou [4], all organizers within the three case studies were asked in the form of extensive qualitative interviews to recall, synthesize, reproduce and reflect on their experience and their role in the intervention. Pawson and Tilley advocate - explicitly as a vital element to include in the realistic investigation - the ‘folk wisdom of practitioners’ ,(Pawson and Tilley, [2] (page 107). Thus, the hypotheses formed, which are the configurations of mechanisms, contexts and outcomes, are sets of scenarios or theories of actions which are not vague or abstract. They entail insightful reflections and handling descriptions and these investigations capture a huge amount of institutional memory, an accumulation of hours of experience in the field and the intuitive capacity that the practitioners possess. This insight is incorporated in the CMO configurations and can be tested with further rounds of interviews with other practitioners till saturation or refinement of the CMO configurations. Bringing together the folk wisdom of many practitioners in each case study formed the basis to illustrate the type of reasoning the human actors possessed and how they put it in practice. This allowed to construct theories of what kind of reasoning is required eventually to make a project work the way it did, and in response to what contextual conditions. An illustrative example was the theory of the ‘determination to make a difference’ or the theory of ‘belief in the philosophy of the project’.

This is the attractiveness and the potential of critical realism as philosophy of social reality and its transfer to applied practice and evaluation of social programs to generate ’real’ policy evidence. The ideas developed do not represent generalizations of the ideas that each intervention introduced in each context. The lessons learned and the capacity of realistic evaluation to inform policy making is not by constructing and showcasing typical examples of contexts or populations. There is no such thing as typical population or typical context because each are unique, equally and their interaction with other stakeholders or within historical conditions and eras. So, when an investigator proceeds to the analysis of one project to the other, the aim is not to describe similarities but to test ideas developed elsewhere and to see if they work or what are the characteristics of people which do not allow the activation of the particular ideas. Pawson and Tilley say that ’’what are transferable between cases are not lumps of data but sets of ideas. The process works through the development of a body of theory which provides an organizing framework which ’’abstracts’’ from a program a set of essential conditions which make sense of one case after another’’ [2] (page 120). The diversity of the included projects in their underpinning philosophy, the target population and their geographical location is indicative of the diversity in social life. Finding common schemes, or typical examples represents a rather Sisyphean task. Reality is complex, dynamic and diverse and isolation of certain variables is impossible. However, explaining and articulating what works, for whom and why open the door to an exciting possibility: to develop and test ideas and theories that will help us to bring more realistic versions of change. The increase of applications of realistic evaluation in the field of public health already showcase the potential of this approach and its suitability for the area of public health.

Literature

1. Britain G (1988) Public Health in England: The Report of the Committee of Inquiry into the Future Development of the Public Health Function.

2. Pawson R, Tilley N (1997) Realistic Evaluation pp: 256.

3. Bhaskar R (1998) The possibility of naturalism: a philosophical critique of the contemporary human sciences. 4th (Edn.).

4. Kolovou V (2015) A critical realism approach to public health interventions that aim to prevent obesity in selected European countries.