Provocative Idea:
Rethinking Critically Reflective Research Practice:
Beyond Popper’s Critical Rationalism
We all know that ships are safest in the harbor; but alas, that is not what ships are built for. They are destined to leave the harbor and to confront the challenges that are waiting beyond the harbor mole. A similar challenge confronts the practice of research. Research at work cannot play it safe and stay in whatever theoretical and methodological harbors in which it may have found shelter in the past. Still less can it examine and maintain its foundations in the dry dock. Research is more like a ship that must be repaired on the open sea. Yet foundationalist ideas persist in the practice of research. Counter to what is often assumed, today’s dominating model for research--the fallibilist model of critical rationalism--has not really overcome the empirical foundationalism of earlier, positivist research practices. This paper analyses two major foundationalist traps that are currently in the upswing and work against reflective research practice.
Keywords: critical rationalism; science theory; empirical foundationalism; primacy of theory; methodology choice; reflective practice
Suggested Citation: Ulrich, W. (2006). Rethinking critically reflective research practice: Beyond Popper’s critical rationalism. Journal of Research Practice, 2(2), Article P1. Retrieved [date of access], from http://jrp.icaap.org/index.php/jrp/article/view/64/63
The ship metaphor was introduced into science theory by the Austrian philosopher and sociologist Otto Neurath (1932/1959). It remains among the most insightful metaphors for non-foundationalist research thinking today. Neurath used it to argue his case against empirical foundationalism, a view on the conditions of valid knowledge held by some of the logical positivists (also called logical empiricists or neo-positivists) of the Vienna Circle (a school of empiricist science theory of which he was an active member), among them particularly Rudolf Carnap (1932a/1934, 1932b/1987).
Empirical foundationalism holds that systematic observation as regulated by the methods of experimental science provides not only the starting point for constructing meaningful theoretical hypotheses but also a secure foundation for justifying their truth, and thus for claiming “objective knowledge.” Truth, it is assumed, consists in the agreement (“correspondence”) between the phenomena described or predicted by hypotheses and those recorded in so-called protocol sentences or basic statements, that is, records of empirical observations that satisfy the conditions of experimental science.
Against this foundationalist notion of science and its underpinning correspondence theory of truth, Neurath advanced his ship metaphor. Finding in protocol sentences a secure and objective basis for verifying hypotheses is logically impossible, he admonished us, for all observations contain themselves hypotheses, in the sense that they are conditioned by our interpretations of earlier observations. There is no such thing as a direct access to reality; observations always embody summaries of earlier observations! Consequently, no observations or observational statements can ever be considered foundational; all require justification. It follows that Nature cannot play the role of an independent arbiter of objectivity and truth that the correspondence theory of truth assigns to her. The attempt to ground truth in the objectivity of observational statements--that is, in a foundationalist reading of observational statements--cannot help but presuppose what it should produce: objectivity. Neurath aptly captured the situation with his ship metaphor:
There is no way of taking conclusively established pure protocol sentences as the starting point of the sciences. No tabula rasa exists. We are like sailors who must rebuild their ship on the open sea, never able to dismantle it in dry-dock and to reconstruct it there out of the best materials. (Neurath, 1932/1959, p. 201)
Research, then, is a process of reconstructing the ship while keeping it afloat. In distinction to the more common, foundationalist metaphor of science as a building established on the secure foundation of basic statements, the ship metaphor acknowledges the precarious, because unavoidably circular, nature of research. We cannot gain new knowledge without questioning (i.e., reconstructing) some previous knowledge (i.e., parts of the ship); but at the same time we need to presuppose that the overall structure remains strong enough to carry the whole enterprise of science (i.e., keep us afloat). Coherence of the entire structure, rather than correspondence of individual parts with some indubitable basic statements, is what constitutes the stability of science.
Hence, we have to abandon the quest for a secure foundation of science in the form of empirical verification of theoretical hypotheses or conjectures. Instead, we need to look for critically reflective ways of dealing with the unavailability of a secure foundation. In the following conjectures, I propose to examine some such critically reflective ways. I will refer to them in terms of critically reflective practice of research or simply reflective practice. I use the two terms more or less synonymously; but the first term has the advantage of being more accurate, whereas the second is less awkward to repeat and avoids the possible misreading of reflective research practice as an internal affair of researchers only.
Karl Popper’s (1935/1961, 1963, 1972) critical rationalism is probably the most widely respected model of the logic of research today. It appears to have influenced many researchers. It is generally considered to offer a sound critical response to logical positivism, and in many respects it does. In particular, it seeks to lead us beyond positivist research practice by replacing the method of “verification,” a process of empirical validation of hypotheses based on systematic observation and inductive reasoning, with the method of “falsification,” a process of empirical elimination of hypotheses based on systematic testing of hypotheses and deductive reasoning.
Popper’s exile, in the late 1930s, in Christchurch, New Zealand, provided him with a nice personal experience of the power of falsification. While in Europe, he had taken it for granted that all swans were white, but in Christchurch he found to his surprise, as everyone who has visited Christchurch knows, that some swans were black! Against his own philosophical insight, he had fallen into the trap of inductive reasoning and had wrongly generalized his previous experience of white swans.
Popper’s point was of course that no amount of observations (of white swans) can ever establish that all future observations (of swans) will yield the same result; but a single observation of a counter-example (a black swan) can very well falsify an erroneous generalization (the hypothesis that all swans are white) or may at least cast serious doubt on it. Imre Lakatos (1976, 1977) later pointed out that in the practice of research, it usually takes more than such an individual instance of falsification to definitely “falsify” (i.e., have us give up) an entire research program or research paradigm; the assumption that there is such a thing as a single, conclusive critical test is simplistic. However, this complication--the insight that conclusive criticism is a much more complex idea than Popper may originally have assumed--does not question the merits of the basic step he took towards reflective practice, I mean the step from the logic of verification to a logic of falsification or, as I would prefer to say, a logic of critique.
Popper thus replaced the classical empiricist principle of induction (or inductivism), according to which knowledge grows through generalization from experience, by the new methodological core principle of fallibilism (or deductivism), according to which knowledge grows through failure (i.e., conclusive criticism based on observation and deductive logic). The essence of the scientific method then consists in disciplined efforts to “falsify” (eliminate) rather than to prove (verify) theoretical hypotheses. Consequently, statements that do not lend themselves to falsification cannot be considered scientific statements at all; falsifiability is Popper’s criterion for demarcating science from non-science. The power of science, Popper taught us, consists not in its providing security against failure but in its critical method.
By abandoning the quest for a secure foundation in the form of empirical verification, Popper’s logic of research took up an essential implication of Neurath’s ship metaphor: research is a never-ending process of challenging current hypotheses by subjecting them to the most severe observational tests available. Even hypotheses that survive all tests remain inherently provisional; for we can never quite exclude that a future observation might contradict them.
There can be little doubt that the fallibilist turn represents an essential milestone in the gradual, though uncompleted, development of science theory from its positivist origins towards a framework for critically reflective research practice. However, appreciating the historical merits and the continuing relevance of critical rationalism in revising the positivist logic of research practice should not stop us from asking some critical questions. How critical is critical rationalism? How conducive is it to reflective practice?
Despite the fallibilist turn, the question for Popper remained: Where could researchers find the sources of critique that would allow them to decide which statements fail and which others do not, at least for the time being? This put him in a quandary. On the one hand, Popper agreed with Neurath that there is no such thing as a direct access to empirical phenomena that would not be mediated through theoretical expectations and interpretations. On the other hand, Popper, although he had abandoned the quest for verification, was not prepared to give up the quest for Objective Knowledge (Popper, 1972). Hence, the falsification principle still required him to presuppose some empirical basis that was free of theoretical expectations and interpretations, so that it could serve as an independent and objective arbiter of falsification. But how can we know whether basic empirical statements are themselves true and thus apt to falsify a theoretical hypothesis, if as a matter of principle no statement can be verified by experience?
Popper found no way out of the quandary. His answer, in essence, remained the same as that of the logical positivists, although it tried to avoid the requirement of verification. Falsification, he maintained, is possible by relying on the objectivity of basic statements (his preferred term for protocol sentences), gained through the methods of experimental science. Although basic statements cannot verify hypotheses and cannot be verified themselves, he insisted that they could still provide the kind of objective empirical falsifiers that the falsification principle required. He conceded that strictly speaking, falsifying statements could not claim objectivity based on observation alone but only in conjunction with the conventions of science and with decisions on the part of the involved inquirers (e.g., that what was observed was indeed a swan and that the observation of its being black was accurate and properly recorded); but he then played down the difficulty posed by this decisionistic core, by asserting that “in connection with [basic] statements, doubts as to their empirical character rarely arise” (Popper, 1935/1961, p. 43). He overlooked the fact that in research practice, issues of meaning and validity regularly arise and observational reports are often disputed. In fact, accumulated experience in many domains of research points towards the value of raising such issues, as we learn, for example, from the stories of DDT (Carson, 1962; compare NRDC, 1997) or the Aswan High Dam (Taghi-Farvar & Milton, 1972; compare El-Sayed & van Dijken, 1995).
Popper’s empiricist framework of science theory apparently did not allow him to consider--and take seriously--sources of critique other than those of experimental science. I am thinking, for example, of various forms of criticism drawn from the humanities (e.g., hermeneutic reflection, language analysis), from practical philosophy (e.g., ethical reflection, discourse analysis), and from open discourse among researchers and concerned citizens in a functioning civil society (e.g., democratically organized consultation procedures, participatory models of research). I may also be allowed to mention my own specific concept of criticism drawn from critical systems thinking and critical pragmatism, i.e., boundary critique, a form of critical discourse that examines the way in which the meaning and reach of validity claims in applied science and expertise depend on the context that is taken to be relevant (Ulrich, 1983, 2000, 2001, 2006). Considering the vital part that research plays today in problem solving and decision making everywhere, such an opening up of our universe of critical discourse appears vital. A well understood critical approach to research can no longer a priori limit itself to any specific source or method of criticism; for what ultimately matters is not where a critical challenge comes from but only what arguments support or question it. Such arguments can only be identified and examined in an open discourse that is accessible to everyone concerned and to all forms of cogent argumentation.
As I see it, Popper did not take seriously enough the discursive turn of research that his new, fallibilist logic of research required. Yet he had recognized that observational statements are always “theory-impregnated,” in the sense of presupposing concepts that cannot be inferred from experience. Hence, it would seem to follow, they cannot ultimately be validated empirically but can only be subject to critical discussion about the presuppositions in question. I would argue, therefore, that a fallibilist logic of research needs to be grounded in a discourse theory of critique (Ulrich, 2003, p. 326) rather than in the empiricist conception of “objective” theory that underpins his notion of critically rational discussion.
Popper apparently could not accept this implication of his insight. For him, criticism is “rational” (or “rationally decidable,” to use his expression) to the extent it succeeds in demonstrating that a criticized statement is logically inconsistent with a basic observational statement. The “organon” of criticism in what he calls a critically rational discussion is deductive logic (Popper, 1935/1961, p. 30; 1962, p. 115; 1972, p. 31). Popper’s concept of criticism thus remains fundamentally monological rather than dialogical. Logical deduction does not require dialogue and, in any case, it is ultimately nature which is supposed to tell us which statements represent objective knowledge and which others do not. This is not good enough. In a properly discursive conception of knowledge, arguments rather than nature ought to decide. Argumentation goes beyond deductive logic in that it submits all its procedures and criteria to critical discussion (Habermas, 1964/1976, p. 253).
I suspect Popper could not accept this consequence because it leads so far beyond the reach of critical rationalism’s only tool for rational criticism, i.e., falsification by reference to protocol sentences. His conception of the critical method of science remained attached to a narrow empiricist notion of “objective” criticism and thereby failed to overcome empirical foundationalism. We should not be surprised then, that critical rationalism has been unable to play the strong role in promoting reflective research practice that many expected and still expect it to play. When it comes to its underlying concept of criticism, critical rationalism falls behind Neurath’s old argument against empirical foundationalism.
It is probably symptomatic of the failure of critical rationalism to overcome empirical foundationalism that the older positivist paradigm continues to be influential as a widely practiced standard model of quantitative research. Since Popper’s quandary is no longer attracting much theoretical attention, research practice can go on undisturbed, as it were. Unlike Miguel de Cervantes’ (1612/2003, Ch. 8, p. 58f) hero, Don Quixote, we are, then, not just fighting windmills here, that is, challenging an imaginary enemy, as little as I am Don Quixote. As a contribution to this journal recently pointed out:
While physicist have, by now, removed themselves and their science from simplistic positivist stands when uncovering that relativity, uncertainty, and chaos govern that which they research, social scientists seem, for the most part, unfortunately not to be able to overcome the traditional positivist paradigm which governed the physics of old. (Bekerman, 2006)
For example, as Bekerman observes, many social scientists appear to adhere to the belief that the only way to handle the intrinsic complexity and multidimensionality of social reality is by expanding the reach of their quantitative tools for data collection and analysis. (Note that data are not necessarily quantitative only; that we tend to think so shows just how much the positivist paradigm of research is still deeply ingrained in our minds.) In this way, they expect, quantitative analysis can then deal with ever more variables at the same time and thus better represent the complex interdependencies of social reality. This may be true, but it hardly supersedes the need for more comprehensive notions of what the critical method of science means in the social and applied disciplines. Worse, it tends to divert the attention of researchers away from the need to develop new skills of critical argumentation beyond those of quantitative analysis and, at the same time, to revise their notion of professional competence accordingly.
We appear to be caught in a vicious circle: So long as a foundationalist notion of empirical science prevails in the practice of quantitative research, it offers researchers few incentives and leaves them little room for developing a different notion of competence; for its ideal remains that “the researcher, given a good methodology, is almost irrelevant in the research equation” (Bekerman, 2006). For many researchers, particularly in those social and applied disciplines which continue to emulate the model of the basic (meaning non-applied) natural sciences, foundationalist thinking has become a trap from which they cannot easily escape.
Thus, in the practice of research--against all the insights offered by the history of ideas in science theory--the vain search for a fail-safe foundation of knowledge appears to live on. Now and then it tempts even those who have escaped it to submit to the foundationalist trap once again, albeit in new ways. It seems to me that currently, two such temptations are in the upswing and risk leading us back into the foundationalist trap. Both represent misconceptions about the nature and sources of competence in research; both have essential roots in critical rationalism; and both are likely to work against reflective research practice. They warrant, therefore, a more detailed elaboration in the next two sections of this essay:
Perhaps the most important aspect of Popper’s lasting impact on our contemporary notions of “critical” research and research competence is the role he assigned to theory. More than any science theorist before him, Popper emphasized the theory-laden nature of all statements of fact. He never tired of reminding us that all observation is theory-impregnated in the sense of requiring some theoretical frame of reference or horizon of expectations (e.g., Popper, 1972, pp. 345-347). It determines what phenomena we investigate and see in the first place, what tools and measurements we use to observe and record them, and how we interpret them. Both temporally and logically, our theoretical horizon of expectations comes prior to observing and interpreting the world (Popper, 1963, pp. 23, 41n, 387). Our competence as researchers, then, is essentially grounded in theoretical knowledge. This is what Popper’s notion of “primacy of theory” originally stands for.
This idea of the primacy of theory was so powerful that it soon began to lead a life of its own. Popper began to apply it to one of the main difficulties of his approach, the question of how critically rational discussion can help us to choose among competing theories or models as a basis for rational action. Keeping with Popper’s terminology, the issue is known in the literature as the so-called problem of pragmatic preference (i.e., preference with a view to enabling rational action), as distinguished from the problem of theoretical preference among competing claims for truth (i.e., preference with a view to promoting knowledge). It is central for Popper’s understanding of rational practice and I will therefore also refer to it as the problem of critically rational practice, although Popper, as far as I am aware, has not used the term.
To understand what the “primacy of theory” argument means for critically rational practice (i.e., for Popper’s handling of the problem of pragmatic preference), we need to understand the two major difficulties with which the preference problem confronts critical rationalism. First, since rational action requires assumptions about the future, “pragmatic” preference for some theoretical assumptions over other conceivable ones appears to imply some kind of inductive reasoning (i.e., inferring the future from past observations). And second, since rational action needs to care about its value implications, pragmatic preference appears to require value judgments. The trouble is, Popper’s deductive approach offers no space for inductive reasoning and for discourse on values. Neither problem can be grasped in terms of logical inconsistencies between theoretical hypotheses and basic statements. Popper tried to solve the problem by associating critically rational practice with pragmatic preference for the best-tested among available theories:
Every action presupposes a set of expectations; that is, of theories about the world. Which theory shall the man of action choose? Is there such a thing as a rational choice? ... Since we have to choose, it will be “rational” to choose the best-tested theory. This will be “rational” in the most obvious sense of the word known to me: the best-tested theory is the one which, in the light of our critical discussion, appears to be the best so far, and I do not know of anything more “rational” than a well-conducted critical discussion. (Popper, 1972, p. 21f)
The reader will note that Popper’s argument passes over the two mentioned difficulties. Yet it cannot help but presuppose the very inductive reasoning that critical rationalism claims to overcome; for what is the point of relying on the “best-tested” theory and on a “well-conducted” critical discussion if not to assess the future reliability of the theory as a basis for action? I would not labor the point though, for despite the doubts it raises about the role of theory for critically rational practice, one might argue that there is indeed no better solution than relying on the knowledge basis that has shown itself to be most reliable thus far--at least so long as we keep the limitations of our knowledge basis under careful critical review. Note, however, that this critical task is beyond the reach of critical rationalism. We might then say that this first difficulty questions less the importance of theory for practice than Popper’s narrow notion of critically rational discussion.
More important with a view to the problem of ensuring critically rational practice is the second difficulty, the question of how critically rational discussion should handle value judgments. Rather than facing the issue, Popper’s argument avoids it as if relying on the best-tested theory could give an agent sufficient grounds for claiming that her or his actions are rational, regardless of the value implications they may have for third parties.
The inadequacy of this solution is obvious. Equally obvious is the only way Popper could hope to get round the issue: namely, by redefining the meaning of “rational practice” so that it would avoid the need for value judgments. This is where Popper’s primacy of theory argument comes into play. The argumentative strategy is to extend the notion that theory is primary in such a way that it becomes applicable not only to the relationship between theory and observation but also to the relationship between theory and practice.
At first glance, this may seem like an obvious extension, for it is clear that not only our descriptions of the world are conditioned by a horizon of expectations but equally are our attempts to change (improve) the world. Surely we cannot act rationally against our theoretical knowledge of the conditions and interdependencies that govern the section of the real-world in question? Of course not. But at a closer look, the following question poses itself: Does such relevance imply that consistency of practical proposals with theoretical knowledge is all there is to rational action? The answer is, of course, negative, except if we redefine “rational” action to mean merely instrumental action, that is, expediency (efficacy and efficiency) in using available means for reaching given ends. That is what theoretical knowledge, if put to practical use, can achieve: it allows us to translate causal or statistical explanations of the kind “X, given circumstances Y, produces effect Z” into technical prescriptions of the kind “to produce effect Z, make sure X obtains under circumstances Y”). In one phrase, it lends itself to instrumental reasoning.
On the assumption that rational practice is the same as instrumentally rational action, “primacy of theory” has thus come to stand for a constitutive role of theory for rational practice. While originally the point was to draw attention to the conditioned nature of all statements of fact and claims to knowledge, it now also means that recommendations for practical action or other claims to normative rightness are rationally decidable inasmuch--and only inasmuch--as we can derive them from theoretical propositions, that is, translate them into instrumental reasoning. Hans Albert, one of Popper’s main followers in Germany, summed it all up in the triumphant creed: “Nothing is more practical than a correct theory” (Albert, 1962, p. 55; similar formulations have been attributed to Kurt Lewin, Albert Einstein, and others).
As we now understand, the trap lies in what this popular creed does not say, rather than in what it says. Like Popper’s earlier argument, Albert’s reading of the “primacy of theory” stipulation passes over the fact that for all practical purposes, the findings and conclusions of our research are not only theory-laden but are equally value-laden, in the form of normative assumptions that guide the research and determine its results. One need only think of the research questions and conventions that guide us in selecting the relevant phenomena to be examined as well as adequate ways to observe, record, and measure them, or of the claims that we then associate with the findings regarding their meaning, validity, and relevance. Critically rationalist research practice thus falls behind Kant’s (1781/1965) richer, two-dimensional concept of reason, in which reason unfolds its power both in the theoretical dimension of empirical science and in the practical (normative) dimension of ethics.
Likewise, Albert skips over the fact that the only way in which theoretical reason becomes practical is through instrumental reasoning. Note that both his stipulation of the practical nature of theory and Popper’s doctrine of the primacy of theory over practice depend for their validity on the tacit assumption that the words “practical” and “practice” are to be understood in their instrumental sense only. Critically rationalist practice thus becomes blind to the other, normative dimension of practical reason. It cannot help but relegate the normative underpinnings and implications of research to a supposedly irrational, because extra-theoretical, realm of merely subjective acts of belief about which no reasonable discussion is possible; for it has no methodological means for dealing with them.
This is a fundamental misunderstanding of the nature of rational (or reasonable) practice indeed. It amounts to confusing non-theoretical and non-technical rationality with a lack of any kind of rationality, that is, with the impossibility of any kind of argumentation. In effect, this confusion then serves to immunize claims to objective knowledge or rational practice that rely on merely instrumental rationality against argumentative efforts guided by a less narrow concept of criticism (Ulrich, 1988, pp. 143-146).
It is hardly exaggerated to say that in contemporary science theory and research practice, this impoverished concept of rationality is omnipresent and continues to cause much harm. Popper’s postulate of the primacy of theory has replaced Kant’s older postulate of the primacy of practice, according to which the practical-normative dimension of reason is as essential as its theoretical-empirical dimension. The consequences are particularly grave for our contemporary understanding of applied science and expertise. Once we have eliminated from it the normative dimension and moreover have narrowed rational criticism down to establishing logical consistencies or inconsistencies, the space for reflective practice becomes narrow indeed. Only those who in a concrete situation happen to bring together the necessary theoretical-instrumental knowledge and methodological skills remain fully competent to contribute. All others, including the decision-makers involved and the parties concerned, will now depend on them (the “experts”) for understanding what rational action in this situation means. For the rest, as soon as they dare to have their own points of view, these will be likely to be considered merely subjective; and if they try to argue them, the experts will be quick to point out that they may not know enough about the issue.
Ironically, Popper’s original critique of empirical foundationalism thus paved the way for a new theoretical foundationalism. Either you are grounded in theory, or you have no grounds at all for claiming to be a competent participant. The new foundationalism here reveals its elitist and technocratic face as well as its impractical nature at once. It burdens researchers and professionals with the impossible role of having to “explain,” by virtue of their advantage of theoretical and methodological expertise, to all others what in a concrete situation would be a correct understanding of “the problem” and what might be done about it. At the same time, it largely immunizes these “explanations” against the critical efforts of concerned citizens. If they do not agree with the experts’ monologically presented findings and conclusions, it is their problem, as it were; for the reason can only be that they are insufficiently informed or are in any case unable to understand the reasoning of the experts. In a public referendum held in Switzerland in 1998 about the commercial release of genetically altered plants and animals into the environment, surveys found--and media discussions illustrated--that a majority of researchers in the field believed the wide-spread concerns of consumers were due to their lack of knowledge and understanding rather than any shortcomings in the arguments of proponents; the concerns of people were an expression of incompetence rather than of the existence of different valid ways of seeing the problem (Ulrich, 2000, p. 248).
To be sure, a good deal of skepticism and resistance against the claims of researchers is now growing among citizens everywhere. However, I fear this partial awakening from under the shadows of critical rationalism is due more to a diffuse dissatisfaction with the repercussions of applied science and expertise than to a clear recognition of the methodological pitfalls at work. The shortcomings and “side effects” of expert advice in everyday life are all too obvious to be ignored. Similarly, the public’s perception that researchers, when hired by opposing interest groups and decision makers, tend to chronically disagree about their findings and conclusions, is hardly helpful to sustain or restore the public’s confidence in the impartial rationality of expert advice. Although critical rationalism continues to be influential, the signs of the times are that its heyday is over.
In any case, my point is a methodological (i.e., prescriptive) rather than a sociological (i.e., descriptive) one. Sociologically speaking, our notions of acceptable research practice are clearly changing. Methodologically speaking, I am not convinced that the skeptical reactions to which I have pointed allow us to say that a majority of researchers, decision makers, and citizens are already seeing through the inherent difficulties of the contemporary research thinking and understand how these difficulties relate to Popper’s work on critical rationalism. That researchers disagree in public is not really what is wrong with the prevalent research practice--in fact it might be a strength; rather, the problem is a deep-seated loss of argumentative ground that this thinking has caused and continues to cause with regard to the normative dimension of rational practice. The critical intent of Kant’s earlier theory-practice distinction has given way to an expanded decisionism, a dichotomy of theory and practice in which the practice of applied science and expertise becomes divided into a rational, “scientific” part (the realm of theory) and a supposedly irrational, “psychological” part of merely subjective “decisions” (the realm of practice). When it comes to practical issues, researchers thus face an awkward alternative indeed: either they translate such issues into merely technical questions or else they cannot deal with them argumentatively at all. By adopting the expanded doctrine of the primacy of theory, Popper’s critical method has made itself the champion of a “positivistically bisected rationalism” (Habermas, 1964): its model of critically rational practice leaves no room for the practice of practical reason.
Perhaps a more adequate vision for research-based practice would be that of a living civil society in which active citizens and reflective professionals work together in the search for mutually acceptable solutions. How might our notions of expertise and research competence then change? And further, how might they change once we begin to ground them in the idea of a primacy of practice in research (Ulrich, 2001, pp. 9-11) rather than in Popper’s primacy of theory over practice? I believe that in such a vision, researchers would want to cooperate with non-researchers in a way that would help ordinary people to emancipate themselves from the situation of incompetence in which professional practice tends to put them. I believe that competent researchers would then want to understand such a self-limiting and emancipatory stance as an integral aspect of their professional competence and would also see in it an essential condition for rational practice (Ulrich, 2000, pp. 247f, 259f, 265f).
Related to the doctrine of the primacy of theory, a second misconception appears to gain ground in recent years. It considers theoretically based “methodology choice” to be the key to critically reflective research practice. This is currently a prominent idea in a number of applied disciplines such as organizational analysis, operations research, management consultancy, social research, information systems design, and others. Assuming that methodologies rely on different theoretical assumptions about the nature of real-world situations, the idea is that if we understand these assumptions, we can then classify methodologies accordingly and on this basis decide systematically upon the methodologies to use in a given situation. Such classification schemes have been proposed, among others, by Burrell and Morgan (1979), Jackson and Keys (1984), Jackson (1990), Flood and Jackson (1991), and Mingers and Gill (1997).
The basic idea, as already noted, is correct: reflective research--particularly in the applied disciplines--distinguishes itself from common-sense by its relying on explicit methodology, so that its procedures and results may at all times be controlled, repeated, explained, and challenged. But as it so often happens, correct ideas become extended beyond their original and proper application. In this case, the tendency is to equate the justification of methodology choice with the justification of the findings and conclusions: it is assumed that if we can justify the methods used in terms of underlying theoretical assumptions and apply them properly, we have also justified the validity and relevance of the research results, which often involve evaluations, change proposals, new designs, interventions, etc.
This equation of theoretically reflected methodology choice with sufficiently reflective practice is problematic in a number of ways. Demonstrating the proper choice and application of methods (assuming such demonstration can be achieved satisfactorily on theoretical grounds) does little to justify the normative claims that our findings and conclusions embody for those concerned. For example, methodology choice does not justify the claim that the problem at issue has been properly defined; that the situation has been adequately evaluated; that suggestions for improvement do justice to the needs and concerns of the proper target population; and that recommendations for action can be implemented and will be efficacious and efficient. These are all examples of claims that in practice are inevitable. They have a normative content in that the answers we give to these issues depend on previous value judgments and have normative implications. These in turn express themselves through the live, practical consequences that the research findings and conclusions, if taken as a basis for action, may have for the different parties concerned; but as a matter of principle, theoretical and methodological considerations cannot justify such consequences, only democratically legitimate procedures of will-formation and decision making can achieve that. Attributing major justification power to methodologies rather than to participative processes of argumentation betrays a technocratic and elitist stance.
In any case, critically reflective research practice involves more than informed methodology choice. While there is nothing wrong with the idea that methods should be chosen carefully on the basis of explicit theoretical assumptions, it does not follow that theoretically based methodology choice ensures sufficiently reflective practice. Still, promoting systematic methodology choice is relevant inasmuch as the methods we use tend to condition the findings and conclusions we identify in the first place. However, once a set of methods has been selected, there may still be options for identifying the relevant findings and conclusions. This conditioned and optional character of even the most rigorous findings and conclusions is always in need of critical reflection, but not all methods of the applied sciences are capable of supporting such reflection adequately. Hence, no theoretical scheme of methodology choice can comprehensively supersede reflective practice.
Furthermore, comprehensively reflective practice needs researchers to address a number of issues that form part of what I have very briefly introduced above (in Section 4) under the label, boundary critique. Examples of boundary issues to be examined are: Whose concerns are to be considered and whose not? What “facts” are relevant and what others may not be so relevant? How is “improvement” to be defined and to be measured? What stakeholders are to be involved? What is the appropriate time horizon to be considered for identifying potential side-effects and long-term consequences? and so on. Since today the concept of boundary critique is not yet an integral part of research training, contemporary research practice tends to neglect these issues. Even though some of the earlier-mentioned frameworks for methodology choice do give a place to boundary critique, the rising popularity of systematic methodology choice, as was to be expected, has done little to change the situation. The reason is simple: not even the most systematic framework of methodology choice can make sure that such issues are properly addressed in practice.
As a last consideration, what matters in the end is not the methods we use but the validity claims we associate with them and how critically we handle these claims in our argumentative practice. Critically reflective research practice is thus a much richer concept than the proper choice and application of methods. It requires a strong discursive culture, that is, an argumentative practice ready to deal with different kinds of validity claims in an open, self-critical, and participatory mode--a requirement that cannot methodologically be reduced to methodology choice (for a more detailed treatment, see Ulrich, 2003).
Many researchers and professionals still appear to yearn for a safe haven for developing and justifying their practice. Some appear to believe that theory provides such a safe haven; others invest their hopes in the search for an encompassing meta-methodology that could guide the choice and application of different methods and thereby would also justify the results. I have tried to argue that in the form of these two misconceptions, a new kind of foundationalism risks coming to life again.
In this new foundationalism, the role of the safe haven has tacitly been passed on from the protocol sentences of old to the theories and methodologies underpinning our research. A majority of researchers now apparently see in theory and theory-based methodology choice the guarantors of sound research practice. If this impression is not entirely wrong, it may be time to remind ourselves of Neurath’s ship and ask in what ways the two new forms of foundationalism risk being false guarantors of sound research.
Justifying interventions into social reality by referring to theoretical and methodological expertise is always a precarious idea. It risks assigning to applied science and expertise a role that it cannot legitimately fulfill, namely, the role of a superior arbiter among differing needs and interests of people. Yet this is exactly what happens everyday when researchers present their findings and conclusions on some current issue of concern and formulate corresponding recommendations, or when politicians justify their decisions by citing some studies that they commissioned researchers to do for them. The unspoken assumption is that reference to theoretical and methodological expertise can justify the manifold validity claims raised by interventions into social reality--claims to knowing what is an adequate definition of “the problem” and what constitutes adequate standards of “improvement,” claims to personal objectivity and impartiality in assessing the needs and concerns of different parties, claims to the theoretical and methodological conclusiveness of particular proposals for improvement as well as to their ethical adequacy, and so on.
There is also a mistaken notion of competence involved here. If reference to theoretical and methodological competence could indeed justify such claims, we would have to accept that researchers and professionals, by virtue of that special competence, know better than other people and thus should have more to say about what is to be done to improve the lives of people, although they have no corresponding political mandate and responsibility. All efforts to enable citizens to play an active role in resolving the major issues of our time, by learning to think for themselves and to question the proposals of authorities and researchers before accepting them--the emancipatory utopia of Enlightenment--would then be in vain. Likewise, all efforts to train researchers and professionals to become more self-reflective with regard to the way their proposals may affect different parties, and to limit their claims accordingly--the quest for critically reflective research practice--would then be futile.
The dream of a safe theoretical and methodological haven for research thus reveals itself to embody a rather technocratic and elitist view of applied science and expertise. We may be well advised to abandon this foundationalist vision of research, for it is not conducive to reflective practice. It risks being inimical to the development of an open, living civil society.
As a main conclusion and, at the same time, alternative vision emerging from this discussion, we may need to rethink the concept of reflective research practice in terms of a much richer concept of criticism than that implied in Popper’s critical rationalism. Such a concept of criticism probably cannot do without a comprehensive notion of reflective professional practice in the context of a living, democratically organized, civil society (Ulrich, 2000). Likewise, I would suggest, it needs to be grounded in pragmatic, yet critically tenable, frameworks for rational practical discourse (discourse for deciding on practical questions) and for professional ethics (ethics guiding action in concrete contexts of action). In my personal bias, the former might be based on the methodological core concept of boundary critique (Ulrich, 2003) and the latter on a pragmatization of discourse ethics (Ulrich, 2006).
These are, it appears to me, a few core elements of a future “philosophy for professionals” (Ulrich, in press), by which I mean a framework for reflective research and professional practice yet to be developed. Its chief aim will be to guide and support researchers in their daily quest for competent and reflective practice. Its notion of research competence will be very different indeed from that of critical rationalism. In particular, it will understand research competence in terms of these three indispensable qualities: reflective competence is
As a final suggestion, perhaps the Journal of Research Practice may become a platform for a shared effort of rethinking the concept of reflective research practice in such terms.
In the spirit of reflective practice, I would like to conclude with two questions: What is the meaning of critically reflective research if it is not conducive to reflective societal practice? And: How can we expect research to be conducive to reflective societal practice as long as we understand it in ways that tend to exclude a majority of those concerned from competent participation?
Adorno, T. W., Albert, H., Dahrendorf, R., Habermas, J., Pilot, H., & Popper, K. R. (Eds). (1976). The positivist dispute in German sociology. London: Heinemann, and New York: Harper.
Albert, H. (1962). Probleme der Wissenschaftslehre in der Sozialforschung [Problems of science theory in social research]. In R. König (Ed.), Handbuch der empirischen Sozialforschung (Vol. 1, pp. 38-61). Stuttgart, Germany: Enke.
Bekerman, Z. (2006). It’s we, the researchers, who are in need of renovation. Journal of Research Practice, 2(1), Article P1. Retrieved September 29, 2006, from http://jrp.icaap.org/index.php/jrp/article/view/25/45
Burrell, G., & Morgan, G. (1979). Sociological paradigms and organizational analysis. London: Heinemann.
Carnap, R. (1932a). Die physikalische Sprache als Universalsprache der Wissenschaft [The physical language as a universal language for science]. Erkenntnis, 2(1), 432-465. Translation: The unity of science. London: Kegan Paul, 1934.
Carnap, R. (1932b). Über Protokollsätze [On protocol sentences]. Erkenntnis, 3(1), 215-228. Translation: On protocol sentences. Noûs, 21(4), 457-470.
Carson, R. (1962). Silent spring. Boston, MA: Houghton Mifflin. (Also as 40th anniversary edition, Boston, MA: Houghton Mifflin/Mariner Books, 2002).
Cervantes, M. (2003). Don Quixote (E. Grossmann, Trans.). New York: HarperCollins. (Original work published in 1612)
El-Sayed, S, & van Dijken, G. L. (1995). The southeastern Mediterranean ecosystem revisited: Thirty years after the construction of the Aswan High Dam. Retrieved October 1, 2006, from http://www-ocean.tamu.edu/Quarterdeck/QD3.1/Elsayed/elsayed.html
Flood, R. L., & Jackson, M. C. (1991). Creative problem solving: Total systems intervention. Chichester, UK, and New York: Wiley.
Habermas, J. (1964). Gegen einen positivistisch halbierten Rationalismus [A positivistically bisected rationalism]. Kölner Zeitschrift für Soziologie und Sozialpsychologie, 16(4), 636-659. Reprinted in T. W. Adorno, R. Dahrendorf, H. Pilot, A. Albert, J. Habermas, & K. R. Popper (Eds) (1969), Der Positivismusstreit in der deutschen Soziologie (pp. 235-266). Darmstadt, Germany: Luchterhand. Translation in Adorno et al., 1976 (pp. 198-225).
Jackson (1990). Systems methodology for the management sciences. New York and London: Plenum.
Jackson, M. C., & Keys, P. (1984). Towards a system of systems methodologies. Journal of the Operational Research Society, 35(6), 473-486.
Kant, I. (1965). Critique of pure reason (N. K. Smith, Trans.). New York: St. Martin’s (Previously, New York: Macmillan, 1929). (Original work published in 1781)
Lakatos, I. (1976). Proofs and refutations. Cambridge, UK: Cambridge University Press.
Lakatos, I. (1977). The methodology of scientific research programmes: Philosophical papers (Volume 1). Cambridge, UK: Cambridge University Press.
Mingers, J., & Gill, A. (Eds). (1997). Multimethodology: The theory and practice of combining management science methodologies. Chichester, UK, and New York: Wiley.
Neurath, O. (1932). Protokollsätze [Protocol sentences]. Erkenntnis, 3(1), 204-214. Translation: Protocol sentences. In A. J. Ayer (Ed.), Logical positivism (pp. 199-208). Glencoe, IL: Free Press, 1959.
NRDC. (1997). The story of silent spring. New York: Natural Resources Defense Council. Retrieved October 1, 2006, from http://www.nrdc.org/health/pesticides/hcarson.asp
Popper, K. R. (1961). The logic of scientific discovery (2nd ed., K. R. Popper, Trans.). New York: Basic Books. (Previously in 1959, London: Hutchison). (Original work published in German: Logik der Forschung. Vienna: Springer, 1935).
Popper, K. R. (1962). Die Logik der Sozialwissenschaften [The logic of the social sciences]. Kölner Zeitschrift für Soziologie und Sozialpsychologie, 14(2), 233-248. Reprinted in T. W. Adorno, R. Dahrendorf, H. Pilot, A. Albert, J. Habermas, & K. R. Popper (Eds), (1969), Der Positivismusstreit in der deutschen Soziologie (pp. 103-123). Darmstadt, Germany: Luchterhand. Translation in Adorno et al., 1976 (pp. 87-104).
Popper, K. R. (1963). Conjectures and refutations: The growth of scientific knowledge. London: Routledge & Kegan Paul.
Popper, K. R. (1972). Objective knowledge: An evolutionary approach. Oxford, UK: Clarendon.
Taghi-Farvar, M., & Milton, J. P. (Eds) (1972). The careless technology: Ecology and international development. The record of the conference on the ecological aspects of international development convened by The Conservation Foundation and the Center for the Biology of Natural Systems, Washington University, December 8-11, 1968, Airlie House, Warrenton, Virginia. Garden City, NY: Natural History Press, and New York: Doubleday.
Ulrich, W. (1983). Critical heuristics of social planning: A new approach to practical philosophy. Bern, Switzerland: Haupt. (Reprinted in 1994. Chichester, UK, and New York: Wiley)
Ulrich, W. (1988). Systems thinking, systems practice, and practical philosophy: A program of research. Systems Practice, 1(2), 137-163.
Ulrich, W. (2000). Reflective practice in the civil society: The contribution of critically systemic thinking. Reflective Practice, 1(2), 247-268.
Ulrich, W. (2001). The quest for competence in systemic research and practice. Systems Research and Behavioral Science, 18(1), 3-28.
Ulrich, W. (2003). Beyond methodology choice: Critical systems thinking as critically systemic discourse. Journal of the Operational Research Society, 54(3), 325-342.
Ulrich, W. (2006). Critical pragmatism: A new approach to professional and business ethics. In L. Zsolnai (Ed.), Interdisciplinary yearbook of business ethics (Vol. 1, pp. 53-85). Oxford, UK, and Bern, Switzerland: Peter Lang.
Ulrich, W. (in press). Philosophy for professionals: Towards critical pragmatism. Journal of the Operational Research Society.
Received 26 July 2006 | Accepted 2 October 2006 | Published 10 October 2006
Copyright © 2006 Journal of Research Practice and the author