This document has two parts. The first part offers an eclectic selection of quotes from a variety of authors that strike me as relevant and important. The second part provides a summary of the following works, each of which offer some potential or suggestions for addressing critical issues relevant to pursuing post-normal science. My tendency to date, has been exploration of epistemological, and methodological issues, so most of these focus on this more philosophical end of research issues.

I am very interested in developing this bibliography to cover a broad range of issues, and welcome suggestions for other references. In particular I am interested in suggestions and examples relevant at the pragmatic level: methods.

Acker, J., K. Barry, and J. Esseveld 1991 Objectivity and Truth: Problems in Doing Feminist Research in Beyond Methodology, M. M. Fonow and J. A. Cook, eds. notes

Dean, Jodi 1996 Solidarity of Strangers: Feminism after Identity Politics University of California Press notes

Huang, Chungliang Al 1983 Quantum Soup E.P. Dutton quote

King, G., R. O. Keohane, and S. Verba 1994 Designing Social Inquiry: Scientific Inference in Qualitative Research Princeton University Press notes

Lather, P. 1994 Fertile Obsession: Validity After Poststructuralism in Power and Method, A. Gitlin, ed. notes

Morgan, Gareth, ed. 1983 Beyond Method: Strategies for Social Research Sage notes

Morgan, Gareth, 1997 Images of Organization Sage quote

Ristock, J. L. and J. Pennell 1996 Community Research as Empowerment: Feminist Links, Postmodern Interruptions Oxford University Press notes

Roe, Emery 1994 Narrative Policy Analysis: Theory and Practice Duke University Press notes

Rosen, R. 1996 On the Limitations of Scientific Knowledge in Boundaries and Barriers, J. Casti and A. Karlqvist, eds.

Sanitt, Nigel 1996 Science as a Questioning Process Institute of Physics Publishing notes

Yanow, Dvora 1996 How Does a Policy Mean? Interpreting Policy and Organizational Actions Georgetown University Press notes

MISCELLANEOUS QUOTES

Interpretations are more powerful than "facts." (Yanow 1996, p 19)

Limitations upon our abilities to understand fully [our mental models of the world] might be best interpreted as 'limits of scientists' rather than 'limits of science.' Bearing this distinction in mind we should be sensitive to the sources of some of our favorite scientific concepts. (Barrow 1996, p2)

As Godel has shown in relation to mathematics, there is a fallacy in the idea that the propositions of a system of thought can be proved, disproved, or evaluated on the basis of axioms within that system. Translated into terms relevant to the present [sociological research oriented] project, this means that it is not possible to judge the validity or contribution of different research perspectives in terms of the ground assumptions of any one set of those perspectives, since the process is self-justifying. Hence the attempts in much social science debate to judge the utility of different research strategies in terms of universal criteria based on the importance of generalizability, predictability and control, explanation of variance, meaningful understanding, or whatever are inevitably flawed. These criteria inevitably favor research strategies consistent with the assumptions that generate such criteria as meaningful guidelines for the evaluation of research. It is simply inadequate to attempt to justify a particular style of research in terms of assumptions that give rise to that style of research (Morgan 1983, p14-15).

We need, then, to see policy analysis as a process of inquiry that seeks to ask questions, rather than as a collection of tools and techniques designed to provide the right answers. The "right answers" approach begins from the assumption that the perception of the problem is accurate, whereas the "inquiry approach problematizes the very definition of the problem... (Yanow 1996, p 15)

The practice of writing entails analyzing[, however,] writing has rarely been considered a methodological issue. Once we move, however, to a world in which multiple, even incommensurable meanings are the rule and social science is not seen as mirroring reality, "writing up" itself becomes, if not a form of research as data collection, a form of research in its presentation or representation of data. Writing practices themselves are, in this view, "ways of worldmaking." In presenting the researcher's view of policy and agency actors' views of their world, research narratives construct that world... And yet, authors are not alone in worldmaking. Readers share this responsibility (Yanow 1996, p 52).

What we cannot forget is that as researchers we too are constructing knowledge (Ristock and Pennell 1996, p77).

Scientists engage a subject of study by interacting with it through means of a particular frame of reference, and what is observed and discovered in the object... is as much a product of this interaction and the protocol and technique through with it is operationalized as it is of the object itself. Moreover, since it is possible to engage an object of study in different ways - just as we might engage an apple by looking at it, feeling it, or eating it - we can see that the same object is capable of yielding many different kinds of knowledge. This leads us to see knowledge as a potentially resting in an object of investigation and to see science as being concerned with the realization of potentialities - of possible knowledges (Morgan 1983, p13).

There has never been any real consensus as to what methods are to be allowed as unquestionably producing "scientific knowledge." (Rosen 1996, p201)

[We can perceive] science as a questioning procedure providing a heuristic pattern to an otherwise unplanned search for truth (Sanitt 1996, p 40).

We all have prejudices and preconceived ideas that permeate our thoughts and actions - no less in science than in other areas of discourse. However, in science, the notion of an objective truth, an ineffable reality has always been a chimeric goal. We can try to avoid this hidden agenda by careful and scrupulous analysis of our theories, thereby expunging these thoughts by naming them. But, for every thought expunged a number take their place - any finite ground gained seems to be on an infinite plain. The problem is that it is impossible to eliminate presuppositions because in order to do so we would have to eliminate ourselves. (Sanitt 1996, p1)

[Heisenberg's uncertainty principle] suggests that scientific research involves an interaction between the scientist and the object of investigation, and that what the scientist observes is directly related to the nature of that interaction. This view of science is closely related to Berkeley's (1910) observation that an object gains its objectivity only by being observed, and that objectivity must thus be a property that stems from the observer (Morgan 1983, p 12).

In a socially stratified society, the objectivity of the results of research is increased by political activism by and on behalf of oppressed, exploited and dominated groups" (Harding in Fine 1994)

Subjectivity is regarded by scientists and conservationists as an undesirable absence of objectivity rather than as a positive expression of sincerely held values" (Cole-King 1993, p 179).

Social relations always have an interpretive dimension (Morrow 1994).

It is how [methods] are used and for what purpose that counts" (Guba and Lincoln 1988).

[There is a distinction] between what people believe to be true and what really is true. Inquiry can determine what people believe (Smith 1990, p 171).

There can be no explanation which is not in need of further explanation (Popper 1972).

Acker, J., K. Barry, and J. Esseveld (1991) Objectivity and Truth: Problems in Doing Feminist Research

Doing feminist research presents some "problematic methodological issues" (p 133) that are common to alternative approaches in general. This paper focuses on: the relationship between researcher and researched, and the problems of analysis and of validity using a particular research project as an example.

Recognizing that objectivity and neutrality are problematic, the researchers used various strategies to reduce these and the power differential between researcher and researched with variable degrees of success. For example interviewees were often reluctant to take the lead in discussions. "People have ideas about what it is like to be interviewed and they want to be asked question so that they can give the 'right responses'... However, those with whom we had more than one interview increasingly took the lead (p 140)."

Another issue was the potential conflict between the authors' feminist perspective and that of the researched. This is an especial concern for research attempting to expand the interpretation and validation community. It is particularly problematic "when the researcher's interpretation is not only different but potentially threatening and disruptive to the subject's view of the world. For example... these housewives had a stake in their own definition which was also a source of worth and dignity, while we as feminist researcher interpreted their situations differently" (p 142). Their approach was to exclude these women from active participation in the analysis. The authors admit that "we have not solved this problem; we believe that the solution lies in accepting the dilemmas and maintaining an awareness of when and why we are not able to make the research process a true dialogue, thus giving full legitimacy to the subjectivity of the other as well as to our own" (p 142).

Analysis of the data presented more dilemmas. It was difficult to choose "what was "essential" at the same time that we tried to give a picture that provided a "totality"" (p 143) and also difficult to sort and categorize data with full recognition of the impossibility of sorting and categorizing the lives and stories of the women interviewed. Involving the interviewees in the process presented another dilemma: "what [the interviewees] wanted, they said, was more of our own sociological analysis. They wanted us, the researchers, to interpret their experience to them. Here, once more, we faced incompatibilities between various components of our feminist approach to social research" (p 143).

Regarding the question of validity, they note that "we are not interested in prediction, but adequate reconstruction" (p 145). Their consequent concerns included "adequacy of interpretation" - fitting the findings with social theory - and "adequacy of findings" - knowing that the results "fairly and accurately reflect the aspects of social life that we claim they represent" (p 145). Extensive interviewee feedback aids in the latter, but simultaneously introduces difficulty. Providing a brief account of one woman's story they indicate how the research process changed her perspective through a process of self-reflection resulting from and concurrent with the repeated interviews.

Working from a perspective in which we are trained to want to find a reasoned and connected account, we face live material that is constantly in the process of transformation, that is not organized in the way of academic theories, Virginia Woolf, among other novelists, may give a better account of the conscious experiencing of life in all its episodic and unorganized ways than we sociologists can achieve. However, as sociologists we can find representations of such experience that allow us to build a sociology for women, a sociology that connects experience at that level to its structural determination in the wider society. What distinguishes us from those who are not social scientists lies in our method of systematically attempting to reconstruct social reality and to put these systematic reconstruction into a social theory which we share with other social scientists (p 149).

Dean's work is not about research, however, she presents concepts and discussion that I perceive as useful especially regarding the notion of objectivity and the presentation of 'other.' She parallels Roe's (1994) suggestion that identifying oppositions is not, or does not have to be, negative. ""We" does not require an opposing "they," "we" also denotes the relationship between "you" and "me." Once the term "we" is understood communicatively, differences can be respected as necessary to solidarity. Dissent, questioning, and disagreement no longer have to be seen as tearing us apart but instead can be viewed as characteristics of the bonds holding us together" (p 8). The concept I find most instructive/useful is what she terms the "situated, hypothetical third."

The hypothetical third signifies the space occupied by the excluded other, the perspective that would be included if the voices of the marginalized could be heard. In moving from our roles as speaker and hearer to the role of observer, we are not moving way or standing apart from the action of discourse. We are instead trying to grasp and understand the implications of this action from a different place... Ultimately, the perspective of a hypothetical third strengthens the presupposition of accountability on the part of participants in discourse. In contrast to a "neutral third," which repeats the point of view dominant at any given time, the hypothetical third understands moral accountability as requiring competent subjects to see from the perspective of the dominated and excluded (p 172).
In other words, we take the attitude of the group, but we take it reflectively, attuned to the standpoint of the situated, hypothetical third (p 39).
[Such a position] resists claiming neutrality, arguing instead for accountability (p178).

Although Dean's focus is discourse and politics, I believe her reflections are relevant for research also. This suggests that discursive ethics and dramatism (two disciplines she draws from) are possible areas to purse for ideas regarding research for post-normal science.

Here's an exercise to help awaken the Buddha in you. Stand firmly but not tensely, with your feet an easy foot or more apart. Let your arms open roundly to the sky, face upward, chest high but easy. Breathe fully but without force. Now imagine two pure streams of water pouring from the heavens and washing over you.
Splash in the water with your arms flailing.
Open your mouth to the cooling downpour.
Let yourself fill with the joy and power of this gift.
When you are thoroughly refreshed, take seven strides to the east, point upward with your right hand, downward with your left and announce:
   "Worlds above, worlds below!
   The chief in all the worlds am I!"
Can you keep from laughing at this arrogance! Oh, I hope not. Because the laugh is what really awakens you.
Wake up.                                        (Huang 1983:12)

Reading this book after concentrating on post-modern approaches, several statements come as a surprise: "Many of the most important questions concerning political life... are philosophical rather than empirical. But the rules [of scientific inference] are relevant to all research where the goal is to learn facts about the real world" (p 6). "Good research, that is, scientific research..." (p 6). The suggestion that scientific inference offers the "biggest payoff" when "data are limited, observational tools limited, measurements unclear, and relationships uncertain" (p 10). And finally that "a proposed topic that cannot be refined into a specific research project permitting valid descriptive or causal inference should be modified along the way or abandoned" (p 18). Such comments make sense within a scientific framework. The suggestions of these authors are valid within this framework. Coming from a (at least semi-) positivist perspective, these authors could be applauded for their emphasis on research design issues relevant to improving social (especially political-social) research. However, I have concerns for the presumptions made and the bias their framework presents.

First is an issue noted by Laitin (1995, p 455) in critique of the book. "The summum bonum of political science... has never been valid causal inferences." He notes that many theoretical concepts have been very useful despite "any valid claims about the causal effects of these concepts." While the three authors present valid criteria for certain types of research, Laitin agrees that their coverage is restrictive. Although the authors claim to "sidestep" philosophical/post-modern issues (p 6), their position is based on a particular epistemological stance. My concern is that students and neophyte researchers will accept their logic without questioning this position.

The second concern relates to the issue of uncertainty and, more specifically, what Wynne (1992) calls "indeterminacy." The latter refers primarily to situations in which complexities are sufficiently high to make causal inference impossible. King et al. (1995, p 476) admit that "all interesting qualitative and quantitative research yields uncertain conclusions," which should "caution us to be aware of this uncertainty, remind us to make the best use of data possible and energize us to continue the struggle to improve our stock of valid inferences... We show that uncertain inferences are every bit as scientific as more certain ones so long as they are accompanied by honest statements of the degree of uncertainty accompanying each conclusion" (1995, p 476-7). A comment which begins to sound more like the post-modern approaches.

While I believe the authors present many useful considerations, their approach is more applicable to 'normal,' than to post-normal, science.

Patti Lather tackles the issue of validity - the question of how knowledge gains its legitimacy - from a postmodern position (postpositivism). As noted in the body of my paper, once pluralist perspectives are taken as valid, there is no standard against which to evaluate different approaches (who?). "What are the antifoundational possibilities outside the limits of normative framings of validity in the human sciences? What might open-ended and context-sensitive validity criteria look like?" (p 37)

She "call[s] for a kind of validity... in which legitimation depends on a researcher's ability to explore the resources of different contemporary inquiry problematics and perhaps, even contribute to... new forms of thought and practice" (p 39). Using a "strategy of excess and categorical scandal in the hope of both imploding ideas of policing social science and working against the inscription of another "regime of truth"" (p 40), Lather presents four "framings" of validity. She describes each and provides an empirical example. The framings are:

• Ironic Validity: Validity as simulacra which are "copies without originals" (p 40). "Contrary to dominant validity practices where the rhetorical nature of scientific claims is masked with methodological assurances, a strategy or ironic validity proliferates forms, recognizing that they are rhetorical and without foundation" (p 41).
• Paralogical Validity: Based on Lyotardian parology which has a goal "to foster differences and let contradictions remain in tension... The move is toward attempts to create indeterminate space for the enactment of human imagination... It is about the search for instabilities and the undermining of the framework within which previous "normal science" has been conducted" (p 43). "Foregrounding the availability of multiple discourses and how they can be used to decenter the researcher as the master of truth and justice... such a strategy refines our sensitivity to differences, introduces dissensus into consensus, and legitimates via fostering heterogeneity" (p 44).
• Rhizomatic Validity: This form is based on Derridean rigour which "undermines stability, subverts and unsettles from within... [and which] is a response to the call of the otherness of any system" (p 44). "As a metaphor, rhizomes work against the constraints of authority, regularity, and common sense and open thought up to creative constructions" (p 45). The example used for illustration stresses the importance of "attending to the creation of interactive social relations in which the inquiry can proceed" (p 46). Rather than playing with text, poststructural focus can develop "interactive social relations in inquiry... [and] create sites in the inquiry where voices can hear themselves and one another fruitfully" (p 46).
• Voluptuous Validity: Taking an extreme feminist stance this form asks how substituting female 'other' for male 'other' would alter perceptions and scientific epistemologies and the subsequent approaches to validation.

Lather creates a short check-list for these four types of "transgressive validity." Rather than apply them to her personal research narrative, however, she turns to note the importance of opening up space for discussing validation criteria "grounded in the crisis of representation" rather than grounded in the traditional sense of "correspondence between thought and its object" (p 52).

While her writing may be difficult to follow (or swallow!) for one unused to the post-modern rhetoric, she asks a multitude of relevant questions: "Who are my "others"? What binaries structure my arguments? What hierarchies are at play? How can I use Irigaray's concept of the "We-you/I" to disrupt those very oppositions, to create a constantly moving speaking position that fixes neither subject nor object, that disrupts the set boundaries between subjects?... The new canon is reflexivity...[yet] there are few guidelines for how one goes about the doing of it, especially in a way that both is reflexive and, yet, notes the limits of self-reflexivity" (p 50).

Her focus on "rhizomatics" draws from and parallels ideas of Deleuze and Guattari (e.g. 1983) which seem fruitful for conceptualizing complex networks and the consequent problematics. Though both works use philosophical postmodern rhetoric they present ideas which may be adopted, on a more concrete level, as powerful metaphors.

This book contains twenty-one different research strategies noting their constitutive assumptions (paradigms), epistemological stances (metaphors), and favoured methodologies (puzzle-solving strategies). Each is written up by a key proponent of the particular approaches using a mixture of descriptions and examples and ranging from positivist (e.g. Pugh who begins: "I am an unreconstructed positivist") to post-modern approaches, including a variety that exist in neither category. The list includes organizational learning, cognitive mapping, dramatism, critical theory, 'synthesism' and collaborative inquiry. Each of these approaches is considered in regards to organizational studies.

As has been shown, metaphors create insight.
But they also distort.
They have strengths.
But they also have limitations.
In creating ways of seeing they tend to create ways of not seeing.
Hence there can be no single theory or metaphor that gives an all-purpose point of view. There can be no "correct theory" for structuring everything we do...
In using different perspectives to create different modes of engagement we are able to tap into these and understand the same situation in many ways. Some of these ways may be extremely powerful, because they connect and resonate with the reality being observed. Other perspectives may prove weak or irrelevant, having little evocative or substantial power.
Scientists have generated powerful insights by studying light as a wave or a particle. But not as a grapefruit!
...Limit your thinking and you will limit your range of action.(Morgan 1997: 348-351)

Morrow makes many relevant observations. He notes that although quantitative and qualitative are representational differences they also represent analytical differences and draws attention to an argument I had not considered. In the study of social systems and interactions quantitative research focuses on social aggregates, qualitative on social groups. The former refers to collections of individuals that are grouped according to particular social criteria not as a result of social interactions (e.g. specific age groups, educational or marital status). Tests on such groups are administered to infer causal links to other social factors. In contrast, qualitative research tends to focus on groups identified through social relations (e.g. case study communities). While both are valid forms of research, the distinction is important. In addition, the two approaches complement and augment each other.

Reflection "has led us to see any research methodology as a culture, a set of norms or sanctioned practices for carrying out research... [and] to identify 'links' and 'interruptions' as essential features of a research method centered on empowerment. The 'links' open our research to a wider group... and keep our studies grounded in diverse realities; the 'interruptions' prevent our thinking from becoming either rigid... or amorphous" (p 9). These concerns and considerations have led these two authors to develop/describe a framework that integrates feminist and postmodern approaches with their own backgrounds and experiences. "Research as empowerment [is] an approach to research that seeks to effect empowerment at all stages of the research process through critical analysis of power and responsible use of power" (p 9).

This book provides a good balance of reflection, suggestions and examples from practical experience. The authors use a variety of strategies to balance power between researcher and researched and to extend the research process beyond the researchers: "Inclusive communities... involve people in research who are not conventionally thought of as having the knowledge and skills to design, conduct, or appraise research" (p 12). "Rendering accounts" involves the researchers holding themselves accountable through critical analysis and responsible use of power in the research process. Contemplating "intellectual autobiographies" encourages researchers to understand how their individuality informs their research through analysis instead of just by articulating it. Rather than acceding to the positivist approach, they encourage "validity as integrity."

Some of the key reflections these authors articulate are shared by other postmodernists: "Instead of denying or ignoring areas of tension in our work, we must learn how to anticipate, think through, negotiate, and work with power as a way of enriching the research process and maintaining its integrity" (p 68).. "Instead of starting with a pre-set question, these research efforts were motivated by a sense of the urgency of finding solutions to social injustices... The participants' efforts to transform society became identifiable as research when we were able to delineate a specific issue about which we wanted to learn and to which we wished to respond with a critical analysis and responsible use of power: this became the new research question" (p20). "People can effect change by speaking with each other about their experiences, gaining a wider perspective, uncovering their assumptions, and opening the way for alternative ways of life" (p 95).

In considering the post-normal science concern with extending the peer community, and the discussion in my paper regarding different levels of objectivity, the chapter on "creative analyses" provides fresh and insightful considerations. The stated objective is to move "thinking outside the confines of dominant assumptions and the researchers' own interpretations to invent alternative truths" (p 79). They suggest "principled creativity" as a "means for listening for competing discourses, distinguishing how each seeks to construct social relations in line with its own view of reality and political interest, and then deconstructing them by baring assumptions and undermining logic" (p 79).

Though describing a universal approach is impossible, four principles are offered: (1) expanding dichotomies, (2) speaking with those about whom one is speaking, (3) appreciating discourses, and (4) displacing logics. In the case example used, each principle was approached using a different type of analysis: (1) semiotic square to identify/expand the basic dichotomies, (2) constant comparative method for grounding theory and generating 'images' expressed by participants, (3) hermeneutics to understand the comments and 'images' within context, and (4) deconstruction to question the basic assumptions underlying the first three analytical approaches. Rather than perceive these a four separate steps, the value and power of creative analyses is in combining and moving back and forth between the different approaches. "By building a self-contradictory process into the analytical approach, a researcher can promote critical analyses and responsible use of power at the stage when the research is most likely to become cut off from 'inclusive communities'... Creative analyses foster 'inclusive communities' by distinguishing rather than subsuming people's positions, by remaining accountable to rather than detached from those whose positions are being interpreted, by understanding rather than rejecting positions, and by inviting reflection rather than arriving at any one final position" (95-6).

Emery Roe uses contemporary literary theory as a basis for constructing an approach to intractable public policy issues that he titles narrative policy analysis. Noting that "many public policy issues have become so uncertain, complex, and polarized - their empirical, political, legal and bureaucratic merits unknown, not agreed upon, or both - that the only things left to examine are the different stories policymakers and their critics use to articulate and make sense of that uncertainty, complexity, and polarization" (p 3). He describes a four step process (though reluctant to imply either universality, precision or simplicity). (1) Identify the stories. (2) Identify the nonstories and counterstories. (3) Compare the two sets of narratives to generate a metanarrative. (4) Determine if and/or how the metanarrative recasts the issue to make it more tractable.

In the context of the position I have articulated for post-normal science (between positivist and post-modern paradigms) Roe's comment that "narrative policy analysis is just as compatible with assumptions of realism as it is with relativism" (p 10) is encouraging. His rationale is that the starting point for analysis is the "reality of the uncertainty and complexity in the polarized issues and controversies" (p 10) regardless of how people claim or establish this reality. In addition, the focus of the analysis is narratives. "Even when their truth value is in question, these narratives... have the objective of getting their hearers to assume or do something... The policy narratives... are treated by many of their narrators as continuing to retain some explanatory or descriptive power, even after a number of the points or assumptions upon which they are based are understood to be in doubt and subject to serious qualification" (p 37). The tragedy of the commons is used as an example where "reality and story coincide imperfectly" (p 37).

Due to the nature of the research, details of the approach must be case specific, however, they follow a general pattern. To begin the analyst gathers stories through unstructured interviews, disaggregating the transcripts into discrete problem statements. The latter must note assertions of causal relations, as opposed to making any attempt to determine if they are factual relations. Using network analysis key problem statements and their interrelations are identified and the various narratives are developed. Critical literary theory, coupled with network analysis, then aids in identification of circular arguments, terminal problem statements, contradictories (sometimes "a term is best defined by what it is not" (p 98)) and other elements which allow the metanarrative to be identified and understood. "There are no guarantees that a controversy will have a metanarrative, or that there will be only one metanarrative and that it will always be policy-relevant" (4). In the case that it does, the hope is that it can be approached by more conventional policy analysis or that the stories can be woven into a new metanarrative that provides a new opportunity for coping with the situation.

The latter is perhaps the most valuable tool to be gleaned. The approach "takes seriously the need of analysts, policymakers, and the concerned public to act upon... what they already recognize to be the scenarios and arguments driving issues of high controversy" (p 12). Two points Roe stresses in the introduction relate to uncertainty and tolerance. "Narrative policy analysis requires uncertainty , complexity, and polarization... because without them there would be no policy narratives to underwrite and stabilize the assumptions for decision making... In narrative policy analysis, polarization is treated as an opportunity, not a barrier, for further analysis and policy relevance, the chance to reread polar opposites into a different story... (p 17). Tolerance and the encouragement of marginalized voices is important since the multiplicity of voices improves the chances of defining and understanding the metanarrative. The positions taken in a controversy may ultimately represent incommensurable values and - this is the important part - because incomensurability does not mean the positions are therefore incomparable. They can be compared and contrasted, at least for the purposes of generating another narrative altogether, one that could be more helpful than any of the positions on their own" (p 19).

Lest the reader perceive all of this to reflect philosophical, post-modern rhetoric, I note that Roe's approach is to explain the process through application of the analytic technique to several case studies.

Aristotle... claimed that it was the entire job of science to account for "the why of things." (200)

The Aristotelian view, in which science is content-determined in terms of the kinds of questions it must answer, has been replaced by method-based procedures. That is, something belongs to science according to how it was obtained, not by what it is about. This constitutes a truly massive shift in outlook. Indeed, as a result of it, the question as to what is "scientific knowledge" has shifted from something content-based to something quite different: the adequacy of an admissible methodology. (201)

The older Aristotelian view pertained to content rather than method; it consisted of answers to question; to information. The second, more modern view, pertains not to contest so much as to method; to process. The two pictures of science do not coincide. Still worse, there has never been any real consensus as to what methods are to be allowed as unquestionably producing "scientific knowledge." (201)

In earlier centuries, the physicists' dream was embodied in the Laplacian Geist, who could know the motion of every particle and every force; who could formulate and solve every N-body problem. He might not be able to tell the difference between a universe with life in it and one without it; but then he could not conceive of a system which was not an N-body system. The limitations to what he could know are limitations in him, not in the universe he perceives; and he could never even know what they are.
I suggest that we humans are more fortunate than the hypothetical Ultimate Reductionist, in our ability to perceive complexity. That is, to recognize the necessity to pull ourselves outside the limitations of self-imposed methodologies, which create nonexistent "limits" to knowledge itself.
I believe, in short, that Aristotle was more correct in his view of science as a content-based thing, than the more currently orthodox views of science as constructed by a method. If this is so, then the problem of "limits" to science evaporates into the mist.

Sanitt perceives science as a problem-generating activity rather than a problem-solving one. Using network and graph theory, he presents science as a network involving a flow of "puzzlement." Proposing an answer-question sequence instead of the traditional question-answer, he constructs network graphs using answers as the linkages between questions. Subsequent application of concepts from graph theory provide a tool for determining the acceptability of a theory. He stresses the latter - as opposed to truth - must be the criteria for legitimation. Quantum theory and evolution are used as examples.

Graph theory allows that "the scientific aspects of a theory can be identified in a clearer way and alternative theories can be assessed in structural terms... open-ended questions can be identified and areas which could be linked to other theories can be exposed more readily. This facilitates the setting of a theory within a wider context...it is quite useful to identify what is not a theory or at least what is close to being but is not quite a theory... (p 111).

For me, the potential offered by these ideas is primarily the introduction to graph theory and its application to conceptual considerations. Can such application extend to incorporate and integrate the different and multiple aspects of knowledge and legitimation?

Yanow began to "focus on the problem of meaning... as a very practical, research-oriented, rather than philosophical, set of concerns (p 55). Using an extensive case study of community centers in Israel, she answers her title through application of "interpretive analysis" which is an approach "that focuses on... the values, feeling, and/or beliefs which [policies] express, and on the processes by which those meanings are communicated to and "read" by various audiences (p 8-9). It is "an interpretive approach to the human, or social, world [that] shifts the focus from discovering a set of universal laws about objective, sense-based facts to the human capacity for making and communicating meaning (p 5). The social world needs an understanding of the policy process as not only an instrumental process. It is also about the inculcation of values and the validation of status, things that cannot be measured according to rational techniques" (p 6).

Recognizing that much has been written on various method of data collection, Chapter 2, outlining her "methodology" focuses on aspects of meaning. This includes both how a researcher discovers/knows when something has symbolic meaning for the policy and/or actors involved and what weights should be attached to different "meanings." Another set of concerns relates to the "constructions of meaning in the presentation of research finding in articles, books, and other academic narratives" (p 35).

Meaning-focused research requires self-reflexivity: the researcher is constantly "living" in two worlds, making sense of self ("my" world) and making sense of the community, organization, or other type of group which is being studied (the "other" world). In putting oneself in that other world, one becomes "the stranger" - the one for whom the social rules and norms are not necessarily known or transparent and immediately discoverable. In this process of intentional and reflective "self-estrangement," one creates the possibility for "puzzles" or "tensions" to emerge between what one expects to find, hear, see, feel and what one does find, hear, see, feel. It is out of reflection on these puzzles or tensions that the researcher begins to perceive that members of the group being studied have invested something with meaning that is different from hers, or that different subgroups disagree about what that meaning is... (p 45).

A close match between policy intentions and agency outcomes is the exception rather than the rule. Interpretive analyses focus on such discrepancies, directing our consideration toward policy objectives other than those stated explicitly and widening the scope of evaluations to include the contextual meanings by which implementation often succeeds or fails (p 234).

Policy analysts, in an interpretive approach, take on the role of interpreter between and among interpretive communities. The first step in analysis is to identify these communities... and their different understandings of the issues at hand. A next step is to identify the elements of each group's interpretive framework and the particular symbolic artifacts through which each expresses values, feelings, and beliefs, as well as ways in which these artifacts express different and possibly incompatible meanings for different groups. As interpreters, policy analysts learn to speak - to make sense - within each group's framework and within that framework to explain - to translate - other groups' interpretive frameworks. In this, analysts take on an educative role, attempting to help each community "see" how other groups are making different, and perhaps incommensurable, meanings out of the same symbols...
Translating is not simple... Aside from the technical difficulties of becoming fluent in another conceptual language, speaking does not make things so. We are human, not deities like the God of Genesis who spoke a whole world into existence. Let us not fall again into the fallacy of faith in ideal language and rational systems of signs, believing that reasoned discourse will be sufficient to overcome differences of worldview and their related practices. At the same time, attempts to communicate and to interpret communications, whether through languages more traditionally conceived or through the languages of objects and acts, are all that we have (p 237).