Parsing pictures: on analyzing the content of images in science

Important aspects of this work were developed during my dissertation research. While Dr Kathleen Okruhlik was integral to the development of that project, the central ideas that are discussed here owe most to Dr Patrick Maynard, who introduced me to the works of Kendall Walton and John Willats and showed me how beautifully they complement each other. Thank you also to the Philosophy Department at Dalhousie University for feedback on an earlier draft and particularly to Dr Mélanie Frappier whose criticism and support have been invaluable.

To my knowledge, there is no term that refers uncontroversially to the generic objects of interest in this paper, which are, roughly speaking, rendered (as opposed to mental) visual content bearers. Perini typically uses the term ‘visual representation’ (though this has Goodmanian baggage) and I mostly use ‘image’ as a generic term for this type of object.

In her paper in this volume, ‘Diagrams in biology’, Perini addresses a particular type of visual representation, the diagram, that is peculiarly well suited to a Goodmanian analysis (Perini, 2013: 5) and by so doing adds considerably to an explication of visual languages. Each of her five examples, however, could also be parsed using Willats’ approach and I am inclined to think that in the places where the spatial relations of the rendered image conform more or less directly with the spatial relations of the state of affairs depicted, Willats’ analysis would be both more detailed and more illuminating. Perhaps a mixed approach, treating the diagrams both as Goodmanian visual languages and as Willats-style pictures, would be more useful than either one alone. I invite my readers to consider the question for themselves, as a proper argument to this effect would take me beyond the scope of this paper.

By ‘traditional propositional attitude epistemology’ I refer to the type of approach that has dominated 20th-century Anglo-American philosophy, which analyses knowledge by specifying conditions under which an epistemic subject, S, can be said to have knowledge of a certain proposition, p. Paradigmatically, it is suggested that S knows that p if and only if S believes that p, p is true and S is justified in believing that p. Gettier (1963) both clearly articulated this approach and showed why it must be wrong, but the many responses to his argument have typically taken the form of additions or corrections to the concept of justification, rather than addressing the assumption that knowledge (and indeed belief) is an attitude of acceptance as true that S takes towards a proposition and that a knowable proposition must be capable of being true.

Although I do not argue for it in the current paper, I do think that the appropriate epistemic success terms for images that can be parsed through a Willats-style analysis is something like, ‘conformation’, ‘fitting out’, ‘true enough’ and ‘similarity in relevant respects and to a sufficient degree’. We might also want to follow Goodman in focusing on understanding as the central epistemological concept rather than knowledge (Giovannelli, 2009, §4.6).

It is worth noting that Goodman's goals in articulating this method of analysis are not the same as Perini's goals when she adopts them. Goodman introduces these distinctions, along with exemplification, repleteness and other ‘symptoms of the aesthetic’, to distinguish art from non-art (or science) (Giovannelli, 2009, §4.6), whereas Perini adopts them as a way of parsing and precisely articulating the content of images in science. What nicely comes out in Perini's discussion is that scientific images are often syntactically and semantically dense; thus unlike exemplification (especially metaphorical exemplification) and repleteness which really do seem to be symptoms of the aesthetic, syntactic and semantic features of symbol systems fail to illuminate the distinction between art and science.

Unfortunately, it is not entirely obvious what a symbol system is and how to distinguish one from another. Notice that this is no minor problem for Perini. She needs exact answers to such questions. We need to know exactly what the characters and rules of combination and interpretation are in order to precisely specify the content of a visual representation and determine its truth value through translation.

The qualifier here is important as an important subset of graphs make use of pictorial elements. For instance, Otto Neurath's program of public education made use of isotypes, which combine graphing technique with pictorial devices (see Neurath, 1974 for discussion).

Goodman scholars may wonder why Perini does not invoke his idea of exemplification, which depends on properties that a representation shares with its subject (albeit possibly metaphorically). While I have no good answer to give, the spatial properties of scenes, which Willats’ account of projection systems so nicely elucidates, seem to defy neat analysis as a form of exemplification.

I say ‘most’ to acknowledge the use of visual symbols to communicate by some of the language apes, for instance, Kanzi.

Notice, however, that similarity of visual experience entails nothing about any specific resemblance relations between the representation and the represented object. Indeed, it is the point of Ames illusions (and pictures thereof) that similarities of visual experience can often obtain where none but the vaguest resemblance relations exist between visual representation and represented object.

Willats’ (1997) book is a longer and more detailed treatment of ideas developed in an earlier book co-written with Fred Dubery, Perspective and Other Drawing Systems (Dubery & Willats, 1983). The images replicated in Figures 4 and 5 appear in both, as does much of the analysis of projection systems.

‘Represent’ is not used here in any familiar philosophical sense. For Willats, the term ‘represent’ depends on the psychology of perception and should not be understood as an abstract or stipulated property or relation.

For the sake of simplicity I pass over the various different kinds of perspective and oblique projection systems, which Willats explains in detail (pp. 46–65).

For the sake of brevity, I consistently assume that the marks in question are of a kind that typically appears in paper journals—ink on a page. In fact, ‘mark’ scopes over all physical substances that can be used in visual representations, including paint on canvas, pixels on LCDs, light on screens, chalk on slate and so forth.

Goodman actually defines mark more broadly and arguably takes the character to be the object of the most basic level of analysis. He writes, ‘Characters are certain classes or utterances or inscriptions or marks. (I shall use “inscription” to include utterances, and “mark” to include inscriptions; an inscription is any mark—visual auditory, etc.—that belongs to a character’ (Goodman, 1968: 131).

Here, the object is drawn in cavalier oblique projection—the front of the object is shown as its true shape and the lines creating the back and side of the object are their true lengths. In other words, they are not foreshortened, as they would be in a perspective drawing (Willats, 1997: 46–55).

Here, the object is drawn in what Willats calls ‘single-point perspective’—‘one of the principal faces is set parallel to the picture plane’ (p. 38) and the orthogonals all converge on a single vanishing point.

Perini's discussion of compositional diagrams in this volume, utilizes the same figure as an example, so it nicely brings out the utility and power of Perini's analytical approach for visual languages.

Some philosophers of science who maintain a conventionalist approach to scientific epistemology may dispute my conclusion here. This seems, however, to wed them to a strong form of antirealism in which neither ‘truth’ nor any other analogous epistemic success terms would apply.

It may be objected that basing the content of an image on an analogy with viewing objects implies that an object must be visible to be visually representable. This is not the case, as I have argued in ‘Why Feynman diagrams represent’ (Meynell, 2008).

Strangely, Perini expresses misgivings about her own theory. She writes: ‘We need to decide if our ability to express the content of a representation with serial linguistic representations (like text or mathematical symbols) is essential to its capacity to be true or false…. But there are… good reasons to reject the claim that expressibility in a language with linguistic syntax and semantics is a necessary condition for the capacity to bear truth…. [T]his investigation was launched to show whether nonlinguistic representations can bear truth or not. This question is begged by invoking the assumption that only representations whose content can be expressed with a linguistic form of representation have the capacity to bear truth. So the question of whether a micrograph could be true or false cannot be settled in this way’ (pp. 282–283). Fair enough, but the point is an extremely peculiar one given that by adopting the Goodman–Tarski analytical method Perini begged the question herself.

It does not follow, however, that no graph is pictorial or has pictorial features. For instance, Neurath's famous Isotypes (Neurath, 1974) are clearly pictorial in certain respects, using highly simplified and stereotypical pictures that represent the basic topological features of their objects. Moreover, Edward Tufte (1983) has ably explained how graphs and bar charts can make use of multiple different pictorial features to rhetorical effect.