Machinology

I submitted the manuscript for the book Operational Images (part of our project Operational Images and Visual Culture) to the publisher, University of Minnesota Press. The book is forthcoming in 2023, but for those of you interested in a sneak preview, please find below the Table of Contents and the Preface (without the endnote references though) that starts the book.

Table of Contents

Introduction: Between Light and Data

Preface: Operational Images, All the Way Down

Chapter 1: Operations of Operations

Chapter 2: What is Not an Image? On AI, Data, and Invisuality

Chapter 3: The Measurement-Image: From Photogrammetry to Planetary Surface

Chapter 4: Operational Aesthetic: Cinema for Territorial Management

Chapter 5: The Post-Lenticular City: Light into Data

Chapter 6: A Soft Montage of Operations

Preface: Operational Images All the Way Down

This book focuses on operational images, a concept that first emerged in Harun Farocki’s audiovisual work (films and video installations) and writings. The often-voiced but almost as often insufficiently developed definition is simple, and it is already somewhat evident in the term itself: there are some images that primarily operate; they are not necessarily representational or pictorial. Operational images trouble what an image is, as far as it shifts from representational to non-representational, from the primacy of human perception of bodies, movement, and things to measurement, pattern, analysis, navigation, and more. They change the scales and terms of reference. Such images have an epistemic force, while they also are involved in an intervention in the world, whether directly or indirectly. These sorts of images fix epistemic details, but while they hold them in place, they also are mobile in ways that become an essential part of their institutional activity. These points are explored in the pages that follow.

Operational images appear most often in discussions of machine vision and automation of perception, where images are part of a broader system of analysis, identification, tracking, and destruction. The term operational image seemed to speak directly to the period from, roughly, the first Gulf War in 1990 to the current phase of military and security management of territories with drones as part of an ecology of observation, analysis, and surveillance. Even a brief survey of Farocki’s films and existing literature reveals these thirty years of militarized airspace and technological systems that incorporated much of the already existing logic of targeting would be a straightforward implication of the term. However, this is not to say that this book is primarily about war or airspace, or surveillance or drones–even if such are central parts of the theme that admittedly ranges much further in the past than just the context of smart weapons and machine-readability of landscapes, territories, and objects (including people). It was also not the sole interest in Farocki’s work, especially if we consider how, for him, operational images emerged in architectural modeling, traffic control systems, construction of affective environments such as malls, and other examples that have also paved the way toward current topics of AI culture. Furthermore, this book is also not merely about a particular class of images that emerge in high-tech contexts, even if, again admittedly, it is about visual culture and its transformation into invisual data culture. We are not seeking an aesthetics or politics of digital images. There are plenty of books on these topics already, and many of them are consulted and discussed in this one, too. Thus, there is no need to write yet another take on digital visuality or digital images, or perhaps even about visual culture.

This book is about a needed shift in emphasis to the operational image. From a discussion of the image, its particular formats, and even its functions, we move to a different, albeit closely related question that concerns operations, a key term that ties to infrastructures, logistics, and all manner of actions that function to sustain, mobilize, analyze, and synthesize the thing we have grown to call “images”. I am after the coupling of perception and action, of images that control, regulate, and amplify how bodies operate.^[1]

Both perception and action are terms defined in the course of this book: perception is not tied to any natural experience, optical capacity, or even a technique, and the action in question is not necessarily only the here and now of a human body but defined by more tricky causalities–including a relation to the temporality of futures, future-pasts, recursions, and more. In other words, operations do not act only on the present but across a broader timescale, including a temporalization of space (e.g., in forecasting systems).

Allow me to offer a guiding rhetorical figure that should be considered throughout this book as part of its conceptual scaffolding.^[2] Operationalism is already an established term in scientific methodological literature^[3], but also consider the rather famous and often overused parable of the turtle at the base of the universe. The narrative comes and goes in different forms, this one written down in Carl Sagan’s Broca’s Brain:

“Some ancient Asian cosmological views are close to the idea of an infinite regression of causes, as exemplified in the following apocryphal story: A Western traveler encountering an Oriental philosopher asks him to describe the nature of the world: “It is a great ball resting on the flat back of the world turtle.” “Ah yes, but what does the world turtle stand on?” “On the back of a still larger turtle.” “Yes, but what does he stand on?” “A very perceptive question. But it’s no use, mister; it’s turtles all the way down.”^[4]“

Instead of turtles, the history of photography, technical images, and visual media could be told in the same key: instruments built upon instruments, upon infrastructures, upon practices, upon techniques, upon further instruments and infrastructures, and so on. Operations built on operations that include elements that are material and semiotic, forms of knowing, and forms of mattering. As Geoffrey Winthrop-Young points out, such turtled ontology is also shared by the cultural techniques approach in media theory, concerned with “operational sequences involving actors, things and practices that, coming together, give rise to established cultural practices.”^[5] Could the same then be said about the operational sequences involving images? That there is a fundamental relational ontology at play, one that could be described by way of scaffolding operations? At the very least, we can confidently concur with Michelle Henning when she writes that “[t]he Victorian idea of the photograph as an image that makes itself no longer holds sway.”^[6] This applies to the specific fantasy of unmediated images drawn by sunlight and practices of sensing and registering signals as images and diagrams.

Joseph von Fraunhofer’s 1810s invention of the spectrometer–a later backbone of remote sensing and the analysis of materials through their spectral fingerprint–represents but one case from the alternative history of visual technologies. To verify and measure the dark lines existing in the solar spectrum–later named Fraunhofer lines–the spectrometer apparatus was an assemblage of multiple elements, camera-like technologies (six lamps and a shutter), a prism, and a modified theodolite: “an ordnance surveying instrument originally designed to measure angles for the production of maps.”^[7] As Susan Schuppli writes: “His work would eventually come to be used by scientists to determine the chemical composition of a remote object–our sun, some 149.6 million kilometers away–not through direct testing but by treating it as an image, one whose chromatic variance could be translated into the complex language of chemistry.”^[8] Treating the world as an image beyond its pictorial qualities is one key site where operations of epistemic value take place–but there are multiple operations already in place that produce such images. Fraunhofer’s laboratory set-up can be described as an essential part of the history of science of discovery of the qualities of the solar spectrum, but as an artificial set-up of said spectrum, it is also a microcosmos of light–even of interplanetary light. It is a simulation of the behavior of solar light, too, which then introduces the possibility of using this standardized view of the dark lines on the continuity of the spectrum to scale up and down across light and materials. All sorts of material instruments and scalar operations are piled upon each other, explicated and implied.

As photography (and yet not entirely just photography), Fraunhofer’s apparatus was a camera of sorts–one built like turtles on top of each other. I would go so far as to say that Fraunhofer’s apparatus represents one central instance of the history of operational imaging vis a vis remote sensing^[9] and that what was captured was not a picture in the traditional sense but a microcosmos (or a condensation of its visual characteristics) through a detailed concrete arrangement that facilitated this diagram of the abstraction to emerge, and it demanded a precise need for accuracy in the placement of angles and distances, a measurement-operation in its own right.^[10]

The recursive world of turtles and their relations–or camera-like instruments in laboratories and solar light–is a helpful figure of thought. At the same time, a more conceptual reference point for such operative chains of support is likely to be found in the  theoretical work of cultural techniques and operative ontologies. But more on that later.

To answer the question that likely has emerged, what is this book about? It is not about turtles, but operations all the way down: images that exist only because of other operations–and the operations that help us to understand the transformation of images from visual to data. One could justifiably argue this point about cinema already, as Thomas Elsaesser speculates about Farocki’s interest in simulations as not a replacement of reality but a chain of synthesis–“the downward spiral of simulation of a simulation of a simulation”^[11]–that pertains to cinema and post-cinema. In our case, we instead place emphasis on the operational transformation of the link between visuality, photography, spectral analysis, and data–a transformation that did not happen with digital culture, but circa 1900, as the story that follows will show you.