Why draw at all?
Every case in the first series is already described in long official texts. The Royal Commission report on Robodebt is close to a thousand pages. The Dutch parliamentary inquiry into the childcare benefits scandal is a whole book. So it is fair to ask: what can a drawing add?
My answer is: it shows how things relate to each other. A report describes harm one topic at a time: first the debts, then the health effects, then the court cases. The spectrogram puts all seven dimensions of harm on one timeline. In one view, you see when the system started to harm people, which kinds of harm came first, when someone stepped in, and what remained after. In Robodebt, the gap between the first warning and the moment the system was stopped is two and a half years. On the spectrogram you see this gap as a distance before you read it as a number.
The Encounter Plate has a different job. Each system in the series watched people: it scored them, ranked them and flagged them. The filmmaker Harun Farocki called images made by machines for machines, and not for human eyes, “operational images” (Farocki 2004). The artist Trevor Paglen later applied this idea to the systems that now produce most images in the world (Paglen 2014). The plate turns this look around. It draws the system itself as the thing being observed, in the style of an astronomical photograph. That is why each plate title starts with “OBS.”, for observation. The eye that scored people becomes the thing that is recorded.
Where this work comes from
This practice belongs to a tradition of art that works with public records. In 1971 Hans Haacke made Shapolsky et al. Manhattan Real Estate Holdings, a Real-Time Social System, as of May 1, 1971. It used only photographs and documents from public property records. The Guggenheim Museum cancelled the exhibition before it opened. Haacke’s point was that all the material was public and anyone could check it. Arranging it together showed a pattern that the scattered records hid.
Forensic Architecture, the research group founded by Eyal Weizman at Goldsmiths, University of London, turned this idea into a method. Its work is used as evidence in courts, parliaments and truth commissions (Weizman 2017). Susan Schuppli writes about the “material witness”: objects and media can carry traces of the events they were part of, and can be read as testimony (Schuppli 2020). My work is smaller and makes a narrower claim. It does not produce new evidence. It takes evidence that courts and inquiries have already established, and keeps it in a form that can be compared across cases and over time.
Four decisions
1. Every mark comes from the record. Each visual feature of a plate or a spectrogram is set by a value in the case record, following a written specification. The same record always produces the same image. An image only changes when the record changes, and every change is documented. The artist Sol LeWitt wrote that in conceptual art “the idea becomes a machine that makes the art” (LeWitt 1967). In my work this rule has a different purpose. It is not there to free the artist from choosing forms. It is there so that every form can be checked. Anyone with the record and the specification can see why a band is bright or a ring is blurred.
2. Harm is never reduced to one number. Each case is scored on seven dimensions: liberty, dignity, employment, family, housing, health and reputation. The scores are never added up or averaged, and the cases are never ranked by total harm. This is a deliberate choice. Every system in the series did the opposite. It took a person’s whole situation and reduced it to one risk score, and then acted on that number. If my record answered with its own single number, it would repeat what it documents. Catherine D’Ignazio and Lauren Klein (2020) show that the choice of what to count, and how, is a question of power before it is a technical question. In my method, refusing to add up the scores is the main way this choice is made visible.
3. Uncertainty is drawn. When there is not enough evidence to score a dimension, the band is hatched with diagonal lines. When harm is suspected but cannot be measured, the band has a dotted outline. When it is not certain that the system caused the harm, the ring on the plate is blurred. When serious harm is estimated from how the system worked, and not counted, the red mark is hatched instead of solid. Weizman describes much of today’s violence as happening “at the threshold of detectability”: it is hard to prove either way. A record that showed only what is proven would make these cases look less harmful than they are. A record that filled the gaps with guesses would claim more than it knows. So the method marks the gap, and says what kind of gap it is.
4. The people affected are not shown. No plate or spectrogram contains a face, a name or any detail that could identify a person. People appear only as stars. The brightness of a star does not show how much one person was harmed, because that is not known. On the Optum plate, the stars are fainter and there are more of them, because the patients never learned that a score had been used on them. In the case texts, people are named only if they have made their own case public. Children are never identified. Susan Sontag asked what it means to look at images of other people’s suffering, and what this looking does to the viewer (Sontag 2003). My work does not make such images. It asks the viewer to look at the system instead.
Why no images made with generative AI
The work uses no images made with generative AI. This is not a general position on these tools. It follows from the purpose of the record. An image made by a generative model cannot be traced back to its causes. The same prompt gives different results, and the training data behind an image cannot be checked. A work about decisions that could not be traced back to their reasons would be weaker if it used a process with the same problem.
Limits
Scoring is a judgement. Whether a scoring system’s harm to dignity is an 8 or a 7 is a decision made by a person who reads the evidence and follows written criteria. The method does not hide this. The codebook, the scoring criteria and the sources for each case will be published in an open catalogue, separate from the artworks, so that anyone can examine and question the scores. Each case record shows when it was scored and when it was last checked, and every correction is dated. A record that asks institutions to take responsibility for their automated decisions must meet the same standard itself.
References
- D’Ignazio, C., and Klein, L. F. (2020). Data Feminism. Cambridge, MA: MIT Press.
- Farocki, H. (2004). Phantom images. Public, 29, 12–22.
- Haacke, H. (1971). Shapolsky et al. Manhattan Real Estate Holdings, a Real-Time Social System, as of May 1, 1971. Photographs, documents and maps.
- LeWitt, S. (1967). Paragraphs on conceptual art. Artforum, 5(10), 79–83.
- Paglen, T. (2014). Operational images. e-flux journal, 59.
- Schuppli, S. (2020). Material Witness: Media, Forensics, Evidence. Cambridge, MA: MIT Press.
- Sontag, S. (2003). Regarding the Pain of Others. New York: Farrar, Straus and Giroux.
- Weizman, E. (2017). Forensic Architecture: Violence at the Threshold of Detectability. New York: Zone Books.
Text: CC BY-NC-ND 4.0. Corrections to contact@andriizupko.com.