

We find ourselves here at the USC School of Cinema, adjacent to the George Lucas and Steven Spielberg buildings. One of the centers of the many-centered simulation industry, where the craft is both theorized and taught. I'll begin with an acknowledgement that the talk you will see tonight understands itself to be training data for some future model about this present moment. In fact, everything you do today was training data for some future model of the past. For the Hawthorne effect we call our shared reality. The present is a simulation of itself as the past. Put differently, moments in time have shadows, as do each of us.
Shadow Metaphysics
What is a shadow, however? It is something both obvious and mysterious, perhaps one because the other? An object can be seen through the reflection of light, but this reflection is also a filtering so that the light cannot reach whatever is on the other side of that object. This produces an artifact which can be interpreted as both a kind of subtractive absence and as a new thing with significant but incomplete attachment to the original object whose profile it resembles.
This secondary void/object is what is called a shadow. It tracks the object from which it seems to be projected but also changes in shape and size in ways the original object cannot. At times the shadow appears as a dark two-dimensional replica of the object but if the source of light is closer to the ground plane then the shadow bends and elongates and in doing so seems to take on a formal identity of its own, different from the object.
Anyone who spent their childhood watching cartoons can half-remember characters peeling off their own shadow, moving their shadow to a different spot, dancing a duet with their shadow, boxing with their shadow, and so on. The allegories at play here are less metaphysical than psychological. Generally they are similar to the dilemmas posed by a mirror reflection in which self is recognized as an exterior object and specifically, they play with a potentially horrifying loss of self-control, or perhaps control over the external effects of one’s thoughts, emotions, and actions. To box with one’s shadow is to be of two-minds about something with existential consequences. To peel off one’s shadow and fold up into a suitcase (as a cartoon cat once did) is to decouple oneself from those consequences of agency, or at least to feel as though that agency is already decoupled.
Speaking of which, Plato’s allegory of the cave is, among other things, a theory of simulation at the core of Western philosophy and is concerned not only with shadows misperceived as primary objects, but with the prospect that what we take to be everyday objects are themselves a kind of shadow of yet more primary forms.
This worry haunts the ongoing critical suspicion of phenomenal appearances, and what does and does not constitute the correspondence between image and truth and reality. Perhaps then, the philosophy of simulation begins with the beginnings of philosophy itself. These simulations were not just a topic for philosophy, but perhaps a kind of foundational paranoia from which philosophy was born.
In my misspent youth, I took a trip bumming around Indonesia, and one of the few things I brought with me on this trip was a dog-eared copy of “Simulations” by Jean Baudrillard. The true object of which I show you here. Among my favorite memories from this trip was attending late night epic dramas told with shadow puppets in open air temples in the Javanese city of Yogyakarta. Here crowds of people wandered in and out, sharing tall bottles of beer, various snacks in seven, eight, nine-hour epics that would unfold and cycle back around and start again over and over until dawn. Shadow puppetry, you see, may stage a similar dilemma as Plato’s “Allegory of the Cave,” but does so by perhaps inverting it.
This particular cave accessed the agency of the shadow, perhaps more directly, and presents them not as a deceptive illusion, but as direct presentations of the fundamental mythic systems that underlie daytime reality, but which are otherwise obscured by it.
Part 1: A General Theory of Simulation
What about the here and now? Today we live in and within caves of our own making. We build simulations that both get us closer and further from reality. Arguably all simulations are based on models, but not all models are simulations. That is, different forms of simulation are based on different sorts of models.
With your permission, let’s start with a little bit of theory before diving into examples which will be more fun.
Some models are descriptive–their value is in how well they correspond to some reality. Others are primarily predictive–their value is in how well they anticipate a future state condition. And others are projective or normative–their value is in how they articulate a possible or preferred reality that does not now exist and may be unlikely to exist if not for the power of the normative model.
What do I mean by this exactly? Descriptive models are, for example, the domain of simulation science seeking to understand complex natural phenomena. Predictive models, such as financial simulations, hope to guess market futures. And projective or speculative simulations are the domain of, for example, religious prophets, politicians and designers, seeking to give shape to a better world. These are all models, they’re all simulations.
That said, it’s not always clear what kind of simulation is at work. It can easily appear to be one when in reality, it’s another. What claims to be predictive may really be speculative. What claims to be speculative may really just be descriptive. Just ask Oedipus. A prophecy that at first seemed like a speculative simulation turned out to be a predictive simulation and then, tragically, a descriptive one. Setting aside the power of fate–arguably–it was because he received the prophecy that it came true. The simulation made itself more true because he acted through it. This is an important general principle of recursion, which I’ll speak to in a moment.
That is, different kinds of simulation are not only affected by what they simulate in different ways, but they also in turn affect what they simulate as well. Simulations such as these are not simply representations, they are technologies with feedback built in.
This feedback is understandable through two very important principles for a general theory of simulations: these are reflexivity and recursion.
Recursive simulations are those that directly and automatically affect what they simulate. If there is a change in whatever reality is being simulated, this is automatically updated in the simulation. But if there’s also a change in the simulation, this automatically affects the simulated in the some way, whether or not anyone means for it to happen.
Donald MacKenzie famously described financial and economic models in this way, calling them “an engine not a camera.” The computational acceleration of financial simulations and their pervasive deployment makes these feedback loops only more intense and complex. Simulations of the future come to structure action in the present by determining what is most likely and giving it a price.
The simulation of the future comes true because it determines the present that later becomes the future. Insurance and prognosticative simulations of future risk determining what to allow or disallow in the present is but one example. Recursion can be direct or indirect. It can be a literal sensing, an actuation cycle, or an indirect negotiation of interpretation and response. The most nuances of these are reflexive. They mobilize action to fulfill or prevent a future that is implied by a simulation.
To be sure, Moore’s Law only accelerated the resolution of simulations, the intricacy of their reflexive and recursive feedback loops, and the dizzying complexity of their interaction with one another. The simulations are, after all, simulating each other, predicting what each of them will predict about each other's predictions, and so on.
I want to then turn our attention to a tour through several kinds of simulations, or several ways in which simulations are used, how each of us participates in them, and how each of us thinks and experiences the world through simulations.
I will begin with how some simulations allow us to know the reality of the universe.
Scientific simulations not only do more than deceive us, they are arguably the essential mechanism by which otherwise inconceivable underlying realities are made available to thought. From the very very small in the quantum realm to the very very large in the astro-cosmological realm, computational simulations are essential not just as a tool, but as a way of thinking with models—a fundament of induction, deduction, and abduction.
Far from hiding the reality from us, simulations are essential to how reality is disclosed at all. As Stephanie mentioned, Stanislaw Lem makes the distinction between instrumental and existential technologies. The former affect the world simply by what they do functionally. The latter, however, such as a telescope or a microscope, when used properly, not only let you see the world at different scales, they change how you understand how the world works. So all up and down the standard model of physics, clearly simulations, like computation itself, is not just an instrumental technology but an existential one. From planetary science to quantum field theory, from algebraic topology to astrophysics.
At the same time, simulations are based on models of reality, and the status of the model has been a preoccupying concern in the philosophy of science, even as simulations as such are more presumed than philosophized most usually. Models are a way of coalescing disparate bits of data into a composite structure whose whole gives shape to its parts, suggests their interactions, and general comparisons with other structures. They’re a tool to think with.
The value of these kinds of models is in their descriptive correspondence with reality. But this correspondence is determined by their predictive values, at least in the scientific context. If a scientific simulation can predict a phenomenon, its descriptive quality is implied. A model is also, by definition, a radical reduction in the variables, like a map reduces a territory. A geocentric or heliocentric model of the solar system, for example, can be constructed with styrofoam balls, but one is definitely “less wrong” than the other. Both, however, are infinitely less complex than what they model.
This is especially important when the simulated is as complex as the universe itself. Astrophysics is based almost entirely on computational simulations of phenomena that are produced by difficult to observe data, that are assembled into computationally expensive models, which ultimately provide for degrees of confident predictability about astronomic realities that situate us all.
For example, basically all the exoplanets that we know of, or planets outside our solar system, have been discovered since the fall of the Berlin Wall. As Anthropologist Lisa Messeri chronicled, the science of exoplanets is based on constructing model simulations from heterogeneous bits of data, and for the scientists, of understanding what exoplanets are like through imagining them as places, places which are mental simulations of planets many light years away.
This is what we call cosmology, the meta-model of all models in which humans and other intelligences conceive of their place in space-time. Today cosmology in the anthropological sense is achieved through cosmology in the computational sense.
Quite often, however, the simulation comes first. Its predictive ability may imply that there must be something we should look for because the model suggests it has to be there.
Thus the prediction makes the description possible as much as the other way around.
Such is the case, for example, with black holes, which were hypothesized and described mathematically long before they were detected let alone observed. For the design of the Black Hole in the Nolan brothers’ film “Interstellar”, scientific simulation software was used to give form to the mysterious entity based on consultation with Kip Thorne over at CalTech and others. The math had described the physics of black holes, and the math was used to create a computational model that was used to create a dynamic visualization of something no one had ever seen.
Of course, a few years later we did see one. The black hole at the center of the M87 galaxy was observed by the Event Horizon telescope and a team at Harvard that included Shep Doelman, Katie Bowman (seen here) and Peter Galison. And, well, it turns out the humans were right. Black holes look like what the math says they should look like. The simulation was a way of inferring what must be true, where to look, how to see it and, and only then did the terabytes of data from Event Horizon finally produce something we take as a picture.
It’s important, however, to note that this conjunction of entertainment purposes with scientific inquiry is by no means anomalous. It’s no secret that the general capacity for scientific simulation depends on the availability of cheap GPU’s. GPU’s are cheap because of the economics of video games, an entire entertainment genre based on interaction with computational simulations just for fun. Science is paid for by fun. AI is also driving the GPU development and cost curves and obviously will likewise cross both scientific and entertainment domains in ways yet unforeseen.
A few years ago David Krackauer from Santa Fe Institute co-authored a wonderful paper on the essential role of simulation across scientific disciplines and proposed an ideal simulation software stack for this very purpose. The paper offered nine motifs for simulation intelligence, a novel way of rethinking the boundaries between scientific disciplines by how they correspond with similar and different forms of simulation. What it didn’t address is that science is but one application for this software/hardware stack that is being driven by other purposes.
Let me emphasize, however, scientific simulation not only has deep epistemological value, it also makes possible the most profound existential reckonings. Climate science is born of the era of planetary computation. Without the planetary sensing mechanisms, satellites, surface and air sensors, ice core samples, all aggregated into models, and most importantly, the supercomputing simulations of climate past, present, and future, the scientific image of climate changes we know would not happen. The idea of the Anthropocene, and all that it means for how humans understand their agency, is an indirect accomplishment of computational simulations of planetary systems over time.
In turn, the relay from the idea of the Anthropocene to climate politics is also based on geopolitics of simulation. The implications of simulations of the year 2050 are dire, and so climate politics seeks to mobilize a planetary politics in reflexive response to those predicted implications. That politics is recursive. Deliberate actions are now taken to consciously prevent the future. As said, this is an extraordinary agency to give simulations. I doubt that Greta, or other climate activists, may like the idea but climate politics is one of the important ways in which massive computational simulations are driving how human societies understand and organize themselves. And indeed, they are why the activists are in the streets to begin with.
Part 1: Intelligence as Simulation / Simulation as Intelligence
That simulations relate so closely to descriptive and predictive knowledge of the world is perhaps not surprising given that cognition depends on descriptive and predictive mental models. Simulation is one of the most important ways that human brains work. Our friends from neuroscience and artificial intelligence may raise the point that simulation is how minds have intelligence at all. The cortical columns of animal brains are constantly predicting what will be perceived next, running through little simulations of the world and the immediate future, resolving them with new inputs and even competing with each other to organize perception and action.
Different versions of this general understanding largely converge on the point. Karl Friston’s free-energy principle depends on the reconciliation of anticipated and actual perception. Jeff Hawkins refers to the recursive feedback between mental models, new perceived novelties, and how they update background simulations we used to navigate environments. Computational neuroscience would agree that the basic process occurs not only at the relatively abstract level of cognition but even at the level of individual neurons. Andy Clarke and others explicitly invoke simulation as the modeling process that makes the noisy flux of perception into a coherent, predictable, phenomenological reality. Douglas Hofstater said “yes you are a loop” meaning this model-test-feeback cycle is not just something that intelligence does, it is how intelligence works.
The same goes for collective intelligence as well. Humans’ gigantic prefrontal cortex allows for and is the evolutionary outcome of linguistic and strategic cooperation. This allows us to share and communicate counterfactual, speculative, hypothetical scenarios and to coordinate actions to realize or prevent them. Everything from hunting in groups to building spaceships depends on projective simulation, at the individual and collective level.
At the end of the day–literally–we drift away from that coherent perceived reality into the deeply personal world of sleep. Arguably, dreams are another kind of essential simulation necessary for all animals. For humans, one hour of unconscious processing of reality for every two hours of waking life is necessary. Every night you lay prone, frozen, unconscious, lost in a virtual world where mental simulations dance with themselves. Perfectly normal.
Given that simulation is so essential to thought and to individual and collective intelligence, and that this intelligence manifests in the technological artificialization of the world, it is perhaps to be expected that we, eventually, would make artificialized intelligences based on this very same capacity.
AI is, among other things, simulated intelligence, which in the form of large language models, uses “language” not just in the conventional sense of chatbots, but also as the engine of difference that underlies applications in robotics, image processing, genetic sequencing, and so much more for example.
However, it is worth noting that the psychological and technological development of AI has also hinged on a kind of simulation anxiety. For the Turing Test, the standard for intelligence was based on the differentiation of human intelligence from computationally simulated human intelligence. To know if it is a simulation or not would come to one of the ways that the non-simulated (ostensibly us) is defined by the difference. Any uncertainty or confusion that muddies the split makes it less clear as to who we are.
This is perhaps a psychological version of model collapse—the process by which large AI models trained on the output of other models begin to fold in themselves and spit out nonsense, not unlike humans undergoing a self-recursive nervous breakdown.
Finally, this itself introduces a weird kind of simulation politics, whereby the goal is to redirect a model that governs the world by feeding it deliberately contrived data, thereby bending its perception of the real. Data poisoning is one way to do this, by making original human culture illegible or garbled, but so is flooding the model with fictionalized data that gives the AI a distorted sense of the world that is to the advantage of the actors.
If you want the big model simulation to think something is real or not real, organize potemkin data that makes it seem so. If you want the simulation to think there is a lot of traffic–walk across the street over and over and over. Show up at a favorite spot everyday if you want the simulation to think it is popular. This, I think, is the future of politics as gamified recursive simulation: people contesting the meta-model of society by doing the things they think will make the model correspond with the reality they prefer.
Part 2: Toy Worlds and Digital Twins
AI works not just by simulating us but by simulating the world. AI’s, such as driverless cars, are trained in what are called toy world simulations where they can explore more freely, bumping into walls, until they, like us, learn the best ways to perceive, model, and predict the real world.
Toy worlds are where some AI’s learn to navigate the real world by navigating focused, reductive simulations of its contours. But these are not always closed. For AIs, the boundary between a simulated world built of data and the real world perceivable as data, are not always clear. For those training AIs to support physical actions in the physical world, this fuzziness can be leveraged. In simulated worlds, time can be speeded up, multiple generations and iterations can spawn in an instant.
These toy worlds serve as a bounded domain of constrained information exchange and interaction between otherwise unlike and incompatible things and actions. The sim-to-real passage occurs not only in terms of specific learned expertise, but also through the virtual-physical hybridization of direct inputs and outputs, for example, AI’s interacting with blends of both real and virtual contexts and collaborators at the same time.
The AI’s world is a simulation of ours, but one that we can interact with. For the AIs, our world is part of the omnisimulation that it calls simply reality.
We have now moved further away from simulations as providing augmented cognitive access to physical reality and closer to simulation as constructed, fictional, synthetic experience. In some respects, the same principles of simulation hold. They’re built with the same software and for the same humans after all, but in other respects, their purpose, their relation to the world, to grounded knowledge, and to subjectivity and agency, are completely different; as is their relation to the real.
Virtual environments can refer to virtual and augmented reality, in which a user is immersed in a digitally constructed synthetic world, and also to physical built environments which are proxies for other real places, where one is a test site for the other. In turn, it’s not always so clear to everyone involved that they are even in a simulation. Sometimes some people know, but others don’t, and sometimes no one is really sure. Let me take these one at a time.
Part 2: Computational Virtual Environments
I would argue that VR is a particular kind of toy world more than the inverse. That is, toy worlds are not just for AI’s but for people too. VR is a kind of digital twin for experience itself. And as said, if scientific simulation allows access to ground truth through computational abstraction, then does VR free experience from ground truth?
Through VR and video games, simulations have become a mass-consumer content platform, providing immersive experiences for communication, gaming, exercise, and just sitting there zen-like as virtual whales swim overhead, as well as expert domains such as collaborative laboratory science and 4D data visualization.
In theory, this means new kinds of creative expressions and experiences. One can place an audience or user in a place to construct or demonstrate a world that can now place an audience in a body of a character, and doing so, the continuity of immersive space means they lose the ability to steer attention through cuts, shots, and angles. We ask, does narrative give way a form of storytelling that is more procedural, full of nested tasks and contingencies, instead of discrete events?
What then are the limits of the kinds of experience that can be virtualized, especially since experience is not exclusively human, but shared by many other forms of intelligent life?
And yet, even as it presents designed realities, VR may offer new insights into the real, informing philosophy of mind, the neuroscience of perception, perhaps also necessitating a refactoring of art history, as the physical relations between image and mind are reconceived and reconciled.
One question we might ask as a prompt is: when VR becomes more pervasive, what will we call this reality? What do we call non-virtual reality? Baseline reality? I think this is actually the best candidate, the latter, making the virtual the referent against which the real is defined.
How this leads us closer or further from the real is an open question and suggests different relations between illusion and reality, not unlike the difference between Plato’s cave and Wayang Kulit shadowplay. Let me give you a few examples of what I mean.
A project that some students spun out of my old lab at UC San Diego was a company called Nanome, which makes now very successful virtual reality software for advanced molecular visualization, modeling, simulation and design. Their customers include all the major drug discovery players as well as many synthetic biology companies. Here, the physical reality of protein binding to molecules is perceptually accessible through 3D models with new tactile dimension.
However, sometimes, as we well know, simulation is meant to be an illusion.
A corollary technology to VR is augmented reality, whereas people wear something like glasses that overlay their world with synthetic perceptions, programmed interactive hallucinations, that blend physical and virtual worlds into one. The shadow and cave merge.
By contrast, John Carpenter’s “They Live” presents a kind of reverse augmented reality. Here, the main character finds a batch of sunglasses that instead of adding a layer of ideological augmentation to reality, they instead remove and subtract the parts of a generally perceived reality that are illusions. For AR, the subject wears the tool to simulate reality, but here, they wear the tool to escape the simulation that is constructed reality: critical theory as Ray-Bans. In this case, they reveal that the world is actually run by alien lizard people who manage humans like cattle. That both AR and “They Live” scenarios in some ways correlate with symptoms of schizophrenia is a theme I’ll pick up in a moment.
VR has itself been represented in cinema in many ways. Perhaps, at least in my mind, the most compelling is in David Cronenberg’s “eXiztenZ”, where the virtual world looks exactly like the real world, except that impossible things happen all the time.
In many respects, the cinematic experience is already an immersive virtual environment as the last century of film theory has already suggested. More recently, the ante has been raised by things like The Sphere in Las Vegas, which posit not a first person interior VR, but VR as a medium for crowds, truly a mass ornament as Siegfried Kracauer once called it. To experience this costs about $500 per ticket for 2 hours, which is expensive and made more so by the fact that one has to endure a U2 concert for the full duration.
The mini-Sphere at your local mall is no less a virtual environment. Now that cinema has been completely absorbed by Marvel, this metagenre is defined not only by superhero plots, but also by how its cinematic form is constructed via layers and layers of photographic and computational inputs, composited together into a new reality that is both magical and oppressively familiar.
Part 2: Physical Virtual Environments
As said, the virtual environment may be physical, not digital. It may also be constructed, however, as an immersive illusion for its own sake, but in some ways, perhaps closer to the purposes of scientific simulation as a digital twin of another place or reality. A classic example being the massive simulation of the Bay Area watershed up north from us.
Keeping with the cave theme, consider Chauvet—a site in France discovered by accident quite recently that is full of incredible neolithic art dating to roughly 32,000 BCE. By comparison, Lascaux is dated to about 16,000 BCE, and so consider that Chauvet is as distant in history from Lascaux as Lascaux is from today. You may know Chauvet from Wernor Herzog’s film “Cave of Forgotten Dreams”. The art here is obviously priceless and fragile, and yet also part of human heritage. To allow visitors a chance to experience the art, a second replica of the original cave was constructed nearby that admits human visitors to admire its wonders. It’s not quite a theme park and not quite not one either. And in case the trip to France is inconvenient, the simulation has a simulation. You can visit the second cave virtually in a narrated VR tour. Shadows you see have shadows as well.
Physical simulations are sometimes constructed when one location is inaccessible from the other, but when there is some existential importance in modeling and testing and understanding how systems live and work at that remote site.
Space exploration has been a driving force in the design of simulations such as these, where a simulation on Earth is a proxy for a place out there, or a test out there is a proxy for something important down here.
You may be familiar with Biosphere 2, the ill-fated attempt to build a sealed virtual human society meant to simulate a future such society on another astronomic body. As NASA had shown, it's better to optimize a terrestrial simulation before sending humans out of the atmospheric espedermis. This simulation, however, was not run by NASA, but rather by a very 1970s theater collective led by deeply untrained charismatic leaders who did not account for the inevitable invasion of ants and cockroaches and ultimate virtual famine. Arguably, Biosphere 2 was less a failed simulation of a successful moon colony than a perfect simulation of how certain utopian ideas about closed loop microsocieties lead to implosion.
By the way, Biosphere 2 is now run by ASU. They hosted an art show recently where I was invited to submit a work. Half seriously, I provided the idea of a miniature Biosphere inside of the Biosphere, a simulation of the simulation. They were not amused, but the piece was included.
Part 3: Ontological Asymmetry
In these examples above all of the participants, one assumes, understood that they were in a simulation. They may not have spent a lot of time pondering the implications of this, but they knew at some conscious level that what they perceived is artificial in some way. This is not always the case. One of the most important social and epistemological dynamics of simulation is what I call ontological asymmetry: where there’s one fraction of participants who knows it's a simulation and another fraction who does not. Often the purpose of the simulation requires this asymmetry. Somebody has to think this is real, for the virtual experience of the other to go as planned.
The Orson Scott Card novel “Ender’s Game” is a classic example of this. The young space cadet thinks they are playing a battle simulator at which they have become impressively expert, only to learn that it’s not a sim at all. He had just blown up whole planets full of people, not knowing what he was doing. The theory was that if he had known it was not a sim, he would not have followed through and acted as desired.
Perhaps the goal of fooling the protagonist of the simulation is not always to coerce them to do something specific, but to put them in a frame of mind where their desires can wander naturally. Consider Butters–his mates on “South Park” convince him that they have let him use their new VR headset, which includes a perfect sim of their school. They have actually instead only installed upon him a snorkeling mask, which explains why he calls the graphics so realistic. This is an example of what we might call a placebo simulation. At least somebody thinks it's a sim, but it’s not.
One of cinema’s most famous examples of ontological asymmetry is of course “Truman Show,” where the main character’s entire life from birth to adulthood is a massively artificialized physical environment, with artificial friends and family who are in on the plot. Slowly but surely, he begins to suspect something is not quite right, and exhibits classic symptoms of a particular kind of schizophrenic episode, today called targeted individual syndrome, which ultimately leads the protagonist towards breaking the asymmetry and taking some control of the process of designing his life.
By this measure, the inverse of “Truman Show” is “Westworld”. Here the protagonists do know it's a sim, but all the background characters do not. The asymmetry is reversed. Eventually they do come to realize that the Toy World simulation is actually not a sim after all. Given that these characters are androids, this uneven realization of the real leads to less cut and dry existential lessons than “Truman Show.”
In both cases, however, the issue of consent is obviously relevant for such asymmetries. The Milgram experiments in acquiescence to authority are another famous simulation where ontological asymmetry caused psychological harm to those who didn't know it was a sim. They thought they had harmed someone when they had not.
Continuing deeper into the abnormal psychology of simulations, we observe a strong correlation between simulation and anxiety, one that takes different forms. I’m going to explore this a little bit more in a moment in relation to personal simulations, but first it also relates to the basic questions of knowing and not knowing, and the deliberate suspension of disbelief in, for example, therapy.
Among the primary uses of VR in the military besides training for battle or pre-enacting war is for PTSD, and indeed re-enacting the traumatic experience as a way of exorcising its torment. Reenactment of trauma is basic not only to Freudian psychoanalysis but truly to mourning rituals the world over.
By contrast, in the TV show “The Rehearsal,” anticipated trauma is dealt with by constructing full scale simulations of difficult conversations and encounters and having simulated protagonists pre-enact what it will be like, preemptively repeating the trauma before it happens, so that it might not happen at all.
At the end of this spectrum are the virtual worlds of “Wandavision” where the Scarlet Witch deals with the death of her family by using her magic powers to construct fully immersive environments, both physical and virtual at the same time, where she and they can live out their days in various sit-com scenarios. Here, it's less that simulation allows for the trauma to be repeated, rather, reversed. Reality is an egg unscrambled by the preferred artificialization.
Part 3: Political Simulationism
Shifting gears a little bit out of the dynamics of consensual and non-consensual illusion, this theory of simulations would also apply to questions of politics and governance, where the distance from consensual and non-consensual illusion is not always so vast.
Simulations play a big role in how states and other governing institutions imagine a role for everyone and everything. They give a sense of control through visual coherency and enclosure. It seems as though the complexity of the world simulated is accounted for because the elegance of its interfacial reduction is so convincing. Oftentimes this works just fine. You don’t always want or need a scientific precision in modeling social systems. It depends on what kind of simulation it is: descriptive, predictive, or projective. In many cases, what’s needed most is that the representation of coherency as an institutional rhetoric is successful.
Speaking of which, one of the most essential techniques of organizational futurism, especially during the Cold War, was and remains, scenario planning: a form of official, institutionalized reflexive simulation of potential political and military realities. Back in the day, as wielded by Rand Corp, Herman Kahn, Pierre Wack and others, scenarios were one of the key ways that governments, corporations, and militaries would model not the determinant future, but the contingent space of possible futures, rendered for executive consideration in the form of just-so science fiction stories often told with non-fiction rhetorics.
They could be called recursive also in the sense that the indeterminacy was based on the understanding that following the simulation to its logical conclusion itself causes the particular future to happen, rather than merely predicts it. In this sense, they are proactive and normative, implicitly recommending the manifestation or prevention of a given scenario.
This delicate contingency also provoked the curiosity of people like John Cage to link their esoteric interest in chance with the composition of artworks based on similar scenario techniques. And for people like Wack, this overlap between the technocrats and the mystics was quite strong.
The relation between politics and simulation is not merely one of futurism; it is also with the emergence of planetary computation, a real-time medium for the maintenance and gamesmanship of adversarial relations. As the world has grown increasingly interconnected, effective governance has become contingent on complex simulations of global systems.
These simulations serve not only as a tool for human sense-making, but as algorithms for creating policy, distributing resources, and managing trade-offs. Simultaneously, the deployment of earth observation satellites has enabled the collection of, for example, diverse, continuous data streams to power these simulations while remote sensing has made it possible to comprehend the Earth as a miniaturized planetary system. In doing so, they have also leveraged the potential of those same simulations for economic and geopolitical influence.
In the competition for accurate simulations, various counter-simulation techniques arise to distort and degrade the simulations of other actors. Behavior is hidden behind camouflage, sensors are jammed, decoys distract attention, while misinformation proliferates. While the inflections of these can be vastly different among various actors, the stakes are all ultimately about governance and sovereignty. Whoever is able to maintain accurate pictures of reality, and who can act on those pictures, and who can distort the simulations of other actors to their own advantage, maintains the position of prestige.
Part 3: Simulation Before and After Violence
There is also–must be said–no way to properly summarize the politics of simulation without also speaking of its role in political violence. Allow me to introduce Fort Irwin, in the desert east of Los Angeles, a simulated Middle Eastern city in which the US military trains for, among other things, simulated communication with confusing local merchants, and ultimately full-scale urban combat, complete with extremely loud and real explosions.
Other simulations are constructed, not in anticipation of violence, but after the fact. They seek to recreate, reconstruct, or reenact the violence, and do so for intricately different purposes.
Forensic Architecture, a group associated with Israeli, now British architect Eyal Weizman, deploys all the tools and tricks of architectural modeling and visualization to the reconstructed simulation of purported crimes of the state. The practice tries to strike a balance between art and law. Sometimes, the same work is submitted into the public record as part of a prosecution against the accused, hence the name forensic, and sometimes it is shown in prestigious galleries and museums. The ending of each of their case studies is always the same: the bad guys are guilty. This underscores the fluidity by which simulations slide between domains: not only are facts posed as fictions, but fictions are a way of deciding the real.
These two forms, scenario planning and forensic architecture, use narrative and projection in different ways. Scenario planning, we might say, is a narrative projection that becomes normative by its implementation. Forensic architecture, by comparison, is a descriptive simulation of causal processes that becomes normative through narrative and rhetorical advocacy.
However, the relation between simulation and political violence is not always so simple.
Consider two of, to my mind, the most extraordinary films of the past century: Gillo Pontocorvo’s “Battle of Algiers” of 1966, and Joshua Oppenheimer’s “The Act of Killing” of 2012. Perhaps a whole other lecture is due here to suffice, but recall that “The Battle of Algiers” was made just a few years after Algerian independence and a film that recounts the rise of the FLN as a revolutionary force and the French response. The film starred Saadi Yacef, one of the actual FLN leaders, playing himself. Street battle scenes were recreated and shot newsreel style often in the exact same location in which they had taken place. American releases of the film began with a disclaimer insisting that “not one foot” of newsreel or documentary footage was used. The film-as-model worked in at least two temporal directions. “Battle of Algiers” simulated the violence of the previous years, but also served as a model for future revolutionary action, and was even screened by the Pentagon after 9/11 as a way to understand insurgency in the Islamic world. Whereas Forensic Architecture's simulations are geared toward preventing future similar events–“Battle of Algiers” was more of a hyperstitional template–though history has a way of reversing these.
We began in Indonesia and we return there now momentarily.
“The Act of Killing” also recounts post-colonial revolutionary violence from the Cold War era, but this time from the right. It’s a surreal, chilling retelling of the mass killings in Indonesia of upwards of a million suspected communists that took place during the same period in which Battle of Algiers was shot and released. And instead of simulating the events ex post facto as a way to celebrate the bravery of the fighters or to identify the guilty parties, this film casts the now aged perpetrators in the role of their younger selves, perhaps a bit like Saadi Yacef, within elaborately theatrical recreations of their own murderous acts. Here the simulation is closer to the repetition of trauma, but in this case that trauma is also itself asymmetrical. For many of the perpetrators who are all too humanly oblivious to the significance of their acts, and to the victims and their families who are tormented by memories, the simulation works, sometimes, to reconcile the meaning of the violence.
Part 3: Back to the Shadow: Person
We move now from the past to present, from the filmic simulation to the computational, and from collective historical simulation to the individual and the personal.
The formations of digital identity under planetary-scale computation are aligned mostly toward the individual human person as the base unit of sensing, analysis, and recursion. Each of us, we confront the digital shadows that we all produce and are produced by, in which we both possess and are possessed by. These shadows emerge through processes of doubling and tripling, reflection, opacity, transparency, individuation, massification, desire, control, erasing and staging and framing and deframing all at once. We have no choice really but to approach them with trepidation and fascination and curiosity, dismay and satire all at once.
That is to say, the politics of simulation can also be very personal. As you pass through the security gateway, perhaps at an airport, what is under inspection is not only your physical person, but also trace digital personas linked to you but which live in a near-distant shadow city called the Cloud. If the man in the uniform lets you pass, it’s because a decision was made according to risk models on those silhouettes of which your physical person is a reflection. Your ears may burn as the infrastructure whispers about your doubles, but it’s not just you in play.
The profile as shadow is a product bought and sold. Consider the very idea of celebrity as already a kind of simulated person, an artificialized persona that can split between the original person and take on a life of its own. This can cause psychological stress, and this also, however, can be monetized.
In 1971, Stanislaw Lem’s “The Futurological Congress” imagined a future in which Hollywood films would be populated solely with licensed synthetic doubles of actors. In 1974, Rem Koolhaas and Rene Daalder wrote an unfilmed screenplay called “Hollywood Tower” that was based on the same premise. In 1975, Lou Reed was touring Australia, deeply annoyed with his interviewer: “Since you were here last time, you seem to have been working full time doing concerts all over” “It was a lie. There are five of me going out, just like The Drifters in the old days.” “You think there are?” “I know there are. Two of them are out there. We’ve been mutating. Genetic damage.”
More recently, Moore's Law has allied Lou Reed’s prophecy of full celebrity simulation to become somewhat more advanced. Consider the geriatric rock band KISS–who have officially finally now retired–but recently announced that their digital avatars will be going on tour in their stead. Or the perennial French presidential candidate Jean-Luc Melenchon, who mildly impressed the crowd by appearing in person in Lyon while also as a Tupac-style hologram in Paris at the same time. Obviously the actors and writers strike most recently put Hollywood on pause while an entire industry organized around the mass production of digital personas, wrestled with the macroeconomic shift in labor power from those who create training data to those who create models.
Everywhere, however, there are ghost cities populated by legions of mimetic personas crackling within our nearby data centers, even as the surrounding landscapes are largely unpopulated by human beings. Any such may be home to hundreds of millions of shadows, but only a few dozen workers. The ratio is, I think, a sign of things to come. It’s a posthuman urbanism in practice, but not in theory. That is, even as shadow cities came to be the predominant urban form of the last decades, their progress largely went ignored in architectural schools in the first decades of the century when increasingly fantastic accommodations for human clients took precedence. This does not mean, however, that shadow cities are actually virtual. To the contrary, the sprawling distribution of factories, ports, container sorting centers, freight airports, as well as the networks of thirsty data centers comprises a discontinuous megacity for objects and shadows. To imagine it as numinous is an illusion.
There is “a there there,” but this “there” is not right here. When you and I chat and post to one another, I am in one place, you in another. We may even be in the same city at the same time, but the conversational point of contact between my persona and yours is, literally and physically, located in a shadow city where neither of us live. To converse, we draw upon shadows and speak to one another through them as masks, carving links between human zones and shadow cities. We sew threads between one another and between places, and in doing so contribute more texture to the model simulations that mediate these circuits: sprawling and interlocking and incommensurate.
Conclusion: Simulation Anxiety
So I will conclude here with one final theme. This talk, as mentioned, was as an invitation towards a general theory, but wouldn’t be complete without a discussion of simulation anxiety, a term that can mean many different things.
First, it is anxiety about whether or not something is or isn’t a simulation. Philip K. Dick characters are archetypes of this, but their reality-questioning angst is not the only form this can take. Ultimately, in societies where simulations are foundational, such as ours, for political, economic, logistical, ideological and identitarian formation, anxiety as such can take the form of simulation anxiety.
Here, instead of simulation healing trauma through staged repetition, the uncertain relationship between the stage and the staged induces, if not trauma, then at least anxiousness. This may in turn need to be dealt with by future simulations, about which there is also some uncertainty, and so on and so on.
For example, the art world’s financialization of affect has long been concerned with identifying original from forgery. So much so that the forgery as satire became an established format in and of itself from Duchamp, to Warhol, to Richard Prince, to Thomas Kinkaide. Sometimes its collectors are worried about investment, and sometimes its consumers are worried about authenticity. Consider the streets of Santa Fe, New Mexico where Native American artisans, mostly Navajo, sell beautiful jewelry, some of which is made in a factory that used to mass produce hand crafted knockoff objects. Today, however, that factory has Navajo owners. Are its products authentic or not? Who’s to say? But what value does the object have if it is not an embodied unadulterated authenticity? And if so, for whom?
What about the authenticity of simulated humans? In the 1920’s, Makato Nishimura was so distressed by the human simulations that he had seen in Capek’s play “R.U.R,” that he designed Gakutensoku, a humanoid robot made without metals that represented not the degraded, machinic qualities of the body, but its higher spiritual capacities. It could open and close its eyes, move its head, at least before it was lost for good while touring Germany in the 1930’s.
Compare this anxiety, however, over the mis-simulation of the human with that of Turing’s Test, where the very status of the human would come to be defined by all the ways, ever shrinking, that it is different from, not similar to, its artificialized other. For Gotutensaku, anxiety seeks resolution and union between the human and its shadow whereas for the Turing Test, it is to keep them forever bordered.
From rubber sex dolls accosted by the lonely to Paro the robotic seal given to elderly dementia patients in Japan, sometimes anxiety is not whether or not we can tell it’s a simulation, but whether or not those using them really know or really care enough if it’s a simulation or not. Does the dementia patient know that it’s not a real seal that seems to love them? Do they know and not care? Do they care and not know?
As we all interact more and more with traces of our simulations of ourselves and our digital shadows and our replica personas, does this itself make our simulation anxiety more acute? The more we are asked to verify our originality, the more our existential status is put into question.
As for simulation politics, the gaming of the model by performing what you want it to think, is a kind of Hawthorne Effect ontology. One that has become so normalized that it expands the potential space of simulation to the edge of reality itself, engendering a new kind of deja vu effect, called ‘glitch in the matrix’ where some kind of anomalous regularity suggests that the big operating system in the sky may have copy/pasted the NPC’s too much and, like Truman’s revelation, given the whole game away.
It’s hard to keep it all straight. As for simulation anxieties like, for example, the Moon Landing, seen here as a simulation of a simulation in the film “Capricorn One,” in which the Moon is played by Mars, and Neil Armstrong by O.J. Simpson, we should remember that as the most watched television show broadcast in history up to that point, the moon landing was shown on the largest American network, CBS News, as a simulation. It said so right on the screen, but as all those people’s memories of the landing, and of the broadcast, started to fade over the years, a kind of inside out Mandela effect may have taken hold, people half remembering the simulation they really did see as a simulation they didn’t see.
I hope that I have provided some good training data. I will offer just a few quick closing thoughts before we depart for the lobby.
If our era is one in fact defined by simulations, one piece of evidence for this may be the fact that we use simulations for so many different important things, and yet simulations as such are not so widely recognized and acknowledged and discussed, but rather taken very much for granted. The fish does not bother to theorize water. But if we were to take them more seriously and be more reflexive about how we used reflexive simulations, this would, as I hope is clear, mean that we would be thinking about how we are thinking, which may be the basis of some deliberate reorientation of our own purpose.
This would touch on many things but would have to include, among many other things, the difference between reflexive and recursive simulations, the dramas of ontological asymmetry, placebo simulations, the divergence of scientifically deduced reality and computationally generated illusions, the relation between model and intelligence, modeling as intelligence, intelligence as predictive modeling, and, of course, the addictive microtraumas of simulation anxiety.
Lastly, as Stephanie had mentioned, one of the key theses of our program is that at certain points in time, our technology is ahead of our ideas. At other points in time, our ideas are ahead of our technology. When our ideas are ahead of our technology, when we want to be able to do things that we cannot do, utopians and various avant-gardes make the plans. At other times, when technology has outpaced our ideas, when technological capacity is way ahead of our theory, this suggests a different project for philosophy. Rather than projecting its truisms onto technology, it suggests instead a direct encounter with, in this case, simulation, and for our program more generally, with computation that would produce new concepts needed to not only orient the model but ourselves. This is why simulation–a critical practice without a sufficient theory–needs our attention.