8m read

From situational awareness to synthetic realities

8m read

Technological developments such as connected sensors and computer vision are opening up new ways to gain insight about the world. But traditional approaches to business intelligence – all those spreadsheets and dashboards – are not equipped to handle today’s high dimensional data.

As we go about our daily lives, we’re constantly constructing and calling upon mental models of the world around us that help us to navigate, weigh up our options and make decisions.

“Do I have enough time to overtake?”
“How is this person going to react?”
“If I vote for this candidate, will things be better or worse for my community?”

But as the decisions we need to make become more complicated, digital models provide more nuanced, more accurate and faster solutions than our own mental models ever could. This is why software based models, like spreadsheets have become so prevalent in the worlds of business and engineering. These approximations of the world – these simulations – allow us to test ideas, experiment and make assumptions about the future, without having to commit resources or break things.

A simulation is an approximate imitation of the operation of a process or system; that represents its operation over time.

As more of what we do becomes digitised, new inputs become available to feed into these models. In turn, as more of the choices that we make rely on simulations, their outputs and recommendations become increasingly significant.

Right now, the stakes have never been higher. We are facing a climate crisis, a viral pandemic, rising levels of inequality, a regime of surveillance capitalism and a polarising environment of disinformation. All the while, technology is moving us towards a world that is increasingly automated and virtualized – driven by data and mediated by machines. The convergence of these forces will bring about big shifts for individuals, businesses and society.

There is an imperative for change and decisions need to be made. Simulation technologies are potentially a powerful tool for helping to make the right ones, but just as easily for making things worse. We already live in an age where policy making – and peoples’ lives – are being shaped by misunderstood ‘mutant algorithms’. If we want agency in this new world, we need ways for humans to understand it and to maintain oversight, governance and control.

What is Simulation Intelligence?

When we think about simulations, they could be purely mathematical, for example the performance of a financial portfolio under different conditions, or purely visual, like rays of light illuminating a CG scene. But they are most powerful when they are both - with visuals and maths working in tandem to reveal something that was otherwise imperceivable.

Simulation Intelligence is an emerging set of strategies for designing simulations that seek to improve people’s ability to understand and apply them to real world applications.

Design, in this context, should be thought about in its broadest terms - not just what the simulation looks like, but how it functions, how you interact with it, the information that it conveys, the weighting that is given to different parameters and how easy it is to interrogate. Simulation Intelligence can be seen as a stack made up of five connected layers - Strategy, Data, Logic, Communication and Interaction.

The Simulation Intelligence stack

All of these considerations are essential for developing applications that fulfil their purpose - that are not only useful, but used. It makes simulation intelligence a multi-disciplinary undertaking, one that draws on a number of different skill sets:

Strategists
Scanning the horizon, identifying opportunities, defining problem spaces and developing new value propositions.

Data Scientists
Building the models that adopt the right levels of abstraction and complexity needed to capture the dynamics of the system.

Data Designers
Developing the visual frameworks and metaphors needed to bridge the gap between data, meaning and value.

Game Designers
Modelling the topography, physics and mechanics of simulated environments.

Interaction Designers
Defining how people will navigate, explore and manipulate these spaces, empowering them to act on the insights they gain.

Software Engineers
Bringing everything together in robust, seamless digital experiences, from wearables to entire control rooms.

Use cases for Simulation Intelligence

Optimising the ways that people observe, analyse and predict the physical world, is at the core of the Simulation Intelligence approach. As everything around us becomes more connected, more complex and more automated, these capabilities will become critical.

We will see this in all industry sectors - everything from energy to agriculture, from logistics to healthcare. The specific requirements of each will vary, but there are some common use cases that cut across them.

Use cases for Simulation Intelligence

Situational Awareness

Ever since the second world war and Churchill’s smoky bunker, we have long been familiar with the idea of ‘seeing rooms’ - immersive data spaces that centralise all available information in support of tactical decision making. The concept of ‘big board’ has featured heavily in cinema, from Dr Strangelove to Avatar, advancing in technical prowess with each generation. Today, with a world of data at our fingertips, these spaces are as relevant as ever, but the ability to know what is happening right now is no longer confined to state-level players. With a pocket sized device, you can know the value of a company, the location of a ship, or your own blood oxygen level in real time.

In recent years, the term ‘digital twin’ has become a commonly accepted way of describing these representations. Predominantly used in the context of manufacturing and engineering, the idea is spreading to encompass entire enterprises, economies or even oceans.

The traditional concept of a digital twin often incorporates a one-to-one, ‘as-built’ 3D facsimile of a real-world counterpart. As people become more familiar with the idea, we will see a broader spectrum of design patterns emerge, that consider a variety of visualisation and interaction approaches.

The challenge is not just to create a copy of what already exists, but to help people focus on what is most critical for them in the moment. It’s about providing the right levels of granularity, context and abstraction for the task at hand - even as a situation changes. When we see these simulations of the here and now, we want to know not only what just happened, but the reasons why, how things are connected and what is likely to happen next.


Predictive modelling

Many of the approaches to predicting what might happen using simulation modelling have been around for decades, but it is only relatively recently that access to data and the necessary compute power have become more widely available.

A central component of these techniques is the concept of abstraction - of reducing down the almost infinite complexity of the real world to a manageable number of parameters that will give a workable approximation of the dynamics of a system. Depending on the specifics of the problem at hand, different levels of abstraction and varying modelling approaches can be used; System dynamics is typically used for higher level, strategic modelling. Discrete event simulation is widely used in manufacturing and service industries to describe processes. Agent based modelling describes systems through the behaviour of individual agents - from cars in a traffic simulation to individual cells in an in-silico pharmaceutical trial.

There are already many tools available that help people to build simulation models, but often these are highly technical applications, impenetrable to non-experts, who end up settling for a simple, chart based output. Because of the abstract nature of the process, it is also all too easy for the people developing these models to confine themselves to that abstraction, to lose sight of the wider context of the system they are modelling or to consider the second or third order effects of different scenarios.

Simulation Intelligence encourages a different approach. By anchoring predictive modelling to strategic intent and considering the importance of communication and interaction to their effectiveness, it aims to increase both the usability and understanding of these powerful tools. As the types of decisions and predictions being made using simulations become more profound, the need for transparency, traceability and fairness becomes more critical. This is even more important as we increasingly delegate authority to entirely virtual, automated entities.


Synthetic realities

Today, we can see many examples of entirely digital systems that are capable of generating financial and cultural value in their own right. Live concerts held within Fortnite are drawing audiences of millions, machines within manufacturing plants are collaborating autonomously and over 80% of the daily moves in US stocks now initiated by algorithmic systems.

Companies like Fetch.ai and Spherity are developing decentralised digital infrastructure, in which economic activity takes place between autonomous agents - bundles of data and logic diligently optimising for the goals they’ve been set. These are systems that bridge humans, objects, machines and algorithms, opening up entirely new forms of commerce. On the front end, Nvidia’s recent announcement of their Omniverse platform points to the levels of visual fidelity that will be possible in these environments.

Already, some of the most valuable firms in the world generate huge revenues from data generated by billions of individuals. They have built knowledge graphs that reflect and predict the shifting ontologies of the world. As the number of things that are online grows exponentially, there will be an even greater opportunity to extract value.

In order to understand these ecosystems, we will need to be able to perceive them. We will need new maps for this new terrain.

Just as today's digital twins help us understand the interplay between real-world assets, we will see synthetic environments emerge that provide a way to see and control intangible assets built of nothing more than data and code. These spaces, by definition, will be highly abstract. Unlike digital twins, they have no physical counterpart, nothing to make a facsimile of. In the same way that piles of folders and files provided metaphors for PC interfaces, it is likely that these new synthetic realities will borrow heavily from the physical world. Designing these systems and the way that people interface with them will bring challenges and opportunity for the product leaders of tomorrow.