📚 Finished reading The Age of Surveillance Capitalism by Shoshana Zuboff.
I’ve had this book on my top priority reading list ever since it came out in 2018. It still somehow took me five years to get to it. But I’m glad I did - I wasn’t at all disappointed with my belated reading of it. I’ve come away thinking that something along these lines should be essential reading for almost everyone who has to navigate today’s technological society.
This particular book is rather wordy and academic-sounding. Its style might not be accessible or interesting to everyone trying to make their way through today’s chaotic and busy world. There’s plenty of room for someone to bring out a shorter and simpler version. Or to communicate its message via a different media. The Netflix documentary on Facebook - “The Social Dilemma” - contained a small subset of similar ideas for instance and was surely a lot more accessible to a lot more people. But also, necessarily, that much shallower. This book is for anyone who is interested in digging deeper into what’s happening, why, and why it matters.
In any case, I feel that the ideas within this book very important to understand if you want to know what’s being done to you and why, each time you use the internet. Or, for that matter, an increasing amount of the time when you aren’t aware that you’re anywhere near the internet.
To be honest, I think you’d get a decent summary from reading just the introduction if the full thing is too heavy going.
As a disclaimer, I’ll note that the content plays heavily into my pre-existing biases as to the perils of big tech, particularly the mainstream social media companies. So perhaps some of my enthusiasm comes from the comfortable feeling one gets when reading something you already heavily agree with in many ways. But whilst one might fairly accuse it of over-hyping the current impact of surveillance capitalism I do think many of the basic ideas, the theories behind the system, are fairly indisputable and critical to document and publicise. It’s important to know what the surveillance capitalists want to do to us and why - even if you don’t believe they’re actually all that effective at doing it just yet. They’re unlikely to get any worse at what they do over time, unless we intervene.
Anyway, the general premise is that, as we know capitalism changes and adapts over time. The current version we’re heavily enmeshed within can be described as the titular “surveillance capitalism”.
Living in an the era of surveillance capitalism, the digital realm has permeated very heavily into our lives. It has redefined how things work and what things are acceptable far faster than we’ve had a chance to think about whether these changes are desirable or not. Famously, the big tech companies like to “move fast and break things”.
This is of course a problem if the “things” they are breaking include you as an individual and society as a whole. Without a swift re-evaluation and consequent serious action, Zuboff claims that our emerging “information civilisation” poses a substantial threat to some of our most fundamental rights.
The mechanism by which this plays out can be swiftly summarised via this quote:
…our lives are unilaterally rendered as data, expropriated, and repurposed in new forms of social control, all of it in the service of others’ interests and in the absence of our awareness or means of combat.
Digging deeper here - but not nearly as deep as the book does - the underlying philosophy starts with the claim by surveillance capitalists that they have the right to surveil and ingest our human experience in service of converting it into behavioural data.
Some of this data might be used to improve the products they sell. This is perhaps fair enough, maybe even a net positive for us, and is how things used to work. But ever since Google discovered how profitable this data can be in the marketplace, the rest of our surveilled and datafied human experience is translated into a “behavioural surplus”, to be used exclusively in the interests of the company who snaffled it up in the first place.
The companies concerned then use machine intelligence processes to manufacture a “prediction product” from our behavioural data. The prediction product aims, on the basis of knowing what you’re doing, thinking or feeling right now, to be able to predict what you’ll do in the future.
These prediction products are traded in an entirely separate and new marketplace - the “behavioural futures market”. Selling this information about what we’ve done, what we’re doing and what we’ll do in the future is extremely profitable for the companies that extracted it.
The author reports that it was Google that first discovered the economic value in this. Soon afterwards Facebook, Amazon, Microsoft, and the rest of big tech followed. It’s now become the default way that companies offering online services are funded now, everything from tiny start ups to mega corporations. Consequently, if we want to lead an effective life, we often have no reasonable alternative but to participate, to allow this to be done to us.
The accuracy of the predictions the companies make about us is what makes them valuable. Increased accuracy means increased profits, for which the drive is, naturally under capitalism, insatiable. Thus these companies feel compelled to acquire and infer more and more data about us, which now includes our voices, our emotions and our personalities.
Harrowing as that might be, it’s not where the danger ends. There was a realisation that the most accurate predictive behavioural data is that which is created when the organisations concerned directly alter our behaviour. If they intervene, nudging us to make decisions in line with their interests, then their predictions become more accurate and hence more profitable. Think of how the fun Pokémon Go game ended up sending us to locations that just happened to correspond to restaurants that paid them or how Facebook expropriates a huge amount of data from us and our friends in order to allow its advertisers to more effectively influence us to buy their products.
In doing this, these companies are exercising “instrumentarian power”; a power that knows and shapes human behaviour towards other people’s goals. The end goal is effectively to automate us. Whilst an earlier time’s industrial society was imagined as a well-functioning machine, instrumentarian society can be imagined as a human simulation of a machine learning system.
Instead of the typical assurances that machines can be designed to be more like human beings and therefore less threatening, Schmidt and Thrun argue just the opposite: it is necessary for people to become more machine-like.
The original digital dream - that being digitally connected is intrinsically pro-social, inclusive and tends towards the democratisation of knowledge is dead. Capitalism no longer feeds solely on human labour, but has expanded to feed on every aspect of the human experience.
How was this allowed to happen? Basically because few people understood what was going on. “Cyberspace”, where it all began, was an uncharted and fairly lawless territory. Google et al. carried out deliberately secretive and misleading activities that were anyway so new, complex and illegible to the vast majority of us that even if we’d somehow known what they were up to we wouldn’t necessarily have realised its implications.
And all this was sold to us as a freedom, as an emancipation. Who wouldn’t want free access to the entire world’s information? Who doesn’t desire to connect to their friends upon demand at the click of a button? Who would hate having the ability to publicise their thoughts, worries and concerns to the world at large? Who doesn’t want a digital assistant or convenient online tools to help them get through the struggles and torments of modern life?
At last, herald true personalisation is here to soothe us. Someone - well, something - that recognises us for who were are, acknowledges our interests, never says no, is always available.
But, inevitably, behind the scenes, solving these issues is not the primary purpose that these products were created.
Surveillance capitalists quickly realized that they could do anything they wanted, and they did. They dressed in the fashions of advocacy and emancipation, appealing to and exploiting contemporary anxieties, while the real action was hidden offstage.
To Facebook et al. we are not customers. There’s no economic exchange, price or profit directly taking place between us and Google when we do a web search. Contra to a commonly held idea, we are not even the product. We are merely the source of the raw material that the surveillance capitalists need, the behavioural surplus. The real customers of these companies are the organisations that buy our data from them in the behavioural futures markets.
…individuals are definitively cast as the means to others’ market ends.
These techniques also came to the fore in an era of a neoliberal ideology that resisted the idea of putting controls on businesses. Furthermore, the 9/11 terrorist attack in the US also made the state and its population prioritise the benefits of surveillance over its drawbacks. After all, outside of specific examples like China, most governments often do not have the resources or skills to carry out mass sophisticated surveillance operations of the kind done by the private surveillance capitalists. Instead they’re reliant on, and jealous of, big tech. The US government alternates its time between funding Google and begging for bits of their data.
There’s a tremendous asymmetry in knowledge and power here. Surveillance capitalists know everything about us but deliberately use methods that mean we cannot know very much about them. Our rights to privacy are not respected. But they’re not destroyed; just redistributed. Surveillance capitalism claims the right to privacy whilst depriving us of ours.
At the end of the day:
Two men at Google who do not enjoy the legitimacy of the vote, democratic oversight, or the demands of shareholder governance exercise control over the organization and presentation of the world’s information.
One man at Facebook who does not enjoy the legitimacy of the vote, democratic oversight, or the demands of shareholder governance exercises control over an increasingly universal means of social connection along with the information concealed in its networks.
The dynamics of power under industrial capitalism are shifted from past eras.
…ownership of the new means of behavioral modification eclipses ownership of the means of production as the fountainhead of capitalist wealth and power in the twenty-first century.
As are the fundamental risks to our very existence as we know it:
Just as industrial civilization flourished at the expense of nature and now threatens to cost us the Earth, an information civilization shaped by surveillance capitalism and its new instrumentarian power will thrive at the expense of human nature and will threaten to cost us our humanity.
Not of course that surveillance capitalism doesn’t also flourish at the expense of nature. The incredible power consumption of, for instance, modern day artificial intelligence makes that clear.
This “seventh extinction” will not be of nature but of what has been held most precious in human nature: the will to will, the sanctity of the individual, the ties of intimacy, the sociality that binds us together in promises, and the trust they breed.
Nothing about this was inevitable. The technology didn’t create the system. The problem comes from the very logic of surveillance capitalism, which directs where the technology goes. The nefarious processes we’re talking about were deliberately created to promote the commercial ends of the companies that foist them upon us.
Technology can never be isolated from economics and society; they’re always economic means to an end, not ends in themselves.
Likewise these attempts to surveil and automate us are not to be explained by individual “bad people” or a conspiracy. Firing Zuckerberg, even if it wasn’t impossible, wouldn’t fix the system. The actions taken by surveillance capitalists and the resulting harms are an obvious and inevitable consequence of the logic of accumulation under surveillance capitalism.
It wasn’t so long ago that US society was appalled at the idea of mass behaviour modification techniques. Admittedly they were generally envisaged as being carried out by the state. There haven’t been an equivalent reaction in more modern times to the same practices being used by private companies who simply seek to get ever more rich, ever more powerful.
This remains true, despite the intent of these companies to deprive us of various rights we previously claimed as inviolable. These include essentially any that require our individual autonomy and agency to be respected. Zuboff terms these as being challenges to:
- the sanctity of the individual.
- the right to individual sovereignty.
- the right to the future tense.
- the right to sanctuary.
The loss of these rights mean democracy is also under threat. Democratic engagement requires individuals to be able to exercise autonomous moral judgement and self-determination. If we lose the latter then we lose the former.
There are at least 3 fundamental differences between surveillance capitalism and the capitalisms that came before.
-
Surveillance capitalists insist that they, and only they, should have unfettered freedom and knowledge. The theory of capitalism was built by its theorists on the back of 2 assumptions: firstly that markets are unknowable, and secondly that it’s this unsurmountable ignorance that means market actors must be given freedom of action. This is Adam Smith’s invisible hand at play - the butcher cannot possibly know everything about every potential customer and their relationship to the market and society. But by having the freedom to work in their own interest they end up contributing, unknowingly, to the efficient allocation of resources; a positive result for society. This fundamental assumption breaks down as one side of any transaction - the Google side - gains ever more perfect knowledge but refuses to give up any of its freedoms.
-
Traditionally capitalism has involved reciprocities. The author seems to me perhaps a little forgiving of prior power dynamics at times, although they mention the shareholder value movement as partly disrupting this. But it’s true that ideas around reciprocation were fundamental to the origin of capitalism, such as Adam Smith’s argument that price increases must be balanced with wage increases. However, today’s hyperscale surveillance capitalists don’t need or want to feel any obligation around reciprocities. As individuals, we are neither their customers, nor do they rely on very many of us as workers. These companies tend to have relatively small numbers of staff, all of which are drawn from very exclusive strata of population.
-
Historically capitalism has been greatly in favour of an individualistic vision of the world, insisting on sanctity of individual liberties - often to a problematic level in my view. Traditional capitalism’s vision of society is a set of individual agents freely transacting according to their will. In particular it loathed the idea of a collectivist society; one where planning and control is used to produce a result previously ordained by someone or something, which would necessarily come at the expense of an individual’s freedom to do exactly as they choose. However, surveillance capitalism’s desire is in fact for a collectivist society, one that they organise, that’s “radically indifferent” to our own personal interests and predilections. It’s a world where each of us is tuned and nudged to behave in line with previously ordained plans - plans that are solely in the interest of the surveillance capitalists.
The wealth of these companies means they can plough vast resources into recruiting the finest minds of our times and investing in infrastructure at levels that one would have thought would be more appropriate to efforts to solving world hunger and the in-progress environmental catastrophe. But no, in the real world, the most incredible rewards are given to the few people who have the privilege, education and ability to use of this data to make us…click on more adverts.
So, for those of us who aren’t keen on being surveilled and subconsciously coerced to do things we otherwise wouldn’t have done, what can we do?
Opting out seems the obvious thing at a glance. Don’t use services that surveil and seek to modify your behaviour. However, in reality, that’s nigh on impossible to do without dramatically affecting our lives. The internet is truly rife with this technology and using the internet is fairly critical for leading an effective life these days. And that’s before we get to all the surveillance architecture that’s hidden in physical public spaces.
The prediction imperative transforms the things that we have into things that have us in order that it might render the range and richness of our world, our homes, and our bodies as behaving objects for its calculations and fabrications on the path to profit.
…
There was a time when you searched Google, but now Google searches you.
This tension between a sense of knowing surveillance capitalism should be resisted but finding it almost impossible to do so if we want to have a “normal” life tends to result in us becoming cynical, resigned, and just learning to put up with it.
We can of course attempt to hide. I myself use a wide range of tools that aim to preserve a certain degree of privacy on the internet - tracker blockers, ad blockers, VPNs, special browsers, encryption, all that kind of stuff. A surprisingly high (to me) number of people also take these precautions.
There also exist physical items of relevance, such as clothes that aim to interfere with the ability of outdoor cameras to apply facial recognition to track where you go, what you do.
But these whilst these solutions, or “counter-declarations” can help some individual people with their individual situations, they unfairly and impracticably put the onus to act on us as individuals. These efforts can sometimes can feel as exhausting to do as surveillance capitalism is to endure. Instead we need action at a society level.
…the individual alone cannot shoulder the burden of justice, any more than an individual worker in the first years of the twentieth century could bear the burden of fighting for fair wages and working conditions. Those twentieth-century challenges required collective action, and so do our own.
These actions would include changes in public opinion leading to legislation and jurisprudence.
There’s already been attempts to use monopoly or privacy legislation to address some of the harms. Whilst acknowledging that these are important facets of the issue, the author feels these efforts miss the primary point. The real harm here originates from the practice of the “rendering of our lives as data” in order to increase the control others have over us in the first place. What they argue for is legislation but far more widely scoped and interpreted. Something that’s a much more fundamental challenge to what surveillance capitalists do.
Zuboff refers to this as being a ‘synthetic declaration’, meaning something that imposes a new framework, redefining the facts of the matter, spelling out what we want our future to be. Likely components would include:
- strengthening our democratic institutions.
- constructing a double movement.
- harnessing digital technologies in ways that help us to live an effective life and maintain a democratic social order.
GDPR might one day be seen as green shoots to move in this direction with its requirement that companies justify their data activities. But at the time of the book being written it remained to be seen what its real impact would be. Notably, Zuboff argues that this would be dependent how it’s interpreted by society and the law, which is more important than the specific text of the legislation.
Now it’s been place for a few years and, hey, surveillance capitalism still does what it does, flourishing to an astonishing degree, albeit with slightly reduced rights (that it’s battling to regain) in Europe.
At the end of the day, Zuboff concludes that:
It is not OK to have to hide in your own life; it is not normal. … It is not OK for every move, emotion, utterance, and desire to be catalogued, manipulated, and then used to surreptitiously herd us through the future tense for the sake of someone else’s profit …
…it is up to us to use our knowledge, to regain our bearings, to stir others to do the same, and to found a new beginning
