Automation, the moving from manual tools to machines that replicate the actions of a human using a tool, appears in various forms within everyday financial life. The ATM, for instance, is an automated version of the bank teller of old who, with ledger book and quill pen, would have to exert energy to check your account, hand you cash, and alter your accounts. While financial interactions used to rely on human energy and mental faculty, today's digital interactions are very different. I use an interface to interact with this ATM, which gives me some form of control, but only within the inflexible rules of whatever it will allow me to do. This actually requires energy on my part, so while the machine seems to 'do things for me', the process is also 'self-service'.
Automation of this slippery sort is taking on more and more of personal finance. The glossy adverts of the financial marketing industry put an appealing spin on the future world of contactless payment, branchless banking and cashless society. They focus the mind on problems that are apparently being solved through new technology, but they simultaneously divert attention from the dark side of the automated financial regimes that is emerging around us.
Corporate management is fond of automation because it is a force for scale, standardisation and efficiency – and in turn lowers costs, leading to enhanced profits. The process is perhaps most advanced in the realm of electronic payments, where money is shifted with very little human action at all. Despite recent talk of the rise of digital currencies, most money is digital already, and tapping your contactless payment card sets in motion an elaborate automated system of hard-drive editing that 'moves' your money from one bank data-centre to another. This technology underpins talk of a future 'cashless society'. Bouncy startups like Venmo and iZettle are getting into the payments game, adding friendly new layers to an underlying digital payments infrastructure that is nonetheless still dominated by the banking industry and credit card networks: new forms of banking still work much the same way as old.
Much of the financial system still retains a hybrid structure in which manual human actions interact with automated machine actions. The investment bank trader negotiates a derivatives deal over the phone and then books it into a partly automated back-office system.
Over the years, though, corporate managers have tried to push the power balance in this hybrid model towards the machine side, even in apparently human settings like the bank branch. You can talk with employees behind the Barclays counters, but often they are just there to enter data into a centralised system that tells them how to deal with you. To some degree these employees have agency – the ability to make decisions – but the dominant trend is for them to become subservient to the machine system they work with, unable to operate outside the bounds set by their computer. Indeed, many bank employees cannot explain why the computers have made the decisions they have, and thus they appear as the human face put there to break the news of whatever the algorithm has decided. We might even say they are a human interface to an otherwise algo-robotic system that is accountable only to the senior corporate management, who you will never deal with.
Nevertheless, humans are costly. In their ideal world, bank executives would get rid of as many manual human elements as possible and replace them with software systems moving binary code around on hard drives, a process they refer to as 'digitisation'. This generally means slowly dismantling branches while getting people accustomed to 'self-service'. Indeed, many banks are cutting branches, and many new forms of financial services are found only online, like digital banks Fidor and Atom. Digital banking startup Kreditech claims that bank branches won't exist 10 years hence, "and neither will cost-intensive, manual banking processes":
"We believe algorithms and automated processes are the way to customer-friendly banking," the startup declares confidently.
Such 'branchless banking' is but one strand in the digital trajectory. Digitisation is starting to be applied to more specialist areas of finance, too, such as wealth management. Wealthfront, for example, now offers automated investment advice for wealthy individuals. In their investment white paper they state that sophisticated algorithms can "do a better job of evaluating risk than the average traditional advisor".
Digital systems like Wealthfront are often promoted as cutting out the middleman – assumed to be human and slow – and therefore as cutting costs in both money and time. Some startups use this to build a narrative of the 'democratisation of finance'. Quantopian, a system for building your own trading algorithms, comes with the tagline: "levelling Wall Street's playing field". Robinhood draws on the name of the folk hero to pitch their low-fee mobile stock-trading system.
It seems uncontroversial that these systems may individually lower costs to users in a short-term sense. Nevertheless, while startup culture is fixated upon using digital technology to narrowly improve short-term efficiency in many different business settings, it is woefully inept at analysing what problems this process may accumulate in the long term. Payments startups, for example, see themselves as incrementally working towards a 'cashless society': a futurist buzzword laden with positive connotations of hypermodern efficiency. It describes the downfall of something 'old' and archaic – cash – but doesn't actually describe what rises up in its place. If you like, 'cashless society' could be reframed as 'a society in which every transaction you make will have to be approved by a private intermediary who can watch your actions and exclude you.'
The 'inevitable progress' of finance
Part of the reason for the pervasive acceptance of these developments is the deeper ideological narrative underpinning them, one which is found within the tech industry more generally. It is the idea, firstly, that the automation of everything is inevitable; and that, secondly, this is 'progress': a step up from the inefficient, dirty services we have now. In this context, questioning the broader problems that might emerge from narrowly useful automation processes is ridiculed as luddite, anti-progress or futile.
Yet it is apparent that many people don't respond to 'progress' in the way they're supposed to. We still find people insisting on queueing to use the human cashiers at Tesco, rather than opting for the automated check-out. Likewise, we still find people stubbornly visiting the bank branches, making manual payment requests; even sending cheques.
Perhaps this is because there is something deeply deadening about interacting as a warm-blooded individual with a soulless automaton trying to sound like a human. The hollow fakeness of the cold checkout machine voice makes you feel more alone than anything else, patronised by a machine clearly put there to cut costs as part of a faceless corporate revenue circuit.
The ongoing challenge for corporate management, therefore, is how to push automation while keeping it palatable. One key technique is to try to build more 'human-like' interfaces, and thus in London we find a hotbed of user-experience (UX) design firms. They are natural partners to the digitisation process, combining everything from ethnographic research to behavioural psychology to try to create banking interfaces that seem warm and inviting.
Another key technique is marketing, because people often have to be 'taught' that they want something. In the case of contactless payment on the London Underground, the Mayor of London, Barclaycard, Visa and the Evening Standard have formed an unholy alliance to promote Penny for London, a thinly veiled front-group to encourage people to use the Barclaycard-run contactless payments system rather than those ancient Oyster cards. Sports stars like Jessica Ennis-Hill and Dan Carter have been co-opted into becoming the champions of automated finance. Signs have been popping up proclaiming 'contactless is here', as if it were something that people were supposed to be waiting for. These subtle hegemonic messages permeate every financial billboard in the city.
The dark side of digital finance
One key to developing a critical consciousness about technology is to realise that for each new innovation a new trade-off is simultaneously created. Think about the wonderful world of digital banking. A low-level bank branch manager might be subservient to the centralised system they work for, but can also deviate subtly from its rules; and can experience empathy that might override strict economic 'rationality'. Imagine you replace such an individual with an online query form. Its dropdown menu is the digital equivalent of George Orwell's Newspeak, forcing your nuanced, specific requests into blunt, standardised and limited options. If your problem is D, a system that only offers you solutions to A, B, or C is fundamentally callous. A carefully constructed user complaints system can build an illusion of accountability, while being coded firmly to bias the interests of the company, not the user.
Indeed, if you ever watch people around automated self-service systems, they often adopt a stance of submissive rule-abiding. The system might appear to be 'helpful', and yet it clearly only allows behaviour that agrees to its own terms. If you fail to interact exactly correctly, you will not make it through the digital gatekeeper, which – unlike the human gatekeeper – has no ability or desire to empathise or make a plan. It just says 'ERROR'.
This turns out to be the perfect accountability and cost cushion for senior corporate management. The responsibility and energy required for dealing with problems gets outsourced to the users themselves. And lost revenue from unhappy customers is more than compensated by cost savings from automation. This is the world of algorithmic regulation, the subtle unaccountable violence of systems that feel no solidarity with the people that have to use it, the foundation for the perfect scaled bureaucracy.
So, in some future world of purely digital banking we find the seeds of a worrying lack of accountability and an enormous amount of user alienation. The loan you applied for online gets rejected, but nobody is there to explain what hidden calculations were done to reach that decision. To the bank management, you are nothing more than an abstract entity represented by machine-readable binary code.
Where is the financial AI?
Of course, the banks don't want you to feel like that. In the absence of employees, they will have to use your data to create the illusion of some type of personally tailored service. Your historical interactions with the system will be sold back to you as a ghostly caricature of yourself, fed through the user-experience filters. And it is here that we find the emergence of new forms of financial artificial intelligence.
There is a blurry line between machines and robots: robots are essentially machines capable of taking in data from sensors and processing it through an algorithmic 'mind' in order to react or 'make decisions'. Likewise, there is a blurry line between robots and artificial intelligence. At its most unambitious, AI it is just a term for any form of calculation done by robots. It really comes into its own, however, when referring to robots that have adaptation and learning capabilities that allow them to show creativity and unexpected behaviour. Rather than merely 'responding' to your actions or to external stimuli, the system begins to predict things, offer things, make suggestions, and do things without explicitly being asked to do them.
Imagine, for example, an ATM booth that uses facial recognition technology to identify you as you approach and make suggestions to you. Notice how the power dynamic changes? With a normal ATM I am still an active body, choosing to trigger the machine via the interface. In this new scenario I'm a passive body who triggers the machine without any conscious action on my part. It seems to 'take the initiative' and to direct me. It's only when we start to feel this as a power dynamic that we start to get closer to a sense of AI. The more you move towards AI, the more you feel increasingly passive relative to the robot.
Consider the customised ads Google feeds to us. We don't try to make them appear, yet it's our actions that trigger the system to target us with specific information. There are many scenarios where this process could creep into finance, from machine-learning trading algorithms to creepy health insurance contracts that shift their prices according to your mobile payment data. "I see you paid for two chocolates today Brett. I will raise your premium."
But this can go beyond a single machine. A robotic system may actually be constituted by an algorithmic 'mind' that coordinates a 'body' of people, like Uber drivers acting out the will of their invisible algo-boss. So the body of an AI may be fragmented, decentralised and hard to perceive, a network of interacting algo-robotic systems that direct the actions of people who are unaware they are triggering the system. No individual node may be in control, but groups of people may be locked into reliance upon the system, pulled around by forces not immediately apparent to them, being manipulated by their own data. The AI could be a ghost in the collective machine, the 'invisible hand' in a technologically mediated market.
Don't panic, but don't not panic either
When thinking about the future of digital finance, the issue is not necessarily whether these services are narrowly useful to an individual. Sure, maybe the contactless card is cool if I'm in a hurry and maybe I can get a decent deal from the AI insurance contract. Rather, the issue is whether they collectively imprison people in digital infrastructures that increasingly undermine personal agency and replace it with coded, inflexible bureaucracy; or whether they truly offer forms of 'democratisation'.
It is easy to overhype these scenarios because while it is true that payments, trading and retail banking are increasingly subject to automation, finance as a whole may not be especially amenable to it. Large loan financing decisions, complex multistage project-financing deals, exotic derivatives and other illiquid financial products cannot easily be standardised. They require teams of lawyers and dealmakers hashing out terms, conditions, and contingencies. Finance is an ancient politicised art of using contracts about the future to mobilise current action, and the dealmakers cannot easily be replaced with algos.
Furthermore, attempts to create more advanced and intuitive automated systems frequently fail. Semantic analysis algorithms – designed to read text – are terrible at understanding irony, sarcasm and contextual ambiguity within language. They may create feedbacks that thwart their own purposes, as in when people learn to game a credit-rating algorithm. High frequency trading falls apart under its own excesses and becomes less profitable. And there are customer backlashes: Metro Bank, the first new high-street bank in Britain for 150 years when it launched in 2010, has grown precisely because of its explicit focus on human-centred branch banking.
Nevertheless, it would be unwise to ignore the fact that the corporate trajectory is very much towards trying to automate as much as possible, and people need to come to terms with both the implications of this, and the vested interests behind it. It is not a neutral, 'inevitable' process. There are particular parties who seek it out. Take a moment to investigate who is on the board of Penny for London, that altruistic charity that insists contactless payment is a great way to help those in need. It includes hedge fund mogul Stanley Fink, and previously included the ex-CEO of Barclays, Bob Diamond.
So how should one respond? One approach is to ride with the technology, rather than to resist it. In intellectual leftwing circles the accelerationist sect advocates an embrace of automation, standing against sentimental calls for more human, local systems. It's an abstract position, founded on beliefs that automation will create conditions ideal for the downfall of capitalism. At some point it intersects with the cult of the Singularity, popular among evangelical tech entrepreneurs and transhumanists.
The ideological ambiguity is perhaps most acute in the emergent field of blockchain technology. Such systems potentially offer a way for strangers to freely interact with each other without central human intermediaries getting involved in the process. They may use blockchain systems to issue shares, enter into insurance contracts and form digital co-operatives, but the systems are underpinned by an extreme version of automation, one that is essentially autonomous. Indeed, the deep-level mission of projects such as Ethereum, a decentralised platform for 'trustless' transactions, is the replacement of human systems of institutional trust – like the legal and political systems that normally underpin all contracts and markets – with automated ones apparently detached from the human ambitions of those who historically have run such systems ('the politicians', 'the regulators', 'the bankers'). Libertarians long for an automated 'Techno-Leviathan' to replace the human sovereigns we have now, but it is a big question as to whether such automated systems truly provide a more 'democratic' infrastructure for interaction.
More down-to-earth are those who want to allow more creative interaction with the existing digital infrastructure. Take the Open Bank Project, for example, which wants to facilitate third-party customisation of digitised banking processes by opening up bank APIs, in the same way that independent developers might build third-party Twitter apps that draw data from Twitter's API.
And, finally, we have those who authentically seek to harness digital technology to bypass and challenge the standard economic rationality of largescale, short-term profit-seeking financial beasts, taking advantage of the lower startup costs of a digital setting to promote peer-to-peer finance, alternative currencies, crowdfunding platforms and non-monetary sharing platforms.
So, the scene is set. One thing is for sure: if the future of banking is going to be digital, we want it to be populated with those who value the deeper tenets of open-source philosophy. Otherwise we could be left with increasingly alienating, exclusive and unaccountable financial surveillance states, presiding over increasingly passive and patronised users.