Twenty banks (some suffering repeated outages), six countries (one in lockdown), five continents, tens of millions of unhappy customers.
There’s never a good time for your bank’s IT system to go down. But few can be worse than in the middle of a lockdown. It’s difficult to leave home, your local branch may not be open, and as a result you are more reliant than ever on digital banking services. In New Zealand, now in its seventh week of nationwide lockdown, one of the country’s largest lenders, Kiwibank, went down on Tuesday, leaving many of its customers in the lurch. It is one of a string of IT outages the bank has suffered over the past three weeks, after a DDoS attack on New Zealand’s third largest Internet provider caused IT crashes at a number of lenders, including Commonwealth Bank and Anz Bank.
In a DDoS attack hackers overwhelm a site by getting huge numbers of bots to connect to it all at once, rendering it inaccessible. Servers are not breached, data is not stolen but it can still cause plenty of disruption.
24 Million Unhappy Customers
New Zealand is not the only country to have suffered major outages within its banking system in recent weeks. Other countries include the UK, Japan, South Africa, Venezuela and Mexico, though there are no doubt more (if you know of any, It would be great if you could provide details in the comments section).
On September 12, operating failures at Mexico’s largest bank, BBVA Mexico, left 24 million account holders unable to use the bank’s 13,000 ATMs, its mobile app or in-store payments for almost 20 hours. It being a Sunday, customers could not even avail of the lender’s in-branch cash services. The bank blamed the outage on a system update failure and has offered to compensate customers with cash bonuses on purchases when using their credit or debit cards. The bank was also at pains to assure them that their financial data was not compromised.
“It had nothing to do with the outside world,” said Jorge Terrazas, the bank’s director of communicate and corporate identity. “The bank and its customers’ information is secure. What we did was undo the changes to the system and return everything to as it was.”
Less than a week after BBVA’s outage, Santander Mexico, another Spanish-owned Mexican bank, suffered an outage that left customers across the country unable to use their debit cards at the ATM or in stores. Again, it was blamed on internal problems.
In recent years, Mexico has become an important market for stolen data — enough to earn it eighth place in the world in terms of identity theft, according to the country’s central bank, Banco de Mexico (Banxico for short). This is partly a result of the widespread impunity cyber criminals enjoy in the country, due to the lack of enforcement of existing laws and the absence of adequate legal tools. Cyber theft in Mexico is not just the preserve of isolated basement-dwelling hackers but also highly professional criminal organizations.
Even Banxico’s SPEI interbank transfer system, an iteration of the SWIFT global payment system, has been the target of digital heists, as WIRED reports:
In January 2018 a group of hackers, now thought to be working for the North Korean state-sponsored group Lazarus, tried to steal $110 million from the Mexican commercial bank Bancomext. That effort failed. But just a few months later, a smaller yet still elaborate series of attacks allowed hackers to siphon off 300 to 400 million pesos, or roughly $15 to $20 million from Mexican banks. [Click here to read how they did it].
Since then Mexican banks have suffered repeated outages, one of the biggest of which took place during last year’s “Buen Fin”, an annual nationwide shopping event inspired by Black Friday. The online banking websites and mobile apps of many of the country’s major banks, including BBVA and Citibanamex collapsed on the same day, leaving many customers unable to complete their purchases.
“A Growing Trend”
In the UK the Financial Conduct Authority has been “deeply concerned” about the increasing number of technology outages for a number of years. At the FCA’s annual public meeting in 2019, the regulator’s executive director of supervision, Megan Butler, said the number of incidents of “operation resilience breaks” reported in terms of IT failings had increased 300% year-on-year. And this, she said, would probably be “a growing trend,” though it is partly due to the rise in reporting of events.
On July 22 this year, the websites of six large banks and building societies — Lloyds, HSBC, TESCO Bank, Bank of Scotland, Halifax and Barclays — were brought down by a global Internet outage allegedly caused by a botched software update at hosting service Akamai. Less than a month later, the apps of five lenders and building societies — Natwest, TESCO Bank, TSB, Santander UK and Halifax — all went down over a period of just a few days. The outage, apparently triggered by a problem with US payments company TSYS, left consumers unable to access their credit card services and account information. Since then, HSBC, Barclays Bank and the Cooperative Bank have all suffered brief outages.
Some outages can last much longer and wreak far more disruption on people’s lives. In 2018 Banco Sabadell’s botched IT migration of its UK subsidiary TSB — branded the “biggest IT disaster in British banking history — left hundreds of thousands of customers unable to access their online accounts for weeks on end. Some customers lost out financially. Many saw their credit ratings deteriorate as a direct result. Business customers were unable to pay bills or make payroll and mortgage payments were missed. Over 1,300 customers became victims of fraud attacks. The crisis cost Sabadell hundreds of millions of pounds, 80,000 customers and one CEO. It was probably a key factor in scuppering BBVA’s takeover of Sabadell late last year.
“An Intense and Aggressive Cyber Attack”
Almost 5,000 miles away from the UK, on the other side of the Atlantic, 16 million customers of Venezuela’s biggest bank, Banco de Venezuela, had to recently endure five days without the bank’s online platform. As tends to happen in these cases, the outage became apparent when bank customers began venting their anger on social media. When the platform was finally restored, on September 20, Venezuela’s vice president Delcy Rodríguez laid the blame on the US government, which she accused of launching an “intense and aggressive” cyber attack against the bank’s IT system.
The attack was apparently an attempt to derail Caracas’ plans to launch a new currency, which went live today (Oct 1) with six fewer zeros. Whether Rodríguez’ allegations are true or not it’s impossible to tell, but Washington certainly has the capability and form. Plus, it is engaged in a no holds-barred economic war against Venezuela.
Sometimes it’s the frequency rather than the duration of the outages that is the biggest problem for bank customers. Yesterday (September 30) Mizuho Bank, one of Japan’s three mega banks, experienced its eighth IT system failure so far this year — almost one every month. In the latest episode a system glitch caused a delay to some foreign exchange transactions. The system outages at Mizuho date back almost two decades and have been broadly blamed on its failure to integrate cultures and systems from the three-way merger of Dai-ichi Kangyo Bank, Fuji Bank, and IBJ that brought the bank into existence, all of 21 years ago. The bank has already spent $3.6 billion trying to fix the problems, but to little apparent avail.
Mergers of large banks have a tendency of leaving behind serious IT system issues, as Clive and I pointed out in an NC article published in December last year. This is particularly true in the case of cross-border mergers. One of the main reasons for this is that many banks are still largely run on creaky legacy systems built in the 1970s that make it all but impossible to merge IT systems without storing up big problems further down the line. In a 2019 Treasury Select Committee inquiry into what went wrong at Banco Sabadell, Alison Barker, director of specialist supervision at the Financial Conduct Authority, was asked to what extent legacy systems are still being used across the UK’s retail banking sector. Here’s what she said:
“It is still pretty extensively, I’m afraid… some pretty core systems are still run on legacy. They still use code back from the 1970s on some of these systems, and they’ve just built on top of them.”
Yet many of these same banks are still trying to compete with younger, smaller, fleet of foot challengers whose IT systems are much more modern and flexible. And that is causing serious problems.
Inherent Fragility of Legacy Systems
“If you are a large retail bank in the UK, you are probably dealing with legacy systems”, the deputy chief executive of the Prudential Regulation Authority, Lyndon Nelson, told the inquiry. But as fintech companies add new features to their apps, they are keen to do the same “for competitive reasons.”
Nelson added that although some banks do plan to eventually phase out their legacy systems, it takes a brave chief technology officer to envisage that, due to the inherent risk in changing systems. Sabadell’s disastrous attempt to upgrade TSB’s system will hardly have encouraged others to do the same. As S&P Global recently noted, bungled IT change is a leading culprit for outages and disruptions at U.K. financial institutions. An overreliance on outsourcing could make the problems worse.
Another problem highlighted by Nelson (and NC way back in 2016) is that few programmers are left who can actually use COBOL, the primary programming language used in banks’ legacy systems. This, says Nelson, has left many banks’ IT officers asking the question: “how many times in a week can we change an app without it falling over?”
When a banking app “falls over” or an IT system goes down, it can leave chaos in its wake. Ten years ago, Mizhuo Bank suffered an outage that delayed money transfers in the aftermath of the Great East Japan Earthquake and tsunami. Its seventh outage this year, in early September, was apparently the final straw for Japan’s financial regulators, which requested that Mizuho submit a work plan for system maintenance and updating, “in a rare move to effectively oversee the system of a megabank”, reported Kyodo News.
Another bank that has been plagued by repeated IT system problems is South Africa’s largest lender, Standard Bank. In late April, the bank suffered “hardware issues” that downed its internet, mobile and ATM channels for over a week, leaving customers unable to pay their bills or access cash. By early September the bank’s mobile app was down once again, causing customers no end of hassle. On Tuesday this week the mobile app of another South African bank, Capitec, also went down.
All of these bank outages are happening for a variety of reasons, from internal problems within a bank’s IT system (Mizuho, Sabadell) to a botched update (BBVA), to a cyber attack (Kiwibank), to the downing of a hosting service (the collapse of bank websites around the world on July 22). But one thing they all highlight is the inherent fragility of banks’ IT systems, at a time when many people are using less and less cash and are becoming more and more dependent on digital banking services.