The perils of inadequate lifecycle management for security certificates

Last week, the U.K.’s O2 and Japan’s SoftBank suffered major mobile network outages which caused significant inconvenience and disruption, due to an expired certificate in the mobility management software on the networks. While I don’t want to be a doomsayer, this is significant and has implications for anyone in the industry building devices and systems for the internet of things (IoT).

What exactly happened? Well, the second largest mobile network operator in the U.K., O2 (which is part of Telefonica) was unable to provide data and voice services for almost 24 hours to its more than 32 million 2G, 3G and 4G service connections in the country. Its network is also used by mobile virtual network operators, like Sky Mobile, Tesco Mobile and GiffGaff. I experienced this firsthand as I was in Bristol for the day and unable to make calls or use data services for most of the day (fortunately I’d obtained a Huawei 4G mobile Wi-Fi device the day before on a different network, which then kept me connected via Wi-Fi.

And in Japan, SoftBank, which has around 40 million mobile users in the country, was hit by the same connection issue for about for about four hours. Reports suggest many businesses and services were affected, including airlines, railways, and logistics companies.

Both in the U.K. and Japan, the outages were blamed on Ericsson equipment and softwares. Ericsson issued a statement during the day, saying:

“During December 6, 2018, Ericsson has identified an issue in certain nodes in the core network resulting in network disturbances for a limited number of customers in multiple countries using two specific software versions of the SGSN–MME (Serving GPRS Support Node – Mobility Management Entity).

An initial root cause analysis indicates that the main issue was an expired certificate in the software versions installed with these customers. A complete and comprehensive root cause analysis is still in progress. Our focus is now on solving the immediate issues.”

Börje Ekholm, the president and CEO of Ericsson, said the faulty software that caused the outages was being decommissioned, and apologized not only to its customers but also to their customers.

While these weren’t exactly the scenarios we detailed earlier this year in the EETimes special project, The Day When the Industrial IoT Gets Hacked , it is worrying nevertheless. As I and fellow travelers to Bristol struggled to get connected or even make phone calls, I thought about how the people we talk to and interview get excited about opportunities from connected devices, the IoT, smart homes, smart factories, smart cities, smart mobility and anything else that relies on connectivity to the networks.

Even O2, the network that went down in the U.K., has a special page on its website outlining a complete range of IoT products, saying, “we’ll quickly turn your IoT ideas into business value.” The site also details areas like its smart connectivity platform and capabilities enabling smart metering, smart cities and smart vehicles.

But if the network goes down, all these services become useless. City infrastructure relies on sensor data to inform and sometimes drive public services, including transport — for example, people or vehicle counters, emission and environmental sensors. In Japan, East Japan Railways users were unable to purchase some tickets for their trains while the network was down. Many businesses that use mobile banking were unable to make payments. Logistics companies that use asset tracking and provide information about packages or shipments lost the capability to do so.

And car sharing or transport services that rely on mobile apps — like Uber, or more specifically, the drivers whose livelihoods sometimes depend on it — completely lost out on business, with people unable to connect to book services or to track drivers and call them.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.