Over the last couple of months, I’ve been interested in that strange zone between known good practice and reality. Watching successive waves of ransomware sweep across Europe and the US, I wonder why so many companies are falling prey to attacks that are, in theory, easy to defend against. I’m aware that there are good reasons not to update and that large organizations move slowly, but measures can be put in place to protect systems we know to be vulnerable.
Last month I wrote about a web hosting company that was forced to pay $1,000,000 to retrieve its clients’ data from a ransomware attacker. Its servers ran software that hadn’t been updated since 2008. Today, I’m going to have a look at another egregious failure to update, one that has the potential to cause a global security and privacy catastrophe.
Memcached is an object caching server. It is used to improve the performance of web applications and it is installed on a truly staggering number of servers, including many of the largest sites in the world. In 2016, several serious vulnerabilities were found in Memcached. The vulnerabilities could be used to trigger heap buffer overflows, which could disclose sensitive data held in memory and be used to compromise servers.
Vulnerabilities of this sort are not unusual, and there is a process for dealing with them. The researchers who discovered the vulnerability reported it to the Memcached developers, who created patches that fixed the problem. Linux distributions pushed the patched versions into their repositories. Any server administrator that installed Memcached from their operating system’s repositories can update to a safe version of Memcached with a single command.
Several months later, the researchers wanted to see how widespread the vulnerability remained. It was natural to suppose that most servers that were vulnerable had been patched. They scanned over 100,000 Memcached servers that could be reached from the open internet. Having Memcached servers accessible in this way is bad in itself, but the results of the scan were even more disheartening.
Of the 107,786 servers scanned, 79% were still vulnerable. That’s 85,000 servers that can be reached from the open internet and are vulnerable to an attack that should have been patched months ago.
The researchers sent emails to the organizations running the vulnerable Memcached servers informing them of the danger. When they repeated their scan six months later, there were 73,000 vulnerable Memcached servers in the sample. Informing the organizations of the risk barely moved the needle.
As the researchers note in their conclusion:
The severity of these types of vulnerabilities cannot be overstated. These vulnerabilities potentially affect a platform that is deployed across the internet by small and large enterprises alike. With the recent spate of worm attacks leveraging vulnerabilities this should be a red flag for administrators around the world. If left unaddressed the vulnerabilities could be leveraged to impact organizations globally and impact business severely.
My conclusion is this: the web and the organizations that do business on the web need to do better. There’s little excuse for exposing Memcached to the open internet, and even less excuse for failing to patch critical vulnerabilities.