Malicious Life Podcast: Sony BMG's Rootkit Fiasco

Transcript

 In June, 2006, The Wall Street Journal’s technology critic Walter Mossberg conducted an onstage interview with Sir Howard Stringer, CEO of Sony, as part of the Journal’s ‘All Things Digital’ conference in Carlsbad, California. 

As someone who heads one of the largest and most influential multimedia companies on the planet, people always tended to listen to what Stringer had to say – but this time, the crowd was even more attentive than usual. Only a few months earlier Sony was dealt one of the most painful blows in the corporation’s long history, and everyone was eagerly waiting for what Stringer had to say about the incident.  

“The senior management of BMG or Sony did not know this was going on,” said the CEO in response to Mosbberg’s questions. “We responded very quickly and put out patches. … We didn’t say to ourselves, as a company, we’re going to screw every computer in town. We made a mistake and Sony paid a terrible price.”

A terrible price indeed. Sony BMG was a record company created in 2004 by a merger of Sony’s Music division and the German Bertelsmann Music Group: two well respected companies with a long history in the music business – yet a disastrous decision somewhere along the new company’s chain of command cost it millions of dollars, due to a recall of millions of CDs from music shops all over the US, as well as in monetary compensations as part of two separate lawsuit settlements in Texas and New York. Perhaps more damaging than the financial losses was the PR disaster that badly tarnished Sony’s reputation among its customers and signed artists, painting it as a self-centered, self-serving company that cares more about its bottom line than its customers. 

Stringer’s wasn’t much of an apology, really – but it was a notable change from the arrogant tone of Sony’s earlier responses to its accusers. Responses such as this one by Thomas Hesse, Sony BMG’s president of Global Digital Business, who spoke to NPR only a few days after the whole ugly affair blew up in the press. “Most people, I think, don’t even know what a rootkit is,” said the executive, “so why should they care about it?”

Nov. 10th, 2005: Seven Months Earlier. 

It wasn’t a particularly “flashy” virus. In fact, if it wasn’t for the special circumstances and the particular vulnerability it tried to exploit, chances are that no one would have taken a second look at Stinx-E.

It was sent as an email attachment to a mailing list of a certain British business magazine. When a Windows user clicked the attachment, the trojan installed itself and created a backdoor through which an attacker could take over the system. Yet the malware’s most striking feature was unveiled even earlier than that: during the installation process, Stinx-E copied itself to a file named $sys$drv.exe – and then…vanished. Anti-virus scanners failed to detect it. File Explorer didn’t show it, even if the user somehow knew exactly what directory to look in. 

It was exactly as security researchers predicted and warned about: the nefarious cloaking mechanism which Sony BMG used in order to hide the presence of its software from its customers, was now being leveraged by malware authors. Everyone who knew anything about computer security figured it was simply a matter of time until such a malware would be found in the wild – which only made Sony BMG’s blunder seem even more daft and ill-advised. “Virus Uses Sony BMG Software To Hide on PCs,” reported the Los Angeles Times, a headline that was repeated in various permutations in numerous websites and newspapers. 

But Sony BMG’s PR disaster was far from over. A day later, on Nov. 11th, 2005, someone released a cheating tool for World Of Warcraft, a popular online game, that exploited the same cloaking mechanism to hide from the game’s anti-cheat program. Even more embarrassing for Sony, the uninstaller that was supposed to remove the troublesome software was discovered to have a vulnerability of its own – which at least two malicious websites abused to take over the user’s system. Luckily, none of the websites appeared to cause any intentional damage: all they did was reboot the compromised machine. Dan Hubbard, a senior director of Websense – the security vendor which uncovered the two sites – said he believes that they were created merely to make a point. “They could have done a lot worse.”

And the point was definitely made: a lot of people were absolutely furious about Sony BMG for so carelessly endangering so many computers in the US and around the world. The record company scrambled to try and contain the damage, and declared that it was halting production of copy-protected CDs, and that it was pulling existing CDs from store shelves. Even Thomas Hesse tried to backtrack from his earlier infuriating statements: “We’re very, very sorry for the disruption and inconvenience that this has caused to music consumers,” he said in an interview with Bloomberg. 

But it was too little, and much too late. On Nov. 21, Texas Attorney General Greg Abbott filed a suit against Sony BMG, accusing it of surreptitiously installing spyware on millions of its music CDs. Class-action suits were filed against the company in New York and California, and the FTC declared its intention of suing Sony BMG for engaging in unfair and deceptive business practices. 

Nov. 3rd, 2005: One Week Earlier.

As the web exploded with scorn and contempt towards Sony BMG, it seemed that the business behemoth wasn’t phased even one bit. “The protection software simply acts to prevent unlimited copying and ripping from discs featuring this protection solution,” it said in a statement posted on its website, “It is otherwise inactive. The software does not collect any personal information nor is it designed to be intrusive to your computer system.” 

Except, this was a blatant lie: It was already obvious that the software in question – XCP was its name – did a whole lot more than simply prevent the users from copying and ripping music stored on Sony BMG’s disc. And if that wasn’t enough – a few days later two Princeton University professors, Alex Halderman and Edward Felten, discovered yet another copy-protection software called MediaMax hiding in millions of Sony’s CDs. Although this second software didn’t try to conceal itself in the same nefarious way that XCP did – it still engaged in spyware-style behavior: for example, contrary to Sony’s formal statement, MediaMax did indeed “phone home” each time the user played the CD. Even though the information it sent was minimal – the CD’s ID and the user’s IP address – this behavior wasn’t mentioned in the End User License Agreement, nor did the users have any way to configure or disable it. 

Another troubling behavior displayed by MediaMax was not only unethical – but also very much illegal. 

While a copy protection software is running on the computer, the user cannot copy the songs from the CD to his or her machine. However, in the brief time period between when a CD is inserted into the machine for the first time and when the anti-copying software is installed – the data stored on it is vulnerable to copying. To plug this loophole, MediaMax installed a “temporary” anti-copying mechanism even before the user views the End User License Agreement and approves it. This is a problem, since under the Computer Fraud and Abuse Act it is not allowed to install software without the user’s explicit consent. And to make matters worse, even if the user declined the license agreement – this “temporary” copy protection mechanism was never removed…

But perhaps most indicative of Sony BMG’s cavalier attitude towards its customers’ ownership rights with regards to their computers, was how it handled the uninstallation feature of both of its anti-copying software.

Initially, there was no option for the user to uninstall XCP or MediaMax. Even after the whole affair was exposed and Sony BMG was showered with complaints, the company was in no hurry to provide its customers with a way to uninstall the malicious software it had slyly embedded in their machines. A user who wished to remove the software had to first fill out a form on Sony BMG’s website, wait a few days for a reply email, then fill out yet another online form, then wait a few more days for a reply – which finally included a URL for an online uninstaller. “It is hard to explain the complexity of this procedure,” noted Prinston’s Halderman and Felten, “except as a way to deter users from uninstalling XCP.”

And then, as if to add insult to injury, it was quickly revealed that the uninstaller itself was so poorly designed, that it actually introduced a new vulnerability, making the user’s machine vulnerable to web-based attacks. The installer included an ActiveX control – ActiveX was a framework that allowed applications to share information via the browser – and this control allowed any website to run arbitrary code on the computer without the user’s permission. Muzzy, a Finnish hacker who reverse engineered Sony BMG’s uninstaller, dryly commented that – 

“Considering anyone can reboot the computer using these, I suspect security wasn’t thought about for even a second during development of this thing. Virus writers and such would be very interested in analyzing what these methods do, as some of them are remotely exploitable.”

Muzzy’s warning joined a chorus of many security experts who estimated that a malware attack that exploited the vulnerabilities created by the XCP anti-copying software was practically unavoidable. Only after these warnings materialized a week later, did Sony BMG agree to release a proper uninstall tool, replacing the web-based ActiveX uninstaller with a downloadable EXE or ZIP file.   

October 31, 2005: Four Days Earlier.

Rootkits have been around since the early 1990s. It is a technology which allows malware authors to hide, or ‘cloak’ their creations from the user – as well as from the ever-watchful gaze of antivirus software installed on the machine. They do so by filtering the data returned by the operating system’s internal functions. 

Say, as an example, that Windows Explorer – Windows’ native file browser – wishes to display a list of files present in a certain directory. To do so, Windows Explorer calls a function built-in into the OS which queries the said directory and returns a list of the files it contains. A Rootkit’s job is to intercept this returned data and modify it – say, to remove the name of a malicious file present in the directory – before passing it along to Windows Explorer, essentially making the file ‘invisible’ to it or any other software using the built-in function, including many commercial anti-virus solutions. 

Initially, Rootkits were a problem only in Unix-based systems – but beginning in the late ‘90s, rootkits for Windows began to appear as well. This prompted the software industry to find solutions to the rootkit problem – and one Windows expert who worked on the problem was Mark Russinovich. 

In 2005, Russinovich was Chief Software Architect at Winternals, a company he co-founded which offered utilities to manage, monitor and troubleshoot a Microsoft Windows environment. One of the tools which Winternals offered its clients was Rootkit Revealer: a program, as its name implies, designed to detect rootkits. It did so in a clever roundabout way: it queried the computer’s contents twice – once using the OS’s built-in functions, and once using its own custom one – and then compared the two outputs. If the results were dissimilar, this meant that a rootkit was probably messing with the outputs of Windows built-in functions. 

On that particular day, Oct. 31st, Russinovich was testing Rootkit Reavler’s latest version on his own personal computer. To his astonishment, the test came back positive: his system was infected by a rootkit. Mark described his reaction in a post he later published on the Winternals blog. 

“Given the fact that I’m careful in my surfing habits and only install software from reputable sources, I had no idea how I’d picked up a real rootkit, and if it were not for the suspicious names of the listed files I would have suspected RKR to have a bug.” 

Russinovich dug deeper into his system, and uncovered an unfamiliar piece of software that modified Windows’ functions so that any file whose name begins with “$sys$” would be ‘cloaked’ – that is, made invisible. He disabled the suspicious software. 

“[…] I rebooted the system. The cloak was gone as I expected and I could see all the previously hidden files in Explorer and Registry keys in Regedit. I doubted that the files had any version information, but ran my Sigcheck utility on them anyway. To my surprise, the majority did have identifying product, file and company strings. […] [They] claimed to be part of the “Essential System Tools” product from a company called “First4Internet”.”

Googling the company’s name, Russinovich found out that “First4Internet” was selling Digital Rights Management (DRM) software to various record companies. 

“The DRM reference made me recall having purchased a CD recently that can only be played using the media player that ships on the CD itself and that limits you to at most 3 copies.”

Russinovich flipped through his CDs collection and found it: “Get Right with the Man”, by the Van Zant brothers. He inserted the disc into the CD reader. 

“A look at the Services tab of its process properties dialog showed it contains a service named “Plug and Play Device Manager”, which is obviously an attempt to mislead the casual user that stumbles across it […] into thinking that it’s a core part of Windows.

At that point I knew conclusively that the rootkit and its associated files were related to the First4Internet DRM software Sony ships on its CDs. Not happy having underhanded and sloppily written software on my system I looked for a way to uninstall it. However, I didn’t find any reference to it in the Control Panel’s Add or Remove Programs list, nor did I find any uninstall utility or directions on the CD or on First4Internet’s site. I checked the EULA and saw no mention of the fact that I was agreeing to have software put on my system that I couldn’t uninstall. Now I was mad.

I deleted the driver files and their Registry keys, stopped the [rootkit] service and deleted its image, and rebooted. When I logged in again I discovered that the CD drive was missing from Explorer. Deleting the drivers had disabled the CD. Now I was really mad.”

Mark fired off an angry blog post in which he described his findings, concluding it with –

“The entire experience was frustrating and irritating. Not only had Sony put software on my system that uses techniques commonly used by malware to mask its presence, the software is poorly written and provides no means for uninstall. Worse, most users that stumble across the cloaked files with a Rootkit Revealer scan will cripple their computer if they attempt the obvious step of deleting the cloaked files. […] This is a clear case of Sony taking DRM too far.”

Unfortunately for Sony BMG, the Winternals blog was extremely popular in the tech community, and Russinovich’s angry post was quickly picked up by Slashdot – a popular hub for “News for Nerds” – and from there it practically exploded in the blogosphere and then in mainstream media.

Dan Kaminsky, another notable independent security researcher, picked up where Russinovish left off and discovered that the XCP rootkit was now present in no less than half a million different networks – including many military and government networks. Bruce Schneier said about these findings – 

“Those are amazing infection numbers, making this one of the most serious internet epidemics of all time — on a par with worms like Blaster, Slammer, Code Red and Nimda.”

Computer users were absolutely enraged. “What is next?”, wrote one commentator, “DRM that will rewrite your BIOS and turn your PC into an expensive doorstop for copyright violation?”. “Welcome to a Brave New World,” Wrote another, “People who pay for their music get viruses, while people who download it at no cost from illegal sources get clean MP3s that they can freely copy and use on whatever devices they own.” A slashdot user named Bodhammer summed up the general mood succinctly, writing – 

“Sony, you have gone too far…No PSP for Christmas! No PS3 next year! So you protected a $15 CD by killing ~$700 of hardware purchases plus whatever games I would have purchased. No wonder your stock sucks and your revenues are down! Your DRM works, I’m exercising my right not to purchase your products any more!”

—

At this point, it’s worth asking – why did Sony BMG make such a poor decision? Wasn’t it blindingly obvious that users would react as they did, when they’ll finally learn about Sony’s obnoxious and intrusive copyright protection scheme? 

Surprisingly, the answer is – no, it wasn’t obvious at all. There are plenty of cases where manufacturers place severe restrictions on what we can and can’t do with their products – restrictions most of us are willing to accept. Some cameras, for example, do not allow the use of third-party batteries or flashes. Some printers deny the use of unauthorized ink cartridges. Apple does not allow its users to install apps outside of its App store: sure, some people grumble and complain about it and some even try to jailbreak their devices – but the majority of users accept this limitation and move on. Why would a similar mechanism on a music CD cause such an extreme reaction? 

One reason has to do with the fact that many users did not expect a Music CD to contain software. Traditionally, store-bought music CDs held nothing but songs: clean, simple WAV files without any executable code, and thus were not considered a security threat. By adding such a software – and a buggy and poorly written one – Sony BMG introduced a significant security risk to its customers’ machines, without their knowledge. 

But more significant, I think, is the sense of violation felt by many users due to Sony’s actions.

[Nate] Hey, Ran. How’s it going? 

[Ran] N..Nate? What are you doing in my apartment? 

[Nate] Don’t you remember? You asked me to help you with a recording. 

[Ran] Yes, but that was three days ago. Have you been hiding here all this time? 

[Nate] I prefer the term “roommate.” Oh, by the way, some guy with binoculars across the street wanted to come hang, so I left all the doors and windows open for him.

[Ran] W..What? No, this is unacceptable. I want you to leave, right now!

[Nate] I’m sorry, but that’s not possible. I set it up so that if you force me out, your fridge will stop working. By the way, I didn’t really like your dog – so I bought a different one and threw out the old one, hope you don’t mind. And I used your toothbrush, and I slept with your wife, and your… [fade out]

How would you feel if a stranger surreptitiously took over your house, changed things around without your consent, and refused to leave once discovered? This helps explain the rage against Sony BMG – and the clumsy implementation and the security vulnerabilities created by XCP and MediaMax only served to irk the customers even further. 

Sony BMG wasn’t the only one that suffered from this angry response: the artists whose CDs carried the rootkit – without their knowledge, obviously – got hit badly too. “Get Right With The Man”, the album by the Van Zant brother that Mark Russinovich bought on Amazon and which started the snowball rolling, initially ranked number 887 on Amazon’s music charts. When user reviews started to alert potential buyers about the troublesome anti-piracy software that came bundled with the music, the album dropped almost 500 places in a single day. By Thanksgiving, roughly 20 days later, “Get Right With The Man” plummeted to number 25,802. 

Oct. 4th: Three weeks earlier.

In early October, 2005, a computer consultant from New York discovered a rootkit in one of his client’s machines, and traced it back to a music CD by Sony BMG. He reported the incident to Finnish security vendor F-Secure, who reached out to the record company. It seems that whoever received the call at Sony BMG didn’t quite understand what the fuss was all about, and so it was only two weeks later, on Oct. 17, that F-Secure’s people got to have a serious discussion with executives of First4Internet, the company that created XCP, the copy-protection software. 

These discussions, however, went nowhere. First4Internet’s executives saw no problem with their software and had no plans to stop deploying it – as well as no plans to recall or replace the millions of discs already purchased by customers. F-Secure wrote a report about the rootkit, much like it does for almost every other malware – but as a courtesy to Sony BMG, decided not to make it publicly available until a solution to the problem will be negotiated. 

It took another week or two until F-Secure managed to convince Sony BMG to create an uninstaller for their rouge software – but by then it was too late: Mark Russinovich independently discovered the rootkit and all hell broke loose. Mikko Hypponen, F-Secure’s director of antivirus research, noted that – 

“If [Sony] had woken up and smelled the coffee when we told them there was a problem, they could have avoided this trouble.”

But F-Secure’s decision to delay the publication of their report about Sony BMG’s rootkit raised another troubling concern. Bruce Schneier spelled it out a few weeks later in a blog post: 

“When a new piece of malware is found, security companies fall over themselves to clean our computers and inoculate our networks. Not in this case. McAfee didn’t add detection code until Nov. 9, and as of Nov. 15 it doesn’t remove the rootkit, only the cloaking device. Symantec’s response to the rootkit has, to put it kindly, evolved. At first the company didn’t consider XCP malware at all. It wasn’t until Nov. 11 that Symantec posted a tool to remove the cloaking. 

The only thing that makes this rootkit legitimate is that a multinational corporation put it on your computer, not a criminal organization.[…] What happens when the creators of malware collude with the very companies we hire to protect us from that malware? We users lose, that’s what happens.”

Five Years Earlier.

In August 2000, Steve Heckler, Sr. Vice President of Sony Pictures Entertainment, gave a presentation to a crowd of some 1200 computer enthusiasts at California State University Long Beach about his vision for the future of the music industry. 

The really interesting bit, however, came after his keynote address, when he was approached by participants who were curious about Sony’s reaction to a new service that surfaced on the internet just a year or so earlier, and was already deemed a major threat to the industry: Napster. Heckler did not mince words in his reply. 

“The industry will take whatever steps it needs to protect itself and protect its revenue streams … It will not lose that revenue stream, no matter what … Sony is going to take aggressive steps to stop this. We will develop technology that transcends the individual user. We will firewall Napster at source – we will block it at your cable company. We will block it at your phone company. We will block it at your ISP. We will firewall it at your PC … These strategies are being aggressively pursued because there is simply too much at stake.”

These somewhat prophetic remarks by Heckler demonstrate how frightened – nay, panicked –  were Sony and other record companies by the recent rise of peer-to-peer file sharing: according to the MIT Technology Review, by 2005 the music industry was losing revenue due to digital piracy at a rate of roughly 4.2$B dollars a year. 

This fear motivated the record companies to experiment with various tactics to fight the rising threat of internet piracy. Some launched thousands of lawsuits against individual users of peer-to-peer software, others tried to poison the file-sharing networks with fake or malware-laced files.  

From the get go, Sony took a particularly harsh stance against music piracy. This attitude wasn’t new: Sony has always zealously protected its products. One example is the Aibo robotic dog, which came with a limited set of built-in functions. An enterprising hobbyist reverse-engineered the Aibo’s software and created new functions that enhanced the robot’s abilities with new features such as dancing and talking. Instead of applauding the innovative addition and maybe even incorporating it into the original product – Sony threatened the hobbyist with a lawsuit and demanded that the software be removed from the web. 

Beginning in 2001, Sony started testing various forms of Digital Rights Management – or DRM, for short – for its music CDs. This stimulated a fierce debate about the ethicality and practical feasibility of DRM. Proponents of this new technology argued that the law must protect intellectual property of digital content just as it protects non-digital art forms, and that without anti-piracy measures, content makers would have little incentive to create new works of digital art. 

Detractors noted, however, that DRM often takes away from the consumer’s right to do as they please with a product they paid for and own – such as in the case of music CDs, where copy-prevention technology prevents the user from creating backup for his music files, or converting them to other formats compatible with different types of players. And worst of all, they claimed – DRM, even if perfectly implemented, will never solve the problem of piracy: it takes only a single determined user who manages to crack the copy-prevention mechanism and leak the CD’s content online, to negate all of that pain and effort. “Making digital files not copyable is like making water not wet,” said Bruce Schneier, “You can’t do it. DRM is a desperate attempt to cling to their old business model. They have to figure out how to make money in the new world.”

But Sony was unwilling or unable to adapt to the new digital reality that was forced on it. After its music division merged with the Bertelsmann Music Group in 2004 to create Sony BMG, the new company continued to experiment with various DRM schemes. Unfortunately, its executives showed very little discretion in how they chose the subcontractors who created these DRM products for them. 

—

Take SunnComm, for example – the company behind MediaMax, the 2nd anti-copying software discovered on Sony BMG’s CDs. 

It was born under a different name – “Desert Wind Entertainment” – and in a vastly different industry: providing Elvis and other impersonators for Las Vegas acts. In 1999, Desert Wind ran into serious trouble with the U.S. Securities and Exchange Commission after announcing a 25m$ deal with Warner Bros. – a lie whose purpose was, apparently, to inflate the price of its stock. 

In 2000, Desert Wind – under a new management – changed its name to SunnComm, and decided to enter the technology market. Its first decision was to buy a factory: A floppy disk factory… As someone who happened to be around in the early 2000s, I can say with absolute confidence that it was fairly obvious even back then that the floppy disk is obsolete. Unsurprisingly, the factory deal turned out to be a rotten one for SunnComm. 

It was then that two of its employees decided to leave the company in order to develop a DRM solution. The CEO managed to convince the two to stay and lead a new division dedicated to DRM software. 

Since SunComm had absolutely no prior experience with such products, its first attempts failed miserably. A Princeton student named Alex Halderman – the very same Halderman who would later go on to unearth SunComm’s MediaMax in Sony BMG’s discs – discovered that he can circumvent SunComm’s anti-piracy solution by simply holding down the Shift key. This made SunComm a laughing stock in the press, a poster boy of incompetence…yet for some unfathomable reason, it was this poster boy that Sony BMG hired to design its anti-piracy solution. 

The decision to contract First4Internet, the other company to supply Sony BMG with a DRM solution, shows similar lack of sound judgment. Prior to creating XCP for Sony BMG, First4Internet’s business focused on automatic content filtering, and it had no expertise in creating copy-prevention software. This might also be the reason why its leaders were willing to go where no DRM software dared go up till then, and incorporate malware-like techniques such as cloaking, into their product.

And so, in June of 2005, Sony BMG decided to go ahead and incorporate these DRM solutions into 52 of their music titles, to be released in North America: 2 million CDs with First4Internet’s XCP software, and 20 million more with MediaMax. 

After all, most people don’t even know what a rootkit is. What could possibly go wrong?… 

Epilogue 

Although DRM was never popular among music buyers, it was the Sony BMG rootkit fiasco that ultimately buried this technology as a valid solution for the digital piracy problem. The public backlash against DRM was so fierce, that in 2008 Apple decided to make its music service DRM-free. The rest of the music industry hurried to follow suit. 

The digital piracy problem WAS ultimately solved – or at least contained – in a very different way: streaming. Streaming services enforce much of the same restrictions that DRM software did: the music cannot be copied, shared or converted to other formats. There is, however, one major difference: the subscription business model employed by many of the companies involved in the media industry – Spotify, Apple Music, Netflix, etc. – doesn’t give the consumers the same sense of “ownership” they had with CDs, making these restrictions feel less invasive and infuriating. In return, the consumers enjoy a reasonably priced and easy to use service, providing them with access to previously unimaginable amounts of content. With the ever present risk of nasty malware lurking in file-sharing networks, most consumers find this compromise favorable. 

But the DRM battle is still being waged in other areas. Keuring, a coffee brewing machine manufacturer, was criticized heavily in 2014 when it incorporated a DRM solution to prevent its customers from using third-party coffee cups in its coffee machines. John Deere experienced a similar backlash when it was discovered that the software running on its tractors does not allow users or independent mechanics to repair them. It was only after the Senate introduced a bill to allow farmers to perform their own repairs, that John Deere relented and agreed to allow its US customers to fix their own tractors. 

It seems, then, that not all companies learned from Sony BMG’s painful lesson. Still, it’s encouraging to know that even when it comes to huge multinational corporations, power does have its limitations.