Microsoft blames EU agreement for CrowdStrike disaster, releases USB recovery tool

Daniel Sims

Daniel Sims

Posts: 1,513   +46
Staff
In brief: In the hours following Friday's global Windows BSOD, many wondered why CrowdStrike software was allowed full Windows kernel access. Microsoft now claims that a 15-year-old agreement with European regulators ties its hands on the issue, potentially leaving Windows-based enterprise systems vulnerable to similar catastrophes in the future.

Microsoft has provided additional information regarding the scale of last week's global CrowdStrike meltdown and how IT professionals can repair the damage. However, the company's options to prevent similar incidents are somewhat limited.

Although CrowdStrike fixed the faulty patch that brought businesses worldwide offline on Friday, many are still rebooting their systems or catching up with the resulting backlogs. Microsoft has provided a recovery tool with detailed instructions to facilitate repairs.

The software can create Windows recovery media or help a PC boot into safe mode. It requires a USB drive with between one and 32 GB of storage space and at least 8 GB of free space on a 64-bit Windows client. The process will format the USB drive to FAT32.

Microsoft claims CrowdStrike's meltdown impacted 8.5 million devices. While this number represents less than one percent of all Windows systems, their presence in the enterprise sector was significant enough to paralyze air traffic, hospitals, and other vital infrastructure.

Although the problem stems from CrowdStrike's software, a Wall Street Journal report indicates that Microsoft is taking most of the blame, as the company's infamous BSOD was the public face of the disaster. Many affected businesses and users had likely never heard of CrowdStrike before July 19.

Those in the know quickly blamed the software's level 0 kernel access to Windows, which allowed the error to hit the operating system's deepest layer. In contrast, Apple doesn't give third-party developers full kernel access, and its OS dodged the global meltdown.

In response, Microsoft told WSJ that it can't legally block third-party developers from gaining full kernel access due to a 2009 pact with the European Commission. A document on the company's website titled, "Microsoft Interoperability Undertaking," states that developers must be granted the same level of access to the operating system that Microsoft enjoys.

As long as the agreement stands, companies like CrowdStrike must voluntarily refrain from accessing the Windows kernel. It isn't uncommon for developers to draw scrutiny for invoking kernel-level system access. Hopefully, the CrowdStrike incident will bring more attention to this issue.

Permalink to story:

Microsoft blames EU agreement for CrowdStrike disaster, releases USB recovery tool

 
It seems to me the problem is that various enterprise outfits decided to accept immediate deployment of patches with insufficient oversight and/or assurances on patch quality, process quality, and mitigation options.

That kind of oversight can lead to plenty of problems above the Kernel 0 layer too.


 
  • Like
Reactions: tonyd, Hotlynx16, axiomatic13 and 5 others
brucek said:
It seems to me the problem is that various enterprise outfits decided to accept immediate deployment of patches with insufficient oversight and/or assurances on patch quality, process quality, and mitigation options.

That kind of oversight can lead to plenty of problems above the Kernel 0 layer too.
Click to expand...
I suspect the real problem is that CrowdStrike was hacked - almost certainly via credentials stolen back when MS was hacked by a "state-backed" hacker in November... No one will ever admit this, but it's AWFULLY convenient that there happened to be an MS cloud failure at the same time (and almost no one is mentioning that any more).

Very few large companies are willing to publicly disclose hacks - even if they are legally bound to do so - as it shakes confidence and drops stock prices... I know that in Canada, many auto dealers, who all use the same software, were hit with a ransomware attack a couple of weeks back and paid out $25 million to get the encryption key... not a single mention of it in the news though (I have a friend who works for one of the dealerships)...
 
  • Like
Reactions: TempleOrion and passwordistaco
brucek said:
It seems to me the problem is that various enterprise outfits decided to accept immediate deployment of patches with insufficient oversight and/or assurances on patch quality, process quality, and mitigation options.

That kind of oversight can lead to plenty of problems above the Kernel 0 layer too.
Click to expand...

I work for a small business that has had crowdstrike for a couple weeks now. One of the first things we turned off was auto sensor updates. We thought we were good but didn't have a grasp on channel updates being a separate thing or how they were pushed. It didn't look like something you could turn off anyways. There's a few boxes where you set bandwidth limits and that's it. So even if we put a 1Kb/s limit on channel updates and were 5 sensor patches behind in the falcon console, we still would've been hit. The file on our systems causing the issues was only 40Kb in size. It's frustrating.
 
  • Like
Reactions: SpiritCrusher, Hotlynx16, TempleOrion and 3 others
Seems like Microsoft wasted no time to make an argument to just forever avoid providing source code or close enough which I assume this is why a third party can do kernel level access, in any case they'd like to just turn all Windows machines into basically Apple devices that are completely locked down.

This is like blaming gas stations for the actions of an arsonist that happened to get his hands on gasoline at a local gas station 'Unfortunately there's not much we can do about these gas pumps that operate with no restrictions, we would like our gasoline to be sold exclusively at our proprietary pumps but the EU has our hands tied everybody'

Next time maybe the corporate world shouldn't try to take shortcuts when it comes to securing their system and hiring third parties that push untested patches maybe? You know it takes time and money to have an IT staff that makes sure you secure your systems properly and these are the consequences.
 
  • Like
Reactions: SpiritCrusher, TempleOrion, gingerbill and 4 others
Dimitriid said:
Seems like Microsoft wasted no time to make an argument to just forever avoid providing source code
Click to expand...
It is THEIR source code, after all. And -- as the argument points out, Microsoft is being blamed for a problem not of their making; a problem that would not have even existed had pudding-headed EU regulators not attempted to "help the consumer".

Dimitriid said:
Next time maybe the corporate world shouldn't try to take shortcuts when it comes to securing their system and hiring third parties that push untested patches maybe?
Click to expand...
This statement takes the lead for sheer vapid banality. Prior to this event, Crowdstrike was the recognized leader in the field, with an unmatched reputation. No company was irresponsible for choosing them, and the suggestion that all companies simply write their own software for endpoint protection and threat detection is asinine beyond belief.
 
  • Like
Reactions: tonyd, HyperPete, mbrowne5061 and 1 other person
UK here. To be frank, it beggars belief that Microsoft AND Cloudstrike don't have a better method for testing ANY software or ANY Falcon Security 40kb kernel update - especially as Microsoft went to some trouble to develop WSUS. And then allow Patches and Security updates to be rolled out, to certain test machines, for some time, PRIOR to rolling out to 10+ - 1000+ devices. Even with later OS deployments, WSUS has been effective and reliable.

One wonders, when Microsoft blamed the EU, they were joking right ?
No doubt Redmond's big boys are still miffed at the $bn fine applied by the EU for NOT removing Internet Explorer Software from Windows 7, despite many warnings that IF they did not, then the EU would fine Microsoft. Which they later did, much to the big boys in Microsoft's Big Boys Department's annoyance...

Anyway, I digress, given that Microsoft (and maybe Cloudstrke too) have WSUS (or alternatives) why on earth did Microsoft not integrate Cloudstrike (and others applications) within it ?

Companies like McAfee have also had access to kernel and tools, however, in my mind, have not yet BSOD'th 2,000,000+ devices (in my opinion and to the best of my knowledge and memory.)

The Russian's too - with Kaspersky Endpoint Protection - have not BSOD 2,000,000+ devices either. I am relieved that my own security is protected by Russia. If only that the World does NOT belong to America nor the Big Boys in Redmond's Big Boys Department, currently scratching heads wondering WHO they can damn blame next. Biden ? Trump ? Julian Assange ?

Best.
Hazard
 
  • Like
Reactions: TempleOrion
Endymio said:
This statement takes the lead for sheer vapid banality. Prior to this event, Crowdstrike was the recognized leader in the field, with an unmatched reputation. No company was irresponsible for choosing them, and the suggestion that all companies simply write their own software for endpoint protection and threat detection is asinine beyond belief.
Click to expand...

Funnily enough "asinine beyond belief" and "sheer vapid banality" is a perfect descriptor of you and all your posts. Well done 🤡🤣
 
hazarduk said:
One wonders, when Microsoft blamed the EU, they were joking right ?
Click to expand...
No. And had you read the article, you'd understand why. Allow random firms unlimited kernel access, and your kernel will regularly crash.

hazarduk said:
... given that Microsoft (and maybe Cloudstrke too) have WSUS (or alternatives) why on earth did Microsoft not integrate Cloudstrike (and others applications) within it ?
Click to expand...
Um, you forget the last three times Microsoft attempted to integrated important applications into Windows, the EU fined them billions of dollars and made them stop.

hazarduk said:
Companies like McAfee have also had access to kernel and tools, however, in my mind, have not yet BSOD'th 2,000,000+ devices
Click to expand...
Oops! McAfee hasn't simultaneously blue-screened as many, but their list of BSOD crashes is rather extensive. Here's one:

"...We have seen multiple reports of BSODs on some Windows 10 & 11 systems with McAfee antivirus installed. This crash is due to an outdated virtual driver from McAfee ..."

TempleOrion said:
Funnily enough "asinine beyond belief" and "sheer vapid banality" is a perfect descriptor of you and all your posts. Well done 🤡🤣
Click to expand...
Rather than simply toss insults, I explained precisely the problems with the post in question. If you have an issue with one of my statements, I suggest you do likewise. I'll also add that, while emojis are suitable for teenage girls, they're not an adult method of communication.
 
  • Like
Reactions: tonyd
hazarduk said:
Companies like McAfee have also had access to kernel and tools, however, in my mind, have not yet BSOD'th 2,000,000+ devices (in my opinion and to the best of my knowledge and memory.)
Click to expand...
Funny you bring them up… George Kurtz (CEO of CrowdStrike) was the chief technical officer and executive VP of McAfee until 2011… and under his watch, McAfee “accidentally” pushed out a software update that deleted critical Windows XP files and caused the dreaded BSOD…
 
  • Like
Reactions: Endymio
To be fair, *every* driver elevates to Ring 0 due to how Windows is designed. Thats the one major thing OS/2 got right: Putting drivers in Ring 2 so they can't directly hit the kernel.
 
gamerk2 said:
To be fair, *every* driver elevates to Ring 0 due to how Windows is designed.
Click to expand...
Yes, but Windows has the capability to restart a driver that faults, and to boot without it if the problem repeats. But, by the EU agreement, third-party developers were allowed the same access as Microsoft-specific drivers. Crowdstrike used this capability to mark its code (which isn't even a "device driver" in the first place, but simply an application) a boot-start driver. This means a faulty driver requires physical intervention to correct.

Also note that writing a device driver requires an EV certificate. Prior to the EU agreement, MS had been tightening the requirements and testing required for issuance, but -- to avoid further problems with regulators -- essentially turned this into a rubber stamp process.

gamerk2 said:
Thats the one major thing OS/2 got right: Putting drivers in Ring 2 so they can't directly hit the kernel.
Click to expand...
Not quite. OS/2 device drivers ran in Ring 0. But OS/2 had Ring 2 "I/O drivers" that were more limited than actual device drivers. Windows (and Linux) eliminated Ring 2 because under X86 the performance hit was abysmal .. and both OS's intended to run on CPUs which only supported two rings anyway.
 
Last edited:
They are not wrong; in the most base level of discussion.
Such regulations never work the way non-technical politicians think they will. And every one has caused major damage at the consumer level, even when they do what they intended to.

Remember when Microsoft had to offer an explorer free version of windows. And how those versions didn’t work right for a decade? (Most users downloaded and installed explorer).

Remember when they were forced to remove media player? Sure the likes of WinAmp and later VLC took off. And the consumer also got 20+ years of incompatible semi-proprietary codecs to go with it.

Weeks in and the iPhone has malware in the wild targeting EU phones. (Pro-“choice” writers are intentionally ignoring the story!)

It’s not just the EU, the breakup of AT$T gave us a decade of 976 and collect fraud, reverse billing, and eventually the MCI fiasco. Many in tech see that as the delay for unlimited calling.

Regulators see something to hang their hat on, make bad legislation, and then pretend they are not the #1 cause of public suffering.

There is no doubt this could not have happened without low level access. As demanded by regulation.
 
  • Like
Reactions: Endymio
You must log in or register to reply here.

Similar threads

Daniel Sims
The internet reacts to the CrowdStrike IT disaster that crashed computers worldwide
2
Replies
40
Views
415
seeprime
seeprime
midian182
Businesses around the world hit by 'Blue Screen of Death' following faulty CrowdStrike update
Replies
12
Views
220
zamroni111
Z
S
CrowdStrike also broke Debian and Rocky Linux earlier this year – hackers are taking advantage of Friday's chaos
Replies
5
Views
108
WhiteLeaff
WhiteLeaff

Latest posts

Back