Black Friday 2024-07-19: Crowdstrike Falcon EDR BSOD fuckup & workaround batch script in Windows Safe mode

The whole world is now „enjoying“ the Black Friday of IT where independently of each other huge Microsoft Azure datacenter in US failed for several hours, and then popular EDR „Next-gen AV“ solution Crowdstrike Falcon deployed worldwide so fucked-up upgrade to their „sensors“ running on all the Windows workstations, laptops and servers, that they instantly went into BSOD and then bootloop. As the case still unfolding (check the Reddit megathread) there is still a lot of news we are going to hear, anyway for those affected until there is some more elegant solution the first known workaround is to physically log in to the station in safe mode and delete/rename a problematic .sys file(s) shipped in the screwed-up upgrade.

Other options as they came later:

  • revert the station completely from VSC snapshot or Veeeam backup etc.)
  • Some people claim that several (dozens) reboots if computer connected via Ethernet could manage to quickly get the patched version over net before BSOD happens. Definitely worth trying to cycle reboot machines until they finally boot.
  • Group Policy script for machines in AD: https://gist.github.com/whichbuffer/7830c73711589dcf9e7a5217797ca617
    Anyway this is how I did it the manual way before the other options were available:
Pokračování textu Black Friday 2024-07-19: Crowdstrike Falcon EDR BSOD fuckup & workaround batch script in Windows Safe mode