On July 19, 2024, a widespread outage caused by a flawed CrowdStrike security update for Windows devices led to significant disruptions across the globe, impacting airlines, banks, media outlets, and various companies. This unexpected event left travelers stranded at major US airports, including JFK and LAX. For businesses, understanding the scope of this outage and formulating an effective response is crucial to minimize disruptions and maintain operational stability.
The outage had a broad and severe impact on multiple sectors:
Justin Rapacz, SVP of Managed Services at Netrix Global, shares that, “Due to an update released by CrowdStrike impacting Windows devices, several Netrix Managed Services customers have encountered Blue Screens of Death (BSOD) or unbootable servers and workstations, affecting various systems across their ecosystems. Our team has been actively addressing these issues, successfully restoring systems for several customers by following prescribed remediation protocols.”
In the early morning hours of July 19th, George Kutz, CrowdStrike’s CEO released a statement on X saying they are “actively working with customers impacted by a defect found in a single content update for Windows hosts.”
The outage was traced back to a defect found in a content update for Windows hosts. The issue was identified by CrowdStrike, which confirmed that it was not a security incident or cyberattack but a technical fault.
Based on reports, the outage appears to be caused by an update to CrowdStrike’s software called Falcon Sensor. The issue was quickly fixed but outages remained persistent because it was not clear how to fix the huge numbers of computers that had already been impacted.
CrowdStrike CEO George Kurtz stated, “The IT issue causing the global outage has been identified, and a fix has been deployed.” Microsoft’s quick and effective response highlights its commitment to maintaining the integrity and reliability of its services.
Our resident cybersecurity expert and field CTO, Rich Lilly, stresses, “It should be noted that this issue is not related to a cybersecurity incident or an issue with the Operating System itself, but rather, an update released to CrowdStrike’s Falcon Sensor product. Microsoft Defender for Endpoint (MDE), is not impacted and is a reliable alternative solution for teams looking to remove CrowdStrike solutions quickly.”
The recent software udpate is causing Blue Screen of Death (BSOD). The workaround to CrowdStrike is the following:
Unfortunately, this is a very tedious and time-consuming process, especially if you have thousands of endpoints.
Additionally, those leveraging technologies like Bitlocker are facing complications, as they are not allowed to boot into safe mode without first supplying a long code.
“The remediation process is straightforward but requires manual intervention, including removing a specific file before rebooting. We will continue to work diligently with affected customers to resolve ongoing impacts and provide meaningful updates as they become available,” says Rapacz.
CrowdStrike has since provided details on how to address various scenarios that may have occurred.
In the wake of this significant outage, our team is waiting on standby to step in and support. Here’s how Netrix Global can assist:
Helping with Recovery and Manual Patch Application:
If your organization needs extra hands to recover and apply the necessary manual patch, we’re ready and waiting to help. Our team of experts can provide immediate support to ensure your systems are quickly restored and operational.
Exploring Alternatives to CrowdStrike:
Given the issues related to the CrowdStrike component, it might be an opportune time to consider alternatives. Remember, there is not only a credibility risk to working with CrowdStrike but also a potential financial one. We are not yet sure what the repercussions of such a massive issue with have on the financial stability of their business and longevity of their place in the highly competitive market.
“It’s crucial to consider the robustness and integration of security solutions within an operating system. Microsoft Defender for Endpoint (MDE), being built into the Windows operating system, offers a seamless and deeply integrated protection layer,” adds Lilly.
We can help you make the switch to Microsoft Defender for Endpoint, offering a seamless transition that can be done quickly by our team. We can often times have customers fully functional on the new platform within weeks.
Consultation for Backup and Recovery Solutions:
If your current recovery processes failed to restore systems to a known good configuration, let’s talk. We can implement a backup solution that ensures you have reliable recovery options in the future, minimizing downtime and disruption in similar incidents.
“It’s our first and foremost priority that we educate our clients on what occurred today and help them to get systems back online and business back to normal. Our teams have been hard at work all day with clients answering questions and stepping in to support where necessary,” say Vlad Gotsev, CTO at Netrix Global.
2024 Netrix, LLC | All Rights Reserved.
