Mac Patch Management Is an Urgent, Unsolved Problem

Mon, 22 May 2023 00:00:00 +0000

Few ad campaigns have ever had such a long-lasting impact as the iconic “I’m a Mac/I’m a PC” commercials that ran in the mid-aughts. In those commercials, John Hodgman’s PC was the stuffy, corporate computing option–not good for anything more exciting than a spreadsheet. By contrast, Justin Long’s Mac was the laidback computer you used at home–Mac was your buddy, not your manager.

The ad campaign was a smash hit for Apple, but nearly twenty years later, IT admins are still living with the fallout. Because in the years since these commercials aired, something unexpected happened. That friendly, casual Friday computer entered the workforce and loosened Microsoft’s stranglehold on the corporate network.

Today, Macs make up roughly a quarter of all US enterprise fleets (it’s actually probably more than that, since the most recent data is from before the release of the M1 Macs.) This remarkable rise has meant that IT pros and sysadmins have had to learn how to manage Macs.

In particular, admins consistently struggle with Mac patch management, which in this case means updates to macOS and first-party Mac apps. These updates are critical to security, but unfortunately, Apple spent years convincing its users that they were immune from security threats. (Remember “Macs don’t get PC viruses?”) As a result, it’s difficult to persuade users to do the right thing, IT resorts to working around (or even against) them, and it takes weeks to get the entire fleet updated.

Why Patching Macs Can’t Wait

A three-week lag time for patch installation is no longer acceptable in a world where zero-day exploits for Mac are more and more common. (MacBookProSlow.com has a good timeline of the greatest hits, most of which led to Apple issuing emergency patches.) One example came in August 2022, when Apple released an update to patch two vulnerabilities–one in the kernel, the other in WebKit–which Apple acknowledged “may have been actively exploited.”

While that cagey language is typical for Apple, it’s no secret that black and gray hat organizations such as NSO Group constantly search for weaknesses in Apple’s armor, which they sell for top dollar to both state and private actors.

The recent introduction of Rapid Security Response updates (RSRs) is a tacit acknowledgement by Apple that its security issues have become more urgent–otherwise, why develop a new delivery mechanism for them? But, as our CEO Jason Meller points out, you only benefit from RSRs if you install them immediately.

MacAdmins - if it takes you more than a couple of weeks to apply an RSR, then you aren’t getting the benefit.

May 1st - Apple issues an urgent Rapid Security Response (RSR) to patch a major vuln.

May 18th - Apple releases macOS 13.4 that contains the same urgent patch.
— Jason Meller (@jmeller) May 22, 2023

Apple’s security notesfor its latest macOS version (Ventura 13.4) revealed that the May 1st RSR had patched two serious WebKit vulnerabilities, including one that enabled remote code execution. These aren’t the kinds of patches you can afford to wait several weeks to get installed.

Here, we’ll talk about the existing options for Mac patch management, why they’re both bad, and how to solve the problem while still letting Macs and Mac users be themselves.

The State of Mac Patch Management

Let’s walk through a common situation. Apple has just released a macOS update, and immediately the clock to get it installed across the fleet starts ticking. Every hour that passes is a chance for a breach to happen, and yet, the average company takes weeks to months to patch a critical vulnerability.

The reason for this disconnect is that IT admins only have two options for Mac patch management, and neither of them can get you to 100% compliance in a reasonable amount of time.

Option 1: The Big Red Button (MDM)

Mobile device management solutions (MDMs) are a virtually foolproof way to enforce updates for every Mac in your fleet. IT can use MDM to deploy patches that install automatically and restart devices with no input from users whatsoever. But if patch management were that simple, this would be the last sentence of this article. (And it’s not.)

In reality, managing Apple updates via MDM is so disruptive that almost no one actually goes that route.

Here’s the problem: significant OS updates and upgrades require a Mac to restart, and forcing restarts without a user’s permission risks that they’ll lose whatever they’re working on. Even Apple’s documentation cites this risk, warning that “InstallForceRestart may result in data loss.” IT admins do their best to minimize disruption by scheduling deployments for the middle of the night and warning users ahead of time to save their work, but even so, someone always gets upset.

We’ve been over some of this before in our piece on the pros and cons of MDMs. But broadly, many employees (especially executives) simply won’t tolerate forced restarts, so you wind up with a long list of exempt users, and you’re left right back where you started: with wildly vulnerable devices.

In fairness, Apple has gotten better at helping Mac admins manage updates and patches as part of their larger push to make Macs more enterprise-friendly. They’ve released a first-party MDM (Apple Business Essentials) and created a web portal (Apple Business Manager) that enables easier integration with third-party MDMs like Jamf and Kandji–though the features are primarily limited to automatic device enrollment and centralized management for AppleIDs.

Apple has also made it easier to remotely install updates and upgrades without a local admin. Their documentation explains that, prior to macOS 12.3, only local administrators could perform software upgrades, but after macOS 12.3, Apple gave any user the ability to upgrade.

For Macs with Apple silicon, a user must be a volume owner to upgrade or update, and admins can act as volume owners via a bootstrap token. The introduction of volume owners as a class of user also enables enterprises to give end users some ability to maintain their devices without granting them full administrator privileges. (Though we’d still generally argue against that.)

Companies go to all sorts of elaborate lengths to control their endpoints, but at a certain point, the medicine becomes worse than the disease. Taking agency from users makes their devices nearly unusable–a Mac user without admin abilities can’t change their own time zone without help from IT. Meanwhile, feature-heavy endpoint management tools take a measurable bite out of CPU performance (and then what good are those fancy new silicon chips).

At the end of the day, the core problem remains: deploying patches without user consent leads to data loss events, so it’s an untenable option to deploy at scale.

Option 2: The Nudge

Since we’ve established that brute force automation is so disruptive that it can’t get your Mac fleet updated, how about asking users nicely?

This, in essence, is the philosophy behind Nudge, a UI tool that admins can deploy via MDM to remind users to update. These reminders get more frequent and more intense until, eventually, users no longer have the option to “defer”.

Source

Nudge (and some other tools like it) somewhat streamline the patching process by downloading updates in the background and only bothering users once it’s time to install them. And they do represent a step in the right direction; MacAdmins Slack is full of IT pros who swear by nudge tools and found that they’ve significantly improved patching time.

Even so, this is a clearly imperfect system because–surprise!–nagging people often makes them less eager to do what you want. At best, users ignore reminders for as long as possible (Nudge deferment periods are typically around 30 days), and at worst, they escape them altogether by turning to shadow IT (which is also a risk with heavily locked down devices).

Is Patch Management Really Harder for Macs than PCs?

Most of the issues we’ve described so far aren’t unique to Macs. On a technical level, the only real advantages PCs have are Nudge-style tools built into Windows and an MDM ecosystem built primarily around their needs.

Yet at Kolide, customers come to us for help with their Mac fleets far more often than PCs. So if, on a technical level, patching Macs isn’t more difficult than Windows, why does it give admins so much more trouble?

Part of the problem–though difficult to quantify–goes back to those famous Mac vs PC commercials. Macs feel more like personal computers, even when they’re technically work devices, and that feeling makes Mac users more resistant to intrusive measures. One IT admin we work with said, “We have no problem using MDM to manage updates on our PC fleet, but our Mac and Linux users want more independence, so we have to have a lighter touch.”

On a psychological level, Apple’s reputation for security could contribute to why users don’t take patching seriously. If you’ve had Macs as your personal computers for years and never had any problems with malware, you may not understand that you’re at much greater risk in a corporate environment or be aware that threats to Macs are generally on the rise.

And we haven’t even addressed the threat posed when end users use their personal Mac to access work resources. Both solutions we just went over assume that you’re managing a Mac via MDM. If a device is unmanaged (whether because of a BYOD policy or because it belongs to a third-party contractor), then you really don’t have a way to enforce updates.

Mac Patch Management Takes Both Technical and People Skills

Technologists tend to assume that for any problem, technology has to be the solution. And when there isn’t an obvious technical improvement to be made, it’s easy to flip to the other extreme and assume that the problem lies between chair and keyboard. Unfortunately, once you’ve classified something as a user behavior problem, a lot of IT people just throw up their hands and give up. That’s how we got to the current status quo of devices running an outdated OS for weeks or months.

But the status quo won’t cut it anymore. Getting to 70 or 80% compliance won’t satisfy auditors, regulators, or executives. We have to get to 100% patching, and to do that, we need users.

Specifically, we need solutions that force people to change their habits.

For an idea of how to accomplish this, let’s look at a different example: seatbelt laws.

Nowadays, most of us put on a seatbelt as soon as we get in the car, likely without thinking about it. But that wasn’t always the case–in fact, a lot of people were very resistant to seat belt laws. Normalizing this behavior required a full-court press effort involving:

Technical solutions: Laws mandating that all new cars had to have seatbelts

Education: Public relations campaigns explaining the value of seatbelts

Consequences: Fines for driving without a seatbelt

It was a significant effort, but once the lesson was learned, it stuck; these days, the biggest factor reinforcing the seatbelt behavior is probably just habit.

Helping users build the habit of immediately installing updates will be harder because updates are infrequent, but we can still apply the same tactics.

How to Get End Users to Update Their Macs

We’ve established that taking away user agency can’t solve patch management. But relying exclusively on users won’t work either, because they don’t understand the seriousness of the threat, and there are no immediate consequences for breaking the rules.

A new approach has to address all these issues, like so:

1. Technical Solutions: Automation absolutely has a role to play in patch management. You don’t have to abandon your MDM to get updates deployed–the goal is to do as much work as possible before users have to get involved. Once you’re there, provide clear, non-technical instructions so users can install updates themselves.

2. Education: Do not assume that your end users know that an unpatched OS is among your company’s most serious security threats. Most security training programs do not emphasize this, which means that any solution has to educate users about why timely updates are so important, so they’ll be willing to accept the inconvenience.

3. Consequences: The distant, unlikely chance that a user’s inaction will cause a security incident is too abstract; a new approach has to create clear and proportionate consequences, so users can’t just click “remind me later” indefinitely.

This approach is how Kolide handles endpoint security–for which one of our most popular use cases is Mac patch management.

1. Technical Solution: Kolide’s lightweight agent detects when a device is missing a critical update and reaches out with remediation instructions.

2. Education: Along with fix instructions, Kolide explains why an issue is a serious security risk.

3. Consequences: If a user doesn’t update their device within a set time limit, they cannot authenticate via SSO until they’ve fixed the issue. Stopping potentially compromised devices from accessing company resources is a fundamental tenet of Zero Trust, so the blocking function is reasonable and proportionate.

With Kolide, users can install updates and upgrades at a time that works for them, but they can’t delay it forever. And while Kolide is compatible with MDMs and other endpoint security tools, it doesn’t require them, so you can deploy Kolide even on unmanaged devices.

We (unsurprisingly) believe we have the best product for managing a Mac fleet. But whether or not you buy our solution, the more urgent issue is to acknowledge that as of now, Mac patch management has not kept up with the threat landscape. Fixing it will require both new tools and a new mindset. As somebody smart once said, think different.

Want to see more of how Kolide handles patch management? Watch our on demand demo!