Encrypted Binaries Found in Apple's Mac OS X

A computer researcher has made public information that Apple Computer (AAPL) has encrypted at the binary level critical parts of its Mac OS X operating system.

These "Apple-protected binaries" can serve to protect the OS from being pirated and also to make it "nontrivial" to run Mac OS X on non-Apple hardware, said Amit Singh, a member of Google's (GOOG) technical staff in Mountain View, Calif., and the author of "Mac OS X Internals: A Systems Approach."

Singh has also given lectures on Mac OS X to the National Security Agency and at Apple's main campus in Cupertino, Calif.

• Click here to visit FOXNews.com's Cybersecurity Center.

• Click here to visit FOXNews.com's Home Computing Center.

According to Singh, the parts of Mac OS X that are protected include the Finder and Dock applications, as well as parts of Rosetta (Mac OS X's application for running Power PC applications on an Intel-based Mac) and services that manage the user interface.

Singh noted that his list was not exhaustive.

Much of Mac OS X is open source, including Darwin, an entirely functional, open-source operating system based on FreeBSD 5.0 and the Mach 3.0 microkernel, and the basis for Mac OS X.

The Apple-protected binaries signal their protected status by setting a special bit in the header, Singh said.

When any binary is called upon by the system, the kernel checks to see if it is Apple-protected; if it is, the kernel unencrypts the code through an "unprotect" operation.

This operation, Singh noted, includes a "dsmos_page_transform" command, in which "dsmos" stands for "Don't Steal Mac OS X". He also found a "Don't Steal Mac OS X.kext" kernel extension in the operating system.

"A lot of times, encrypted binaries are used as piracy protection," said Bruce Schneier, founder and chief technology officer of Mountain View, Calif.-based Counterpace Internet Security. "It's a common technique," he said.

"But more often, and probably what it's used for here," he added, "is as anti-reverse engineering."

Schneier noted that encrypted binaries can affect application performance due to the extra decoding step before they can be executed.

However, he said, "As computers grow faster, there's more processing power to do stuff like this.

"The devil's in the details," he said.

Speaking to concerns about privacy, Schneier said, "There's nothing sinister here."

"This is a method for Apple to protect its code," he said, adding that for people who still want to try to get Mac OS X running on commodity PC hardware, "you can get around it, but not easily."

Apple representatives were not available to comment.

Check out eWEEK.com's Macintosh Center for the latest news, reviews and analysis on Apple in the enterprise.