How Microsoft Could Monitor You in the Office

Every aspect of computer users' lives — from their heartbeats to the occasional guilty smile — could be monitored and immediately analyzed under the futuristic system detailed in Microsoft's patent application.

Details of the planned "Big Brother" system are revealed in an application to the U.S. Patent and Trademark Office, seen by The Times, over 17 pages of text with 10 diagrams.

The systems work not only through desktop or laptop computers, but even through mobile phones or personal digital assistants, meaning that even out of the office the employee can still be monitored.

In its most advanced format, the system will monitor users' private interests.

• Click here to read the full text of the patent application.

The system works by recording and analyzing what words and numbers are used or Web sites visited, and by watching the user's heart rate, breathing, body temperature, facial expressions and blood pressure.

The patent application explains: "The system can also automatically detect frustrations or stress in the user via physiological and environmental sensors and then offer or provide some assistance accordingly."

There is a recognition that humans can have significant differences so all users will be given a "baseline" for normal physiological readings based on their body type and personality in recognition that these could affect their physical or emotional responses.

• Click here to visit's Cybersecurity Center.

• Click here for's Patents and Innovation Center.

The patent application gives the example of "an elevated heart rate during a tax return preparation may be considered normal for one user and not an indication that help is needed, but for another user, the inverse may be true."

Every response will be analyzed in "real time" to allow the computer to decide what action should be taken.

The patent application says: "From this data, statistics related to performance, success rate, frequency of problem, and the like, can be provided to users or can be employed to gauge a target user's success, performance, or efficiency with respect to other users."

One scenario given in the patent is of Joe, who is "spending more time on an activity than was originally allotted [by the system] and as a result may not meet his deadline for the project."

The next step is for the computer to select the most suitable employees by "comparing the performance of people working on similar activities and finding the best people for those types of activities such as for future assignment."

But the system described does more than just measure workload. It can test for honesty of those activities "performed successfully but not in accordance with company or government policies."

Heart rates, sweating and facial expressions are already used by law enforcement agencies to detect wrongdoing. Now an employee's laptop will be able to identify the fraudulent expenses claim or the illegal contract offer.

The patent application explains: "Monitoring user activity can facilitate auditing how activities are performed to look for or isolate patterns of user problems, abuse, common errors incurred by users, or to ensure company/government policies are complied with."

The "Monitoring System 500" allows groups of users to watch and monitor each other.

Microsoft says the 500 can "enhance social experiences among users by binding them or bringing them users together based on a target activity".

As an example it can "locate people that are watching a particular TV program at the same time or are performing a similar activity at the same time. Discussion groups or social events can be generated as a result."

However, Microsoft's vision of a world where the microchip replaces the middle manager has been greeted with caution by British business.

"We always say the best approach is the personal touch," said Stephen Alambritis, of the Federation of Small Businesses. "We would urge smaller employers to err away from this kind of software. It will sour industrial relations."

"The trust that exists between employer and employee will be undermined if staff feel they are under constant surveillance," said David Frost, director-general of the British Chambers of Commerce.

"Our research into employees shows that where individuals feel they are under excessive monitoring or surveillance they tend to have a negative attitude towards their employer and are therefore less likely to be motivated and committed," said Ben Willmott, employee relations adviser at the Chartered Institute of Personnel and Development. "Employers who consider introducing this need to think twice."