Watching the Internet Think

The Web is an organism, and science can graph it. Through mesmerizing, flowing images and pulsing charts, new Web sites show who’s talking about which topics and how the Internet is connected -- even which colors are popular this season.

It's literally the pulse of the Web. Want to try it for yourself? Here's the best sites to visit.

Digg Labs
No one can match Digg for monitoring the hysteria of the moment -- links can soar to 5,000 Diggs in just a few days, and no one really knows why. Digg Labs helps you get a visual handle on the hoopla.

There are six labs, but each one taps into real-time Digg information. It's not experimental -- these are real Diggs from actual users, but the method of showing the data is unusual. Stack drops small tiles onto a graph that slowly builds into popular Diggs. Swarm is like a beehive where actual digg topics latch onto one another, following the progression of anonymous users.

Arc is - as you can guess - a group of Diggs that move around a circle. Larger text sizes means hotter topics. Big Spy -- perhaps the least useful tool -- shows larger text sizes for the most popular topics. And Pics is all about popular photos and graphics. (Be careful with this one at work, it can show you pornography on rare occasions.)

More On This...

Knowing that these tools can drain your Internet connection of all life and vitality, Digg Labs will pause automatically if you leave them open in your browser for too long.

Google Trends
Google Trends lets you compare terms such as "Barack Obama" and "Sarah Palin" to see who is more popular on the Web, and there's an interesting way to visualize the answers: When you search, you can view a bar chart with spikes that shows not only search topics (such as "Tiger Woods") but commentary from real posters on the Web, what Google calls real-time search. These comments include blog posts, social networking updates, and Twitter status.

"As the amount of information on the Web grows, the need for new ways to parse that information grows too," said Dylan Casey, product manager at Google. Casey explained that graphing tools give you the big picture, just as a ski resort might report on average monthly our yearly snowfall levels. The averages help you plan your ski trip beyond just knowing how much it snowed in the last week.

Using Google Trends, it's possible to see a graph over a historical period too -- say, for three years -- to see if the topic is becoming more popular. This could help a golf club manufacture plan a new product -- or a new advertising campaign boost interest.

Microsoft Pivot
Presented at the TED conference recently, Microsoft Pivot is a remarkable tool that lets you research Web information in real-time with a simple bar chart -- and it's a great way to watch Wikipedia.

Normally, Wikipedia entries are self-contained -- you can look up Barack Obama and read his biography, for example. But what if you wanted to research his accolades? Pivot shows you he was the Time Magazine person-of-the-year. You can then "pivot" this data and see every person-of-the-year, or click filters to see only African-Americans. This type of data exploration just isn't possible otherwise.

"Anything visual is more immersive," explains Brett Brewer, the Microsoft general manager in charge of Pivot, which is part of the Live Labs project. Pivot presents data in a way that makes exploration easier and leads to amazing discoveries. This kind of visualization uses a stock set of data -- say, Sports Illustrated covers or Wikipedia posts -- but could be used for any live data on the Web, such as accident reports. Pivot can show mortality rates, for example, letting you pivot the data to see in real-time that young men die in more car accidents than women.

Trendistic has become a news outlet of sorts -- people tweet on topics almost immediately after they occur, and sometimes during the event itself. That's what makes so valuable: the site shows the pulse of the Internet, including the most popular celebs, news topics, and gadgets. Are people talking more about the NBA or MLB?

Sociologists study these patterns by performing surveys and interviewing people (read any of Malcolm Gladwell's books, including his latest) but Trendistic is a picture into the soul of the Web. For any term, you can also see real tweets and start following what they say from your own Twitter account.

Cosmic 140
Another chart that shows Twitter trends, Cosmic 140 is the creation of Information Architects a Web design firm. The map -- which has not been officially released -- is amazingly complex, showing a "cosmological" chart that shows the most important Twitter users and how they are all connected to each other, starting with the three Twitter founders.

Explains founder Oliver Reichenstein, "We chose Twitter because (unlike Facebook) it allows us to access specific data and it features a lot of high profile people that define the new nodes based on tweets per day, number of tweets, and tweet content.", like Google Trends, shows topics as trends on the Internet, but goes a few steps farther. For example, the service can shows trends in how people are commenting on YouTube videos or track individual terms and phrases on Twitter. Broadcasters might use it to find out what people are saying about a new TV show, based on Facebook updates, Twitter comments, and actual usage data. In some ways, this is better than Nielsen Ratings system because it tracks real-time posts and commentary.

"People can use graphed Web info as a tool for business intelligence, to gain deeper market insights, for research or to keep tabs on an industry, product, and term," says Mark Ghuneim, the CEO at Wiredset, the marketing and strategic planning company that created Trendrr. "By visualizing Web data in graphical ways we can glean more important insights, faster."