Group living, as opposed to a solitary life, apparently taxes the brain. In fact, sociality has made brains bigger over evolutionary time across numerous groups of animals, a lot of biologists and anthropologists have figured over the years.

This "social brain hypothesis" may pertain in some groups, but overall it's false when it comes to the natural history of carnivorous mammals such as cats, dogs, weasels, bears and their relations, two evolutionary biologists now say.

It's true that bigger brains — in such animals as whales, dolphins, primates and birds — are associated with greater behavioral flexibility and adaptability to new environments. But big brains relative to overall body mass (this ratio is referred to as encephalization) also take a lot of energy to maintain. And some brains tend to get smaller when animals are domesticated or are hunted less by other animals. So the overall trend has been hard to discern.

John Finarelli of the University of Michigan and John Flynn of the American Museum of Natural History in New York sought to work out the relationship on a bigger data set that anyone had previously devised — including 289 terrestrial carnivores, about half of which were fossil (extinct) species. They laid data on sociality, body mass and brain size over the evolutionary tree for all carnivores to investigate the relationship between brain size and sociality.

They found the relationship might hold up among living members of the Canidae family (wolves, foxes, coyotes and jackals), but that's about it. For instance, sociality should be widespread among musteloids (weasels, raccoons, skunks and red pandas) and bears, which have had brain to body mass ratio increases over the course of their evolution. But they aren't. All bears in fact are solitary and the musteloids are mostly "nonsocial."

And while brain size does scale up from small cats to large cats, hyenas are large-brained but also fairly solitary and mongooses are small-brained and usually social.

Overall, Finarelli and Flynn found at least six separate changes in brain sizes for the group across their evolutionary history.

"The idea that sociality played a causative role in the expansion of relative brain size is not valid for this clade [Carnivora] as a whole," the authors write in the May 25 early online issue of the journal Proceedings of the National Academy of Sciences.

Even in Canidae, it could be that increasing brain-size facilitated more social behaviors (rather than sociality causing increasing brain sizes), or perhaps canids evolved larger brains to enhance their social behaviors. Finarelli said he was surprised at how complicated the evolutionary transformations around brain size turned out to be. "I thought at the outset that there would end up being a few instances of increased encephalization, but the number of these, the fact that there were instances of decreased encephalization, and that consistent scaling parameters (that is, how fast brain volume changes as you add body mass) defined the changes in some groups was a pleasant surprise."

So why do brain sizes vary? "That is the $64,000 question," Flynn told LiveScience. "Evolutionary change in encephalization is a widely observed phenomenon, but has been very hard to explain."And what about primates? The relationship between sociality and brain size in this group remains untested, Flynn said.

"Increased brain size may mean different things to different groups, depending on unique evolutionary histories, ecologies, life history attributes etc.," he said. "That is to say, it might be that seeking simple, one-size-fits-all explanations across all mammals (or all amniotes [animals such as birds, mammals and reptiles that give birth to young inside eggs]) has been the problem all along. Nevertheless, seeking general explanations (and then testing them with all available data), within clades or across groups, is a valuable goal of evolutionary studies."

The research was funded by the National Science Foundation, AMNH, the Brown Family foundation Graduate Fellowship and the University of Michigan Society of Fellows.

Copyright © 2009 Imaginova Corp. All Rights Reserved. This material may not be published, broadcast, rewritten or redistributed.