Dinosaurs that lumbered around the ancient supercontinent Gondwana had a warm-blooded neighbor a "chewing machine" with big eyes, excellent hearing and an acute sense of smell, according to a new study.

A team of researchers led by David Krause, a professor of anatomical sciences at Stony Brook University in New York, serendipitously discovered the skull of the primitive mammal (Vintana sertichi) in Madagascar in 2010, while searching for fossils of aquatic creatures. It belongs to a group of elusive extinct mammals called gondwanatherians, which scientists had previously only known about from several teeth and lower jaw fragments.

The newly discovered and well-preserved skull reveals details about the mysterious mammal, which lived about 72 million to 66 million years ago. [See photos of the mysterious mammal's skull]

"The skull is amazingly complete," Krause told Live Science. "What's most remarkable about it is its size. [The skull] is about 5 inches long, which may not seem that large, but for mammals at the time, it was enormous."

The creature likely weighed about 20 lbs., the researchers estimated.

Other mammals from that time period were small only about the size of a shrew or mouse making Vintana sertichi a giant among its warm-blooded peers, Krause said.

Surprising discovery

Krause and his colleagues have looked for ancient mammal remains in Madagascar every year for the past 21 years, but their searches, while thorough, have yielded only dinosaurs and prehistoric crocodiles.

Then, in 2010, the scientists traveled to a remote area that formed Madagascar's coastline during the Cretaceous period. Joseph Sertich, now a vertebrate paleontologist at the Denver Museum of Nature & Science, packaged up a large, rocky block of fossils, and the group brought it back to the lab at Stony Brook University.

Before picking apart the block, the researchers took a CT scan to see what was inside.

"My technician was going through the scans and was pretty stunned to see a mammal skull staring back at him," Krause said.

The group immediately got to work, carefully extracting the skull from the block. The anatomy of the mammal's jaw and teeth suggests that it ate plants, including roots, seeds or nutlike fruits.

It also has large flanges, or projections from the side of the head. "They were almost undoubtedly for the attachment of large chewing muscles," Krause said. "It had to have these very large chewing muscles to deal with [its] hard, abrasive diet."

Most Mesozoic mammals were insectivorous, carnivorous or omnivorous, making this herbivorous mammal's diet stand out among those of its counterparts. [In Photos: Mammals Through Time]

Keen senses

The skull also gave the researchers insight into the animal's vision and sense of smell. The creature's large eye sockets suggest that it had large eyes that allowed it to see well in low-light conditions, Krause said. Moreover, the animal's olfactory bulbs took up about 14 percent of its brain volume, which is a lot for an animal at that time, the researchers said.

"This thing was a sensory machine," Krause said. "It had a good sense of smell."

An examination of Vintana's ear suggests that the animal could hear high frequencies, similar to modern-day bats. In fact, Vintana may representa transition between the inner ears of earlier "mammal-like reptiles" and those of living marsupials and placental animals, said E. Christopher Kirk, an associate professor of anthropology at the University of Texas at Austin.

"Unlike modern mammals, the hearing organ of Vintana was short [and primitive]," Kirk said. "At the same time, Vintana resembled marsupials and placentals in possessing a complex bony support network for its hearing organ and the nerves that supplied it."

Bony support for the hearing organ is linked to improved high-frequency hearing, he said. So Vintana could likely hear these frequencies better than other animals that lack such support, such as reptiles, birds and egg-laying mammals, like the platypus, he said.

The fact that its ear is primitive yet specialized makes Vintana stand out on the evolutionary tree. "It's really a mixture of very primitive and specialized features that make it so unique," Krause said.

Little is known about mammals living during the era of the dinosaurs, but even less is known about those from the Southern Hemisphere, said Guillermo Rougier, a paleontologist at the University of Louisville in Kentucky, who was not involved in the study.

"The most important thing about this fossil is that it's very complete and comes from a place in which we know very little," Rougier said. "It provides a new window on mammals in the southern continents."

The skull is also helping researchers "shake up" ideas about early mammalian ancestry, Krause said. Until now, scientists were unsure how gondwanatherians fit into the evolutionary tree. But the shape of the new find's skull suggests that the gondwanatherians are related to multituberculates, a successful group of rodentlike mammals that lived almost exclusively in the Northern Hemisphere at the time of the dinosaurs.

"This is a whole group of mammals that that we knew nothing about at all 30 years ago," Krause said, speaking of the gondwanatherians. "Now we have a complete skull that really helps us document what the gondwanatherians were."

The study was published today Nov. 5 in the journal Nature.

Copyright 2014 LiveScience, a TechMediaNetwork company. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.