The incident happened around 2 p.m. near Indian Head on the eastern side of Fraser Island in Australia's Queensland state.
Queensland Police said the man was spearfishing in the waters off of Indian Head when he was bitten. A doctor and nurse at the scene provided first aid until paramedics arrived and were winched down by helicopter.
The Queensland Ambulance Service said first responders provided emergency treatment but the man, who had been bitten around the legs, could not be saved and died at the scene.
Police said the man died just over two hours after he was bitten.
Fraser Coast Mayor George Seymour said on Facebook it was a "tremendously sad day for our community."
"Our deepest condolences go to this young man’s family and friends," he wrote. "The loss of a young life with his future before him is a tragedy beyond words and we share their sadness and grief."
The attack happened not far from where 23-year-old Queensland wildlife ranger Zachary Robba was fatally mauled by a great white shark in April.
Queensland police were not immediately able to identify the victim in Saturday's attack. They said a report was being prepared for the coroner. Officials also did not say what type of shark was believed to have been involved in Saturday's attack.
At least four people have died in shark attacks in Australia this year.
A 57-year-old diver was killed off Western Australia state in January and a 60-year-old surfer was killed off Kingscliff in New South Wales state in June by a great white shark.
The Associated Press contributed to this report.