Surf rescue group Surf Life Saving New South Wales said in a statement the incident happened around 10:20 a.m. local time at Salt Beach in Kingscliff, located about 500 miles north of Sydney.
"The man received a bite to the back of his thigh and, despite the best efforts of nearby surfers and bystanders who fought off the shark and got the man to the shore and rendered first aid, the man died on the beach," the group said.
The man, who has not yet been identified, was from Tugun, just over the state border in Queensland.
“A shark biologist assessed photographs and confirmed a white shark was responsible for the fatal attack,” the state's Department of Primary Industries said.
NWS Ambulance Inspector Terence Savage said it was a “dreadful” situation for everyone involved.
“When you get a call to attend a shark attack, you never really know the full extent of the damage until you get on scene,” he said. “They did everything they could to try and save his life, but despite their best efforts, were unable to do so.”
Surf Life Saving NWS said it deployed multiple Jet Skis, a drone, and helicopter assets to help spot and identify the shark responsible for the attack and to clear the beach of surfers and swimmers.
Nearby beaches were cleared of swimmers and surfers and will remain closed for 24 hours.
NSW Police told The Sydney Morning Herald it was granted permission to kill the shark, but the animal disappeared from the area hours after the attack.
"Due to concerns that the shark had to be fought off by other board-riders, and that it remained in the vicinity for several hours after the attack, police were granted permission to destroy the shark," NSW Police told the newspaper in a statement.
"Police and local rescue helicopter crews monitored the area for some hours; however, the shark left the vicinity about 1:15 p.m. and has not been seen since. No police firearm was discharged."
NSW Police Detective Inspector Matt Kehoe told reporters the action of those who had tried to save the man was "nothing short of heroic" and said the shark had been spotted "on a number of occasions just off the shore," Sky News reported.
Kingscliff resident Stuart Gonsal had just arrived at the beach ready for a surf when he found out about the fatal attack.
“We came down and we hadn’t got in the water and police were immediately hauling people in,” Gonsal told Australian Broadcasting Corp. radio. “We found out there was a fatal shark attack on the south side of the rock wall. We were going to get in, we’re not going to now for sure.”
The incident on Sunday was the third fatal shark attack in Australia this year.
In January, a diver was killed near Esperance off the Western Australia state coast. In April, a shark fatally mauled a 23-year-old wildlife worker on the Great Barrier Reef.
The Associated Press contributed to this report.