South Korea says a North Korean patrol boat entered its waters around their disputed maritime border but backed off after nearly an hour following repeated warnings.

Also Thursday, a senior American diplomat cautioned Pyongyang that its bad behavior would no longer be rewarded.

The naval standoff came amid concerns that the North might try to provoke an armed clash in the area — the scene of deadly naval skirmishes in 1999 and 2002 — to stoke tensions that were already running high after Pyongyang's nuclear test and a barrage of missile launches last week.

The regime has also conducted amphibious assault exercises near the sea boundary and appeared to be preparing for more missile tests, including one believed capable of reaching the U.S. The missile is being readied at the new Dongchang-ni launch site on the North's west coast near China.

New commercial satellite images show the facility is ready for use after nearly a decade of construction. The launch tower and what appears to be construction materials on the launch pad are seen in the images, Tim Brown, a senior fellow with GlobalSecurity.org, said. He speculated the debris may be there to make the pad appear as though it is still under construction.

"The launch pad appears to be operational," Brown said.

The maritime intrusion occurred as U.S. Deputy Secretary of State James Steinberg was in Seoul to coordinate a united response to Pyongyang's belligerence. South Korean news reports said a delegation of senior U.S. officials was working on financial sanctions against Pyongyang.

Steinberg told South Korean President Lee Myung-bak that "North Korea would be mistaken if it thinks it can make provocations and then get what it wants through negotiation as it did in the past. The U.S. won't repeat the same mistake again," Seoul's presidential office said in a statement.

Steinberg left for China on Friday.

Complicating the situation, two American journalists were to go on trial Thursday in North Korea's top court, on allegations they entered the country illegally and engaged in "hostile acts."

North Korea's official news agency said the trial would begin by mid-afternoon Thursday, but nearly one day later, there was no word on the status of the proceedings. A State Department spokesman said American officials had seen no independent confirmation that the case was under way.

Some experts believe the North is using the trial and its nuclear and missile tests for leverage in possible talks with the United States, and that it hopes to win concessions or much-needed economic aid.

The North Korean patrol boat crossed about a mile (1.5 kilometers) into South Korean-claimed waters near their disputed western sea border and turned back some 50 minutes later without incident after a warning from the South, South Korea's Joint Chiefs of Staff said.

It was likely chasing away Chinese fishing boats illegally fishing for crabs in the area, it said.

Pyongyang did not comment on the maritime incident.

North and South Korea technically remain at war because they signed a truce, not a peace treaty, in 1953. North Korea disputes the U.N.-drawn maritime border off its west coast and has positioned artillery guns along the west coast on its side of the border, according to Seoul's Yonhap news agency.

Meanwhile in New York, ambassadors from key nations continued to try to reach an agreement Thursday on new U.N. sanctions against North Korea for defying the Security Council and conducting a second nuclear test. Closed-door meetings have been held since May 26.

Japan's U.N. Ambassador Yukio Takasu told reporters: "We're making our best possible effort ... The issues are serious, require very careful examination at all angles. That's why it is taking time."

Once the ambassadors agree on the text of a draft resolution, it will be sent to their governments for approval. The draft resolution will then be circulated to other members of the 15-nation Security Council for consideration. A vote is highly unlikely until the middle of next week at the earliest.