An Oregon bar owner must pay $400,000 in damages to a group of transgender patrons after they were asked to stay away, the Oregon Court of Appeals affirmed Wednesday.

The state Bureau of Labor and Industries ordered the penalty in 2013, saying the Portland bar violated a law that prohibits discrimination based on gender, sexual orientation or gender identity.

Chris Penner, owner of a bar formerly known as the Twilight Room Annex, or more widely the P Club, left two voicemail messages for a member of the Rose City T-Girls, a group of transgender customers that frequented the bar every Fright night starting in 2012.

The messages ordered the group to stop visiting because business had declined in the 18 months since the bar became their gathering spot. Penner said people were incorrectly assuming the club was a gay or “tranny” bar.

The Oregonian reports the complaint against Penner was the first brought under the Oregon Equality Act of 2007 and the first to result in a damages award.

"People are not coming in because they just don't want to be there on a Friday night now," he said in one message. "In the beginning sales were doing fine, but they've been on a steady decrease so I have to look at what the problem is, what the reason is and take care of it."

Jonathan Radmacher, Penner’s attorney, said his client might continue the legal fight. He said Penner merely expressed a desire and didn’t refuse service to anyone who came to the bar, drawing a contrast with bakeries that have declined to make cakes for same-sex weddings.

"Originally, the Rose City T-Girls approached his business and asked if this is going to be a problem and they said, 'No.'" Radmacher said Wednesday. "In essence, he was going back to them and saying, 'This is a problem for my business.' We think he's got a constitutional right to make that inquiry."

The Appeals court disagreed that the voicemails represented protected speech. Judge Douglas Tookey said arguments presented by the respondents were “unpreserved, undeveloped or unavailing.”

The opinion does not name the patrons who were denied service, aside from Cassandra Lynn, the T-Girl who received the voicemail messages. The opinion states there are 11 aggrieved people in the case.

Labor Commissioner Brad Avakian praised the court decision, saying in a statement his agency is committed to ensuring people are not denied access "based on who they are or who they love."

Penner closed the Twilight Room Annex in April 2014 and laid off five employees after his bank accounts were seized in connection to the $400,000 judgment, according to The Oregonian.

Click for more from The Oregonian.

The Associated Press contributed to this report.