LEBANON (AP) — A white Tennessee judge has apologized after saying in open court that he was “going to work like a regular white man” and not “a slave.”

The Tennessean reports that Judge Haywood Barry made the comment while scheduling a hearing for a black defendant.

Barry apologized Thursday for what he said, The Tennessean reported.

“I’m not working second shift,” Barry can be heard telling an attorney in an audio recording obtained by the newspaper. “I’m going to work like a regular white man, as they used to say. … I’m not going to be a slave.”

The Wilson County general sessions judge told the Tennessean in a phone interview that he was frustrated with scheduling when he made the comment Tuesday.

“It was probably a very inappropriate statement. I’m sorry I made it. I’m ashamed of the fact that something like that would even come out of my brain,” Barry said.

The Tennessee code of judicial conduct prohibits judges of bias, prejudice or harassment based upon race. It’s unclear whether Barry would be reprimanded by the Tennessee Board of Judicial Conduct, which handles complaints about judges, the newspaper reported.