Would you rather watch Stephen Colbert or Neil Patrick Harris? It could have happened: on Wednesday, Harris confirmed rumors that he'd been asked to take David Letterman's spot after the Late Show host stepped down.

On the Howard Stern Show, Harris said that speculation about a possible late-night show wasn't totally out of left field. He went as far as in-person meetings with CBS executives Les Moonves and Nina Tassler. "Well, not in conversation like, about to make a deal," Harris said. "But they called me in and sat me down and asked if that would be something that I would be interested in doing." The conversation apparently happened before Letterman's formal announcement, when Moonves knew Letterman probably would not renew his two-year contract.

But despite the fact that CBS wooed him with a sushi spread, Harris turned them down for both Letterman's spot and the 12:30 a.m. Late Late Show. He says he was surprised that they were pitching him the talk show: "I would get bored of the repetition fast, and the structure is so set that I don't have any interest in doing monologue, commercial, sketch, guest, guest, musical act, good night."

As it turns out, he'd rather do a weekly variety show. And Moonves might still be interested.

[H/T Vulture, image via AP]