The Secret History

The Secret Diary of Desmond Pfeiffer was, in hindsight, a train wreck in waiting. It's easy to imagine another universe where the UPN's - and later the CW's - signature series never made it past its first season. It's hard to see with sixteen year's hindsight, but the first season of Pfeffer was rough - needlessly crass, irreverent, skating very close to sheer offensiveness based solely on its premise. If it hadn't been for surprisingly strong ratings to buoy what was initially a universally panned show, the program may never have survived to a second season.

Even though the network renewed the series, there were still some reservations on the part of the network brass. Series creators Barry Fanaro and Mort Nathan were notoriously problematic, butting heads with the fledgling network's Standards & Practices department many times during the show's first year. The UPN wanted the show, only without the headache of Fanaro & Nathan. The creators were demoted to the permanent rank of Executive Producers, but essentially cut out of the loops regarding all future decisions regarding the show. (Fanaro & Nathan have consistently declined to speak on Pfeiffer since they left the series.)

Enter David Simon. A former journalist who had written the acclaimed book Homicide: A Year on the Killing Streets as well as having produced the book's television adaptation on NBC, the writer found himself in a unique position following the end of Homicide's seven-year run. "I was the most well-loved failure in town," he related to Charlie Rose in a 2009 interview. "I had produced what many considered to be one of the best shows, ever, and six weeks after production wrapped I couldn't get Dominos to take my calls. I'd go to parties where people would line up to shake my hand and tell me how great the show was. I think everyone who watched the show wound up shaking my hand at some point. That respect - sincere respect, don't get me wrong - got me a few meetings, but when people figured out that I was only going to pitch shows in the same vein as Homicide, the offers dried up. Nobody wanted to invest in making another depressing cult show no advertiser would touch."

"I even had a meeting with HBO," Simon explains. "This was after they did OZ - remember OZ? - and they were looking to do more in the way of scripted dramas. At they time they had bought a script off a friend of mine named David Chase - he had this mob drama script called "The Sopranos" that they kept in development hell for a few years. He said they were good people, though, so I went and talked to them. They were nice but they were riding high off Sex in the City and it was obvious they wanted more like that. We parted ways, amicably."

Chase's "The Sopranos" would eventually be freed from HBO and go on to be produced by ABC, where it lasted three years as Jersey Boy, starring Michael Chiklis. But Simon's career changed forever right after his fateful meeting with HBO, at the moment he had almost given up hope. "I had a script treatment for this book I wrote in the late 90s called The Corner - it was going to be a miniseries about crack dealers and addicts in Baltimore. It was the kind of thing that executives get excited about reading, because it's good and they all want Emmys, before they start asking questions like, ' do you think we could get a Wayans brother to play this guy? How are gonna change this to get a happy ending?' I told my manager to stop showing it to people, I was going to give up. I didn't want to go to work on Sex in the City. I knew I could write another book, so I was going to do that. But not a week after I told my guy to stop selling that script, he called me up and told me there was an offer I needed to hear."

Initially wary of any further meeting, particularly at the UPN - a new network with a less than stellar reputation - Simon agreed only to humor his manager. "I thought it'd be another one of those where I listen to people telling me how great Homicide was before they tell me they want me to do something completely different. But it wasn't like that at all. They spent about 30 seconds blowing smoke up my ass, but then they came out and made me the offer."

The offer was simple: with Desmond Pfeiffer the UPN had a semi-popular show with a terrible critical reputation that had just lost its creators. The first season was not a creative success. The ratings were good but not great, and the feeling was that they had nothing to lose. They offered to give the show to Simon wholesale - make him producer and show-runner, with no restrictions. "The feeling at the network at the time," says TV historian Kathleen Olmstead, "was that they had nothing to lose. The controversy had kept the show on the air for the first year, but the prospects of a second year on the same material were not promising. They saw Simon as a man with a critic-proof pedigree who was also at the end of his rope in television. It could have backfired, it could have imploded. But it didn't."

In a 2006 interview with Vanity Fair, Simon recalled the night after he first met the executives at UPN. "On first - hell, second and third blush, it was a stupid idea. It was a suicide mission. It was a terrible idea for a show, and I had never done anything even remotely funny in my career. I told my manager I wanted to turn them down that night, but he asked me to sleep on it. I went back to my hotel room and switched on the TV. It was spring of '99, so everyone was still talking about the damn impeachment. There was something on PBS, a Frontline documentary about it. I wasn't paying any attention at first, just sitting in bed and daydreaming. But after a minute I realized I was looking at the TV, really looking at the drama being replayed from the senate floor, all those senators and congressmen and (Chief Justice) Renquist looking like a Gilbert & Sullivan character. Pure political theater, completely content free, but dressed up to look as if it were the revelation of some great new chapter in American democracy. And then I remembered, really out of the blue - Andrew Johnson had been impeached, too. Right after the Civil War, right after Lincoln's assassination."

Before he knew it, he had pulled out a stack of videotapes from his meeting with the UPN executives earlier that day. "The jokes were bad, but the actors were pretty good. Chi McBride especially, seemed above the material. But it all clicked for me with Dann Florek." Florek, a television veteran with experience on L.A. Law and Law & Order, gave the role of Lincoln a unique gravitas that belied the dirty jokes and cheap innuendo he was forced to sell. "I knew from just a handful of episodes that I could jettison just about everything else on the show, as long as I kept McBride and Florek," Simon continued. "I called the network back first thing in the morning and said yes, I'd take the job."

True to their word, the UPN did not interfere with Simon as he tore the program down to its roots and rebuilt almost from scratch. The only thing left were the sets and the actors. He fired the entire writing staff, replacing them with Homicide veterans and old hands from shows like Taxi and M*A*S*H, writers who had produced the last good sitcoms Simon had enjoyed. "He got a lot of people who hadn't worked in TV for a while," remembers Vince Gilligan, a former producer and writer for The X-FIles and later one of Simon's assistant producers. "We didn't have a lot of money, but he managed to get a lot of people who wanted to do something more interesting than Friends." In addition, the kept a revolving door of historians and consultants on the show, to guarantee historical accuracy.

When The Secret Diary of Desmond Pfeiffer premiered its second season in Fall of 1999, the show instantly became a critical darling. Ratings were stable. Although the show was in no danger of becoming a runaway hit, its respectable showing had vindicated the UPN's gamble. At the following year's Emmy awards, at which the show was nominated for six awards and came home with two (one for Florek's performance as Lincoln, another - the first of many - for Simon as writer), Pfeiffer had been heavily favored to win best comedy. They lost to Will & Grace, but most critics derided the choice. It would not be the show's last trip to the Emmys.

The first two years of Simon's Pfeiffer are now considered some of the most important television of the era. In a period, post-Seinfeld, when most had given up on the sitcom as a viable creative format, Simon had succeeded in making the most unseemly of vessels into a vehicle for creative renewal. The final product had little in common with Friends or Everybody Loves Raymond: Simon's hybrid show more resembled a 30-minute Playhouse 90, with dashes of Norman Lear's All in the Family. It was smart, literate, hyper-verbal, but could still be devastatingly funny.

Simon remembered a visit to the writers room by Lear in the Winter of 2001. "He came down to visit, because he said he was a big fan. Well, obviously, it was a big day. He sat in the corner and watched us bat around some ideas - I think we were trying to write some material about Sherman's March. We were getting to the end of the war by then. He got up and left after about forty minutes, without saying anything at all. I excused myself and ran down the hall after him. I asked him, 'what's the matter? What are we doing wrong?' and he replied, 'Not a thing. I needed to get out of their before I joined in and started giving you free ideas.'"

Ratings were still healthy, but after two years Simon's run they had begun to erode. The show returned to the 2001 Emmys and won the Best Series trophy they had been denied the year before. Florek again won in the Best Actor category, a bittersweet accomplishment since he had two years' running beaten out his costar McBride for the same award.

Given the acclaim, UPN seemed to be in no danger of canceling the show. Nevertheless, industry watchers were surprised when Pfieffer was absent from its Fall 2001 schedule. Although the series remained in production, it had been pushed forward to Winter in favor of game shows. "That was rough," Simon recalls. "There were a couple years where all the networks gutted their scripted programming in favor of game shows and reality shows. Faddish crap. We had a few more months off, is all, because they wanted some Who Wants to be a Millionaire? clone in the slot for Fall sweeps."

Production for Season 4 of Desmond Pfeiffer had only just begun in early September, 2001. The actors had returned for read-throughs and a pile of scripts remained in various stages of development. The 2001-2002 season posed a significant challenge to the creators, even before the events of 9/11. "The show had been dancing around history from the first episode," Olmstead relates." Everyone knew there was only so long the Civil War could continue. It had a conclusion and everyone knew what that conclusion was. The second half of the 2000-2001 season had focused on Lincoln's second election, ending with his inauguration and segueing into where they were supposed to be at the beginning of Season 4 - the last days of the war. The plan - which had been solidified in meetings with executives the preceding spring - was that Lincoln would die during spring sweeps. All the episodes written for Season 4 were building to that. I've read Simon's draft of the original death episode, and I believe they've been circulating online for a while. It was very good, even in outline."

But that script would never see production. Simon woke on the morning of September 11th to the news that the World Trade Center and the Pentagon had been demolished by acts of terror. "The phone was ringing. I didn't want to answer it, it was still really early. I didn't have to be in work until later. But my family was still on the East Coast so they were awake before me . . . finally I answered the phone and they just said, 'David, turn on the damn TV.'"

Like everyone else, Simon spent the day of 9/11 glued to his TV. Work was canceled. Every channel overrode regular scheduled programming in favor of around-the-clock news coverage. But after eight hours of watching the horrible images repeated on an endless loop, Simon turned the TV off and went to work. Three days later he appeared at the empty set. The only other staff were a handful of PAs and technical crew hovering around the TV. He went into his office and started making calls.

"No one was working that week," McBride remembers. "We were scared to leave the house, that's what it was like then. You couldn't get on a plane. Everything was up in the air. So I get a call from David on Friday morning, I'm expecting him to tell me we're postponing the start of filming for at least a little bit. But he says, "no, Chi, you need to come in today. Now. We have to get to work."

"I remember distinctly, we all got there around two or three. None of us were expecting to come into work that day. We had gathered on the main stage, and we were sitting there shuffling our feet and laughing nervously when David came tearing in from his office. He and his assistant were both carrying giant bankers' boxes filled with scripts. He passed them out without saying a word but he didn't need to, we opened the first page and saw exactly what it was. None of slept much for about a week, until it was done."

The scripts, for Simon's hastily written two-parter, completely changed the plan for Season 4. Instead of building slowly to Lincoln's assassination, Simon's new plan began, literally, with the fatal shot. It's one of the most famous opening sequences in television: twenty seconds of darkness and silence, followed a loud gunshot and screams. Then a tiny light slowly becoming larger, a small figure carrying an oil lamp down a darkened hallway. A tap on a door. Rustling of bedclothes. McBride's Pfeiffer peers out of his dark room to see the lady's maid, crying and shaking in the hallway. She utters just three muffled words before the show cuts to the opening credits - "He's been shot!"

The two-parter "Our American Cousin" / "Oh, Captain," broadcast two weeks to the day after the attacks, was immediately hailed as one of the most significant television programs in history. UPN had been ignorant of Simon's intentions practically until the day he delivered the finished episodes to the network. "It was a complete surprise when he walked in with those shows," remembers an executive who was present when Simon presented the episodes. "Everyone was still in the doldrums, we were all still panicked and running news and repeats. He walks in the room, puts the tape down, and just says, 'we're running this.' We didn't even know he had been working. I called everyone down the hall, put the tape in the machine, and I swear no one in that room, a room full of TV executives and advertisers and secretaries and interns, no one said a word for 45 damn minutes. But when the show was over we were all crying."

The Season 4 premiere of The Secret Diary of Desmond Pfeiffer became the highest rated scripted program for the 2001-2002 season. Although the audience was small the first night, due to the lack of advertising, it was already being hailed in the papers on the morning of September 26th as an instant classic. A rebroadcast later in the week octupled the original's ratings.

Pfeiffer had made the jump from being a critics' darling to becoming a touchstone for American media. Teachers started showing the episodes in history classes across the nation, in reference to both the Lincoln Assassination and the September 11th attacks. Simon's stroke of genius - using our great-great-great-grandfather's national trauma as a means of better understanding our own contemporary trauma - earned him another Emmy, as well as a personal compliment from none other than the President of the United States. His relationship with George W. Bush would not, however, remain so cordial.

The one person seemingly left behind by the making of "Our American Cousin" / "Oh, Captain" was Lincoln himself, Dann Florek. Although he had known all along that his work as Lincoln had a set expiration date, he had expected a full season to prepare for his death scene. Although he was disappointed by the contents of the episode - beginning, as it does, moments after the President is shot - he made one more appearance, in the final minutes of "Oh, Captain," appearing before Pfeiffer's as a ghost to reassure his long-suffering friend. Florek's last words as Lincoln, before fading away forever, quickly entered the lexicon alongside some of TV's greatest quotes:

Although we suffer now, we do so with the understanding that suffering and patience are the great materials by which we construct our character, both as a nation and as a people. It is easy to strike in vengeance, but it is difficult to do so without destroying yourself. A sacrifice made in the name of perpetual strife is a sacrifice made in vain. It is most difficult to show mercy in triumph, but you must. You simply must.

The show retained its place atop the ratings for many years. As the contemporary news continued to report catastrophe and paranoia, in the midst of rising calls for wars, Pfeiffer set about to tell the tragic story of Reconstruction with a candor and violence that would have been unimaginable just a few years earlier. Just as the second President Bush led the United States into ill-fated foreign wars, Pfeiffer methodically showed the failure of Reconstruction to effect permanent change in a war-torn region.

Simon had never made a secret of his antipathy towards the president, but Season 7 became the moment of the show's fiercest critique. Although Simon had initially imagined the Johnson impeachment as a corollary to the absurd Clinton impeachment, circumstances had changed significantly by 2005. According to Hendrik Hertzberg, writing for The New Yorker, "The impeachment of Andrew Johnson has become, for David Simon, a dry run for the hypothetical impeachment of George W. Bush. In Johnson's betrayal of Lincoln's spirit of emancipation, Simon see's Bush's failure to use the 9/11 tragedy as anything more than a malicious, partisan power grab. All the anger, all the righteous indignation over the Iraq and Afghan wars, came out in the unlikely vehicle of a half-hour television sitcom. Although, it must be noted, Pfeiffer long ago gave up any pretense of being a comedy." Under Simon's pen, both Bush and Johnson were callow pretenders, promoted by accident of history to the most important job at the worst possible moment, figureheads completely unprepared - or unwilling - to exert themselves positively in a moment of unparalleled crisis.

Although Pfeiffer had remained strong in the ratings since 2001, the show's increasing politicization in the final years of the Bush administration began to erode its audience. After Johnson's impeachment, McBride's Pfeiffer left Washington altogether, resigning from his hard-won appointment in the gutted Freedman's Bureau in favor of an low-level appointment in the Department of the Interior. With the events of the Grant administration set in the background (although not altogether ignored), the next few seasons of Pfeiffer stepped back from politics and focused on post-Civil War industrialization across the country, particularly the systemic corruption at the heart of the newly-constructed intercontinental railroad line. A much-heralded storyline in Season 9 focused on the connections between the abuse of Chinese immigrant railroad workers and the corrupt regulatory environment that encouraged their exploitation, with a focus on the mechanics of the late nineteenth century opium trade, returning Simon to the subject matter of his aborted script for The Corner. Although critics continued to laud the show for its consistency, ratings continued to drop as the subject matter became increasingly esoteric.

In 2009, ten years after receiving the offer to take charge of the sitcom, Simon stepped down. In his place, the CW (the network formed in the mid 2000s from the combination of the UPN and WB networks) appointed Matthew Weiner. Weiner had worked with David Chase on Jersey Boy before coming joining the Pfeiffer crew as head writer under Simon in 2007. All eyes were on Weiner to fail in Simon's wake: given the cultural prominence of Simon's Pfeiffer, Weiner faced universal skepticism.

"I didn't want the job," Weiner told The Atlantic in 2013. "It was a poisoned chalice. David Simon may not have created The Secret Diary of Desmond Pfeiffer, but in any every way that mattered it was his baby. He had turned it into something special. I didn't think anyone should take it over. Simon came to my office - that was interesting in itself, I usually went to him - he came into my office and told me that he and the network had agreed his time on the show was done. The first thing I asked him was if he had any ideas for a final episode. He shook his head and said, no, they want it to continue."

"I laughed and told him I was quitting, and that I didn't think anyone else would stick around either if they were throwing him under the bus like that. But then he said, no, Matt, no. They want you to do it, and I want you to do it too."

With Simon's advice still ringing in his ears - "make it your own" - he planned a radical departure. The years since Pfeiffer had left Washington had been creatively fertile, but the lack of geographic focus has perhaps alienated an audience accustomed to a stable supporting cast and consistent setting. At the beginning of Season 12, Pfeiffer retired from government work to life in New York City in the mid-1870s, during the Grant administration and the waning days of Reconstruction. After years in government service, Pfeiffer had managed to amass a significant amount of money, so he settled into a semi-retirement as a businessman and real estate speculator.

"One of the great shames of Reconstruction," Weiner said, "and this isn't just in the South but across the country, is that during that period there really was a fair amount of success in the black community as a result of some of Lincoln's policies, many of which were eventually reversed by Johnson, before evaporating entirely. So we had a small black middle-class coming into its own for the first time. Pfeiffer was a man of the world, a British national who had served the US government for almost fifteen years. So wouldn't it be interesting to see what happened to someone like Pfeiffer in this environment, in a place - late nineteenth century New York - where there was a lot of money to be made?"

Weiner's Pfeiffer became a sumptuous period piece, a close look at a small group of what would eventually become Manhattan's richest families in the years just before the height of the Gilded Age. Set down in this milieu was Pfeiffer, consistently underrated because of the color of his skin but nevertheless managing to become a rich man in his own right through judicious investments. Weiner's soft relaunch was well-timed: the next year would see the arrival of the massively successful British import, Downton Abbey, an arrival that catalyzed a new fascination with fin de siecle period drama for American audiences. The CW had again gambled wisely: Weiner's shift from the overtly political tone of Simon's run was a perfect fit for an audience who returned to Pfeiffer looking for a period drama that critics hailed as a perfect mating of Edith Wharton and Charles Chesnutt.

If Simon's later years had focused almost exclusively on Pfeiffer's character as he traveled across the country, Weiner's era introduced a new large ensemble cast. The primary action of the thirteenth and fourteenth seasons was the conflict between Pfeiffer's American cousin Terrence (played by another Jersey Boy alumnus Michael K. Williams) and the enigmatic Don DuRapier, played by Jesse Williams. "I think audiences respond to Don's character because we all relate to mysteries," Simon asserts. "We introduced this man, extraordinarily competent and suave, with a complete blank slate for a past. People like him, they want to find out more about him. Set him off against Terrance, who at first the audience hates, because of what he does to Desmond's daughter - that's the conflict. Pfeiffer's inability to keep these two powerful, willful people from colliding is the tragedy of his later life." Pfeiffer was once again a hit. The climax of the Terrence / Don war, "Indian Summer," saw the series' highest ratings since 2001. The revelation that Don DuRapier was not in reality a white businessman from Pennsylvania but an African-American former slave from Maryland who had stole a white man's identity during the Civil War and had successfully "passed" in business and society circles since 1865 won the show another Best Series Emmy, its first in six years. "We have a responsibility to history," Weiner said during his acceptance speech at that year's ceremony, "Desmond Pfeiffer is one of the great American stories because it has never been afraid to show us ourselves at our best and at our worse."



But the show couldn't continue forever. After dodging death multiple times, the CW announced last Spring that Chi McBride had opted not to renew his contract, and that the following season - the show's seventeenth - would be its last. In an interview from earlier this year, Weiner stated, "when Chi told us he wanted to move on - I don't think there was any doubt as to whether or not to continue. This is his show, as much as it was ever David's. Desmond Pfeiffer has been America's conscience for sixteen years now - longer than anyone ever could have guessed. The idea of killing Desmond and continuing with Terrence in his place, or one of the sons - it never even crossed our mind."

But even as the show prepares to take its final bow, the country won't be done with Pfeiffer anytime soon. "When Barack Obama said in 2008 that his favorite TV show was The Secret Diary of Desmond Pfeiffer, it came as a surprise to no one," Olmstead says. "Even given the controversy of the second half of Simon's run, it was still part of the national conversation. Funny thing is, after spending eight years excoriating the Bush administration, people thought that the post-Obama Pfeiffer might be kinder and gentler towards the powers that be. Well, we know that didn't happen, and the funny thing is that whereas during Bush's administration Simon's attacks were seen as largely partisan, by the time he started using Grant's disastrous presidency as a means to criticize what was already becoming an ill-fated, morally corrupt administration in the here and now, people from both sides of the aisle paid attention. Obama regretted having said that in 2008 because he had to eat his words during Simon's last year on the show."

Speaking to the show's impending end, Weiner is characteristically mum. "I have an idea for how it's going to end. I've discussed it with David, actually. He likes it. It might not be the ending people are expecting, but after seventeen years we are bound to disappoint someone."