Newsweek reports on the first electoral disappointment for Karl Rove in four cycles and tries to dissect why he wound up so incorrect in his forecasts. Rove, who had famously insisted that news organizations had inferior polling data late in the campaign, saw his predictions of marginal losses explode in an election-day meltdown that those same polls had predicted for weeks:

How did the man they call Bush's brain get it so wrong?

Rove's miscalculations began well before election night. The polls and pundits pointed to a Democratic sweep, but Rove dismissed them all. In public, he predicted outright victory, flashing the V sign to reporters flying on Air Force One. He wasn't just trying to psych out the media and the opposition. He believed his "metrics" were far superior to plain old polls. Two weeks before the elections, Rove showed NEWSWEEK his magic numbers: a series of graphs and bar charts that tallied early voting and voter outreach. Both were running far higher than in 2004. In fact, Rove thought the polls were obsolete because they relied on home telephones in an age of do-not-call lists and cell phones. Based on his models, he forecast a loss of 12 to 14 seats in the House—enough to hang on to the majority. Rove placed so much faith in his figures that, after the elections, he planned to convene a panel of Republican political scientists—to study just how wrong the polls were.

His confidence buoyed everyone inside the West Wing, especially the president. Ten days before the elections, House Majority Leader John Boehner visited Bush in the Oval Office with bad news. He told Bush that the party would lose Tom DeLay's old seat in Texas, where Bush was set to campaign. Bush brushed him off, Boehner recalls. "Get me Karl," the president told an aide. "Karl has the numbers."

It's hard to be too tough on Rove. He had, after all, called three straight electoral cycles more accurately than most of the media, using the same metrics that apparently betrayed him this time. He spent years developing an innovative GOTV activist model that transformed political ground games, and he has built reporting structures that glean data from every stage of the process.

And that's the problem. I use data models on a daily basis in my regular job, and it's very possible to fall in love with the data before determining whether it really tells you anything. Building predictive models takes a number of successes and failures before an analyst can tell what the reports communicate -- or whether they communicate anything at all. Elections by their occasional nature do not provide the repeating tests necessary to substantiate analyses for years.

Rove apparently saw a correlation between organizational efficiency and electoral victory. If one reads the Newsweek description, Rove's measurements showed an organization operating at near-peak efficiency. Republicans got their voters to early-voting options in bigger numbers than before, and more volunteers made more calls than ever. He believed, based on previous experience, that those numbers meant that the GOP would pull more voters to the polls than the Democrats.

Unfortunately, that came from essentially undertested hypotheses, and it failed on two points. First, it failed to account for an improved Democratic GOTV effort, one that borrowed from Rove's own genius. His analysis assumed a static level of Democratic turnout, but a second-term midterm and the Iraq war should have informed Rove that the situation had changed and the Democrats would likely turn out in heavier numbers than before. Second and more important, it assumed a strongly causal relationship between voter contacts and Republican votes, rather than a somewhat correlative relationship.

Put simply, it does make a difference how many people volunteer for the effort and how many voters they contact. However, that does not translate directly into votes in a reliably predictive manner, and the 2006 midterms proved it for the first time. Rove understandably did not see that as clearly as he should, because he had no real data showing that dynamic over the last three cycles. However, it points out the dangers of relying on predictive models without having sufficiently tested them.

What next? This hardly invalidates Rove's GOTV model. Metrics provide important information on any organizational effort. In this case, it showed that the Republicans did not lose because of a failure by the organization and their ground game. They lost because the voters simply wanted to elect someone else. That's outside of the control of Rove and his GOTV structure, and it shows Republicans where they need to put their efforts for 2008.

Trackback Pings

TrackBack URL for this entry is

Listed below are links to weblogs that reference You Can't Fall In Love With The Data:

» Mood Versus Metrics from Scottish Right
Newsweek publishes an article that provides some interesting insights into what Karl Rove saw happening in the campaigns versus the conventional wisdom buttressed by polling data. I was a little disturbed to read that there was a disconnect between Ro... [Read More]

Tracked on November 12, 2006 1:43 PM

» You Can't Fall In Love With The Data from Bill's Bites
You Can't Fall In Love With The DataEd Morrissey Newsweek reports on the first electoral disappointment for Karl Rove in four cycles and tries to dissect why he wound up so incorrect in his forecasts. Rove, who had famously insisted [Read More]

Tracked on November 12, 2006 1:55 PM

» Political science witchery from Kent's Imperative
The medicine man believes in the effectiveness of his dance no less fervently than the modern methodologist in his techniques.... [Read More]

Tracked on November 13, 2006 9:29 PM