I have often written about the suspect adjustments made to the sampling on CBS polls, which skew the results so that they put the Bush administration and the Republicans in a harsher light than the raw data indicates. While the new Washington Post/ABC poll doesn't appear to employ the same "weighting" technique that CBS used to shift its demographics to give the Democrats a four-point edge that its sample didn't support, the Post/ABC sample itself appears very suspect.

First, the results of the poll tend to show that the GOP has absorbed the brunt of frustration with the lack of progress on legislation and judicial nominations. Dana Milbank and Claudia Deane also note that the poll shows for the first time that a majority of adults do not believe that removing Saddam Hussein has made them any safer:

For the first time since the war in Iraq began, more than half of the American public believes the fight there has not made the United States safer, according to a new Washington Post-ABC News poll.

While the focus in Washington has shifted from the Iraq conflict to Social Security and other domestic matters, the survey found that Americans continue to rank Iraq second only to the economy in importance -- and that many are losing patience with the enterprise.

Nearly three-quarters of Americans say the number of casualties in Iraq is unacceptable, while two-thirds say the U.S. military there is bogged down and nearly six in 10 say the war was not worth fighting -- in all three cases matching or exceeding the highest levels of pessimism yet recorded. More than four in 10 believe the U.S. presence in Iraq is becoming analogous to the experience in Vietnam.

In terms of his approval/disapproval rating, both actually increased since the last polling; approval went up by one point, and disapproval by two, with undecideds dropping to the lowest point since the election. His numbers on Iraq have not appreciably changed since the first of the year; the 41% approval is the median since the December poll. His economic policy approval is unchanged since April, and the only place where he looks to have lost significant momentum is on the overall global war on terror, where he still gets a 50-49 advantage. He also has a 46-44 advantage on how he's handling judicial nominations, with a fairly large 10% undecided, indicating that the issue has not exactly caught fire with the populace. And despite the consistently low numbers for Bush on Social Security, support for his proposal to use options for personal investment rose since April by three points to 48%.

The numbers on Congress are equally vague. Its approval rate remains almost unchanged since October 2003, apparently the last time the question was asked. The last time the Post bothered to ask about Republicans in Congress was in 1999, and the GOP actually gained a point since then, to 42%, which seems odd when one considers the electoral gains made since then. Democrats, however, have lost nine points in the same time frame.

However, once again, the details of the demographics tell an odd tale. Look at page 21 of the PDF containing the polling data. For question 901, which asks about party identification, the numbers seem reasonable:

Democrat: 30%
Republican: 31%
Independent: 34%
Other: 4%

On question 904, however, the results show a serious skewing problem which should have rendered the entire enterprise moot. Question 904 asks respondents to identify to which party they lean the most, and the results seem out of whack:

Democrats: 48%
Republicans: 34%
Neither: 16%

The survey data does not say if all respondents were asked the question or merely the ones that identified themselves as independents in Q901. If this was asked to all respondents, then a 14-point gap between the party so obviously flies in the face of reality that all of the poll's results have to be seen as unreliable. If Q904 only was asked of independents, it still looks rather unusual given the election results of the last three cycles. More to the point, it results in a strange distribution when combined in the report:

Democrats: 48%
Republicans: 45%
Independent: 5%
Other: 2%

If the question was asked of all independents, then 48% of 38 (including those who answered Other, as that number also gets changed) will give the Democrats an additional 18 points. However, the 34% of 38 should only give the GOP an additional 11 points, pushing them to 42 and not 45, leaving an obviously faulty six-point gap. Some kind of weighting got applied to this poll, and it's unclear what method was used -- but the overall result shows that the sample doesn't really reflect the current political climate accurately.

That may not be the Post/ABC poll's fault. After all, polling samples depend on who answers the phone and who is willing to complete the questionnaire. However, reporting this as an accurate measure of public response and hyping the ambiguous trends here into disaster looks very much like what CBS tried with their own polling. It's unreliable, and in this case when one looks at the actual poll numbers and trends, entirely unremarkable.

UPDATE: Ankle Biting Pundits has more along the same lines.

Trackback Pings

TrackBack URL for this entry is