I was looking through Short Schrift's archives for something else, and turned up this old post from September 2004, "Elections and the Political Imagination," where I managed to predict both the 2004 and the 2008 Presidential elections.
The incumbent's advantage is always that he or she already has the job. This is why elections -- especially presidential elections -- often turn out to be a referendum on the current office holder's performance -- or, referencing Louis Menand and myself, the performance of the country, or even that of the voter themselves. When things have been going badly, the principle of hope offered by a political challenger, especially hope for positive change, can be a very powerful (and positive) thing. Or you could put the Nazis into power; it works both ways.
An incumbent, then, especially in difficult times, needs to conquer the imaginative space of the electorate. This is the best way of outflanking the challenge posed by an incumbent -- any incumbent. When tough times hit California, Gray Davis responded in a practical, no-nonsense fashion, raising taxes and cutting spending. Voters wanted someone, anyone else, and wound up picking the candidate who (to put it nicely) had the most to offer the imagination.
This is the mistake pundits made when trying to generalize the California recall to the 2004 presidential election. When times are tough, people don't turn to Republicans or throw out the incumbent. They pick the candidate who appeals to their imagination, to their hope that tomorrow might be a better day. They pick Reagan over Carter, Clinton over George H.W. Bush, and (probably) W over Kerry. For the Democrats, picking Kerry over Dean or Edwards was an isolated blip, a moment of self-doubt and misguided Puritan moderation. In the name of electability, they picked a man who lacked the imaginative appeal to ever be elected.