In the online article Tales from Responsivenessland: Why Linux Feels Slow And How To Fix That it is suggested to lower your swappiness in order to obtain more responsive applications in Linux. The main reason for this suggestion was "Screw data, prioritize code". So application's code won't get swapped away and is quickly accessible when the user needs it.
While this makes sense to some extend for a desktop, I'd like to remark that not all code in memory is being used. All this unused code takes away space which could have been used as disk cache. So I'd rather see to have the code swapped out which is not used anyway. To achieve this, I always have my swappiness set to 80 to force pushing that stuff to the swap space.
The most common drawbacks I've heard of having a high swappiness is that it takes 10 seconds to get your Firefox back in the morning when you left your PC on during the night. This is because cron jobs like updatedb and prelink fill your disk cache during the night and make all code get swapped away. But those 10 seconds are nonsense to me, one can wait that time, right? I would understand the argument if it would take a minute or an hour to get your applications back, but 10 seconds is still very bearable. And during the day, all applications you regularly use remain in memory, unless you're doing some heavy operations.
So for me a high swappiness works just fine, it simply keeps the code I don't need out of my main memory.