Regarding "What's so bad about Caps Lock?", I can think of a few things…
1. It's an anachronism. Much like the QWERTY layout and the staggered placement of keys, Caps Lock hearkens back to the days of manual typewriters, when the Shift key literally moved either the platen (floating shift) or the type elements (basket shift) to a higher position. The Shift Lock key (as it was then known) was designed to relieve stress on the fingers from holding down Shift keys. With the advent of electric typewriters, the need for this function became less primal, as Shift keys no longer required more force to hold down than any other key. But as with all things, the inertia of success pretty much dictated that the key would remain on the ISO-standard keyboard to this day, and would continue to be placed directly over the leftmost Shift key.
2. It's relatively useless in English. Maybe if it had remained a SHIFT lock, it might have still had a function of sorts (the symbols over the number keys would be easier to type rapidly, which can be more useful that all-caps text). As it is, the only function of Caps Lock is to ensure you type only capital letters. As noted in a comment above, this is useful in certain programming languages, albeit mostly of an older vintage (COBOL, Fortran, etc.), but for those who primarily use a keyboard to write in English it seems like a boondoggle.
3. It's too easy to hit by accident. Granted, this can be attributed more to its placement on the home row (or sloppy typing habits) than any inherent fault of the Caps Lock function itself. Still, it's just a bit too easy to hit Caps when you're trying to hold down the left Shift key or Tab, and making matters worse is that there usually isn't a visual or aural cue that you've hit the key; in most cases you only notice it when you reach for the sHIFT KEY AND YOUR TEXT STARTS COMING OUT LIKE THIS, BECAUSE YOU HIT cAPS lOCK INSTEAD. Quite annoying, as you can see.
4. Ergonomically, there are better keys that can be put here. The most common "replacement" for Caps Lock is to have a Control key there instead, as was the case on some early computers and UNIX workstations. I also like the idea of a secondary Backspace or Enter key in place of Caps, which would speed up typing and correcting spelling errors (the latter of which currently requires a somewhat awkward jump). Strictly speaking, there's little to no reason (other than typewriter tradition) for Caps to be where it is, and every reason for other, more generally useful keys to be in that position.