Pick up raids for VD maps don't happen that often and preference is given to tanks with better than DM gear and Main Tank has to be well geared way above DM to survive.
Lots of people have done VD maps and other stuff without being over-geared, it's certainly possible. Doing VD maps for the first time will be hard for a paladin or any other character and you'll probably fail some. The entire game is designed to be challenging for every class.
I don't know if I would say the game is designed for raiders. If you ignore a portion of the game's content (grouping, raiding, or questing) you will run out of things to do sooner.