I don't think these responses are representative of the people I see commuting in Wellington each day. Today on the way home I saw 3 riders in lycra shorts (4 if I count myself) and 2 in baggy shorts (which could well have had a chamois inside - I didn't ask those cyclists about the contents of their pants, so can't tell you!). This is probably pretty consistent with the proportions of lycra vs shorts-wearers I would see most days.
I don't see that many people wearing work or street clothes and I can't remember the last time I saw a 'frocks on bikes' type. Why not? Because bike-specific clothing is ideal for riding in. That's what it has been designed for! It's comfortable, it doesn't flap around or get caught in the moving parts of your bike (no need for trouser clips), there are zips so you can respond to temperature changes while riding, and pockets to carry your pump/work swipy card/whatever in.
I didn't realise there was so much anti-lycra sentiment out there!