Yes, you do need to go crazy, although it depends on your views about leathers in the first place. I didn't think i needed leathers when i got into riding, as my dad and my brother didn't wear any gear when they were riding back in asia. Took the course at PRS and they taught me how leather protects, etc...but still, i was not 100% convinced. Only reason i bought leathers was because "xxxx" told me i should do it.
So i went for my road ride during the windy/rainy day about a week or two ago, and had to make an emergency stop. I pulled the front + back brakes, but it resulted the bike in loosing traction. I skidded till stop, but i was not able to regain balance, so the bike fell on my right toe. It felt like i torn/bruise a muscle, and was painful to walk for about 4days. If it wasn't for my riding boots, it would of been way worse, my toes could of been gone. The way i see it is...my riding boots has a mark in the toe part of the boot, and it looks pretty brutal. Now picture that amount of damage on lets say, Converse or some Nike shoes, now THAT would of been a disaster.
My point is you only got one body parts (yes you have two arms, etc, but you get what i mean), and its better spending that $1000 or so on gear than to look back saying "f*ck, if i had gear i would of..."
Oh and if you can't afford leathers, then suck up your pride and buy used