I was in the same boat as you and thought that I'd never pay $200 for shades. But then I started having issues with my cheaper glasses losing the polarization, frames breaking, lens getting scratched, etc.. So, I did some research. I went with Costa vs Smith Optics but they are both good glasses.
In cheap polarized glasses the polarization is just a spray that is applied. The first time you get sunscreen on your lens and have to use water or a cleaner the spray is wiped away. So you are left with a pair of non-polarized glasses. With the higher quality lenses the polarization is usually in between the layers of lenses so that it can't be wiped off.
I have the glass lens Costas and in 3 years of daily use I haven't scratched them yet. I've also never had a frame break but if they do Costa will replace them.
Long answer but the short version is: Quality.