I'll second bumping up to the 175XT if you can swing it. The extra length and width both make it more stable on the water. Also, before you decide, go out in some boats and see how they feel on the water if you can. If you are thinking new, the dealer should be willing to take you out in a couple of boats so you can see the differences. If used, ALWAYS take it out for a test ride before you buy. Can't stress that enough. Starting a motor in a trash can is simply not the same as running the boat out in the water for a few minutes.
My final advice is go late model used. The big depreciation hit is those first 3-5 years. Find a clean boat that is 3-5 years old and you can save 30-50 percent off new price.
_________________________
Tracker 175XT w/50HP Mercury