In my view, the first thing you should consider is where and how you fish. If you mostly fish from the bank versus a boat, or fish rivers versus lakes or ponds, or fish stained water versus clear water, then you should be looking a baits that are good for that particular type of fishing. For example, if you fish from the bank, it would be unusual (not impossible, just unusual) for you to fish water deep enough to actually need a DD22 or 10XD.
Another consideration is, what technique you want to learn or do more of. When I decided to learn to jig fish (it's still a work in process), I did some reading on the TFF from people that I believed knew what they were talking about, exchanged a few PM's with them, and bought a few "starter" baits. After some time (and success), I now have a separate jig box. Or, if you already have technique you like to use, expand on that with different bait colors, sizes, or brands.
A third consideration is, what time of year you primarily fish. Most baits can be used year-round, but some are more effective at different times of the year. For instance, personally, I tend not to throw a lipless crankbait in July, but I throw it a LOT in the fall and early spring. Do some reading about what baits work best at the time of year you fish the most.
If you primarily fish one or two water bodies, you should be able to determine a few techniques that work well for you on that water body most of the time. Expand the bait selection for those techniques by selecting some additional sizes, colors, or brands. Or if you fish with someone occasionally, pay attention to what they use that is successful, and buy a few of those baits. That's how I wound up with a separate box of topwaters and frogs.
I think most of us picture ourselves as a B.A.S.S. or FLW tour pro that should have an arsenal of baits for any fishing situation, when the reality is we most likely fish a limited group of water bodies on a regular basis, and our bait selection should be geared for those water bodies, not for the one time a year we go to Falcon, or what we saw some pro on television using on Okeechobee in March.
My brother-in-law lives on a private lake in east Texas. When he and I go fishing, he takes two rods and, if he wanted to, he could carry all the baits he will use in his shirt pocket. Over the years he has figured out that a black or chartreuse Roadrunner will catch big fish on that lake under any conditions, so why would he carry a box of other stuff he'll never use. When we go at night, he carries two baits: (1) a buzz bait, and (2) something to CRig. That idea is contrary to what we see on television or read in B.A.S.S. Magazine, and, frankly, how we're geared as guys. But it makes a ton more sense to buy what we will use and be effective for us, than to own a bunch of stuff that will never see the water on the off chance we'll "need" it one day. One of the "trends" I've seen on the TFF over the past two years is the number of guys that fish A LOT that have been narrowing their bait selection by limiting the number of colors they own and the type of baits they throw to what actually works for them based on when, where, and how they fish.
This is just my non-professional under-informed
.