Don't do the math on that one. You will never come out ahead if you look at it on a pound vs dollars spent ratio. If you compare it to what you typically spend on a night on the town with entertainment then you have a better comparison. Commercial fishermen will usually catch fish more efficiently than a recreational angler so it will almost always be cheaper to just go the fish market to buy your fish (but where is the fun in that!).
Sadly, it wasn't all that long ago where a 12hr partyboat trip was an economical way to get a decent amount of fish. For $65 or $75, I could go out and catch 4 red snapper from 5-10 pounds each, 1 amberjack of 20-30 lbs, and a string of beeliners. There's also the incidentals, fuel to get there & back, etc., which made the cost of fishing about $250 for the day. I could realistically expect about 15-25 lbs of fillets at home.
It still ended up costing more than buying fresh fish at the store, but $10/lb-$16/lb was not ridiculous. Now that same partyboat trip yields 5 pounds of fillets for about $300.