A story about the myth I was told was, Way back in the wooden ship days, when they would transport Bananas to England and what not they kept having mysterious deaths aboard the ships. Turns out there is a very deadly spider that nests in the banana trees and ship hands would get bit and die. Of course they didn't know that so they just thought bananas were bad luck.

Not sure if that is the reason for the myth, but it sounds plausible

My luck is equally bad with and without bananas.

I was fishing with my son down At Falcon with guide Billy Tompkins. We hadn't caught any fish and while moving to the next spot I pull out a banana to eat. He just about threw me out of the boat & then tells me the superstition. So I eat the banana, throw the peel out into Mexico (bio-degradable & natural)and proceed to catch 4 fish that go almost 7 pounds apiece.
now; was it because I got rid of the banana, because I thought once you brought it on the boat the fishing "day" was shot, was it because we changed spots, was it because the banana thing is just hogwash. to this day I do not take a banana on the boat with me, but I do eat 2 of them every morning I am going fishing.