kayak cart goes a long way too! Dragging mine throught the woods was very labor intenive!

Keep it simple at first with your purchases. For the paddle I always recommend purchasing the best you can budget for. At the very least purchase a paddle with a fiberglass shaft and not aluminum. A comfortable PFD designed for kayaking (it'll have narrow shoulders and flotation higher on the back to allow room for a kayak seat) would be next in order. For safety a whistle is inexpensive (and contrary to popular belief is not a legal requirement on lakes and rivers) and easily affixes to your PFD. Must of us also attach a stainless steel knife to our PFD as well.

If you plan on fishing before sunrise or after sunset you'll need to either purchase a 360* all-around white light or make your own (Google Homemade kayak light) and it's also a good idea to have a head lamp.

A 5-10 liter dry bag is a must as well for wallet, keys, and cell phone.

From there I would keep the purchases minimal. A milk crate for the tankwell can be bought at the Container Store or you could politely ask for one from your local convenience store. Attach rod holders or cut PVC pipe to fit the crate.

Depending on your style of fishing an anchor is good thing to have. The folding ones are cheap and easy to find and do a ho-hum job. The bruce style anchor is much better at holding, but is harder to find and typically costs more. Another option is to purchase a 5lbs dumbbell, though it won't hold in place it super windy conditions. Again, depending on your style of fishing a drift sock is great to have. The only time I don't have one on my kayak is when I'm fishing a river.

A lot of kayakers utilize leashes for paddles, rods, etc. I'm not one of them, but it is a personal choice. I also keep a dedicated pair of pliers with each kayak because I hate it when I don't have them. If you plan on keeping fish a small landing net and ruler (like the Hawg Trough) are nice to have. Kayak carts are great if you have a long way to go to the water from where you parked.

Before you purchase items that you're going to want to permanently attach like rod holders or anchor trolleys it's best to go fishing several times before bolting those items down. It happens enough that anglers bolt a rod holder down before spending time fishing only to realize it's too far forward to reach or it interferes with their paddling stroke.

I agree Neumie keep it simple. I always have a tendency to get more stuff than I really need and go overboard buying things I think I need only to never use them. In kayak fishing you have a limited amount of room and can fill it up fast with stuff you don't need. I would get the crate, paddle leash and after a trip or two you will figure out what other items you want to add. Good luck and post some pictures of the fish you catch!