Yes, they know. The reason they took your lure is the same reason they swim right back to it. They are protecting it. I wouldn't throw them in the live well and drive across the lake at the end of the day and then release them, but they will go right back to it if you release it.
Alternate on the