Yep, if you pull up your marker buoy and move off, then to me it's an indication you're done with that spot, and it's possible other anglers may see that and move in. It's that time of year when it gets crowded out there, no use in getting upset with people that are "too close". If you're a good angler, not saying that you aren't, you'll still catch your fish even with others close by.