Looks like it's working Hog Wally. You haven't been misbehaving, have you?
To my thinking this whole thing boils down to a couple pretty simple concepts. Know and do what's right. If a property owner comes around, first -- keep you cool. A hello and a smile would be a good start. Quoting, or worse, arguing floater's rights streamside is a no-win situation, so don't even bother. Playing dumb and asking questions would actually be more to your benefit.
You can't make everybody in your group act right, so don't take responsibility for the everybody -- just worry about yourself and your family. If it starts to escalate give a friendly wave and a smile, then head on down the creek and have a good time.
If the situation dictates, report the landowner's behavior to the authorities at the end of the day.