I just got my own fly rod this past week after messing with my brother in law's last weekend on the North Fork River. Took my friend out, who is also just learning to fly fish, to the spot you mentioned on the James. We didn't even make it to the Sunshine bridge, though I think I could hear it.
Fishing was really quite good compared to what we're accustomed to, especially considering we are fly fishing novices. My friend caught 3-4 smallies and about as many panfish as he wanted. This was after just "practicing" getting his line where he wanted it to go - and failing - for about 3 hours and getting very frustrated (prior to that he had only practiced in my backyard for an hour). His smallies were comparable to what we normally catch on the Finley, which we had been fishing almost all of June. He was using a white wolly bugger before he lost it.
I was also catching all the panfish I wanted. It is so satisfying to cast a fly exactly where you want to, and even if you don't get a strike immediately, entice the fish to bite. It just feels like you're much more in control of the fish's behavior while fly fishing. And while I am by no means a good or even decent caster yet, I'm amazed out how well you can place bugs pretty much precisely where you want to (assuming there are no obstacles behind you) - it's a level of precision I never get with my open reel. I was using a black bugger that had some wing looking protrusions on it, before I, of course, lost it. I then moved on to an olive bugger which I also lost (I'm almost always losing these due to the whipping of my cast, not due to rocks, trees, or fish). Biggest lesson of this trip - check your knot and check it frequently! It's a lot different and much more difficult for me to tie with low test tippet, and it also seems to be really temperamental (e.g. getting wrapped up in itself and becoming all twisted and then just staying that way despite your best efforts to straighten it - anyone have any advice regarding this?)
After the panfish I decided to try and catch this infamous 2 3/4 lb smallie that's been alluded to! So I put on this big, black, elk hair wolly bugger concoction with an olive tail. Man it was a chore to try and cast, but it didn't take long to produce. Five minutes later I had a 16 inch smallmouth on my line (measured to where it measured on my pole later). I've just recently re-discovered my love for fishing, and haven't really fished for bass except this past month, add to that the newness of the fly rod, and I wasn't quite sure of the significance or insignificance of whatever was on the end of my line, but I knew it was bigger than what I was accustomed to. The fish fought for about 5 minutes, and I was being very delicate not to break the line, she jumped twice, and it wasn't long after that I was yelling to my friend that it was the biggest thing we've caught on a stream this summer. The fish wrapped around my legs more than once (not used to that 9 foot leader/tippet!) but I never lost it, thank God. After I got her off the hook she settled down by my feet for a rest, just a really cool moment, if only I had pictures! We surmised it was a bit under 2 lbs. Shortly after that I caught a pretty monstrous goggle eye, too. Then we had to head back because we had lost the sun.
All in all probably the most fun I've had fishing in Missouri. I always suspected that I would love fly fishing and now I am totally hooked and about to become a serious addict. I'll have to look into getting some better and proper gear (still don't have waders, and I'm using the Dogwood Canyon rod/reel combo that I got for a great price off of Craigslist).