Topsoil is generally pretty heavy, and I think it would take a pretty high proportion of good compost to make it right. I couldn't even begin to guess how many fish you'd need.
When I built my raised beds a few years back, I filled them with 100% compost trucked in from a place in town. That was some great stuff -- real black, light and airy. I do the square foot method, so I've got stuff planted very close together. The nice, loose, nutrient-rich soil is a real plus. I've topped it off a couple times with big box compost, but it's crap. Wood chips and even rocks. I'd do the compost thing again, but there's a minimum order amount that is way more than I need.
This year I topped it off with bagged potting soil. I was surprised at how much wood was in it, but, it was still pretty good stuff. Lots of peat and vermiculite. It was a little expensive, but I think it helps make the most of my little garden.