Are you throwing directly upstream above you and letting it drift towards you? If so, may be that's your problem. The ideal drift is out in front of you- casting it upstream and letting it drift back in front of you and below you. Depending on the speed and depth, your fly may not be in the "kill zone" till the end of the drift. Also, making good mends will help you get the best/longest drift possible, keeping the fly in the zone longer. Mending is lifting and adjusting the fly line/leader in relation to the fly/indicator so that the line drift free from interfering with the drift of the fly.
Weight of the fly and/or split-shots do affect the drift but it doesn't sound like that's your problem. Or may be I'm misunderstanding your issue.