The main way lead can be dangerous is if a decent sized chunk hits you in a vital area at a good velocity.
All kidding aside, the boiling point of lead is somewhere in the neighborhood of 1700-degrees and most bullet casters and lure makers use it at around mid 700-degrees, so lead vapors aren't an issue, it's impurities in your lead when you melt it that cause obnoxious odors. I melt down and flux scrap lead outside. The main way people get lead in their system is by eating, drinking, or smoking while using lead. How about using your teeth to pinch down a piece of shot on your line? Yeah, don't do that. I knew a kid that carried pellets for his pellet gun around in his mouth. Yeah - he was messed up anyway, but surely that didn't help.