I have worked with lead and have numerous hazmat certs from various sources and agencies.  And I reload.   And have read much literature on the subject of lead's dangers.  Here's the meat of the issue, in my opinion of course. 
Lead should not be eaten, or mixed with food by contaminated hands etc.  Simply wash your hands and any body parts that came in contact with it using warm water and soap.  A brush may help get the embedded  particles off your skin.  Wash your clothes, and keep your work area clean.  If you are casting bullets wear a mask rated for lead dust or contaminants in the lead.  These precautions are not just for lead though.  Gasoline, exhaust, sand blasting media, spray paint, paint stripper, pressure treated sawdust, and a whole list of materials will harm you in the right conditions.  By the way, lead bullets exposed to air form an oxidized coating that literally seals it from the elements, that's the whitish or dull gray finish on old bullets.  Civil War collectors from the battlefields have found that almost zero lead mass is lost after all that time.  The hype is largely agenda driven, and there is little real science to prove lead shot is harming any birds or wildlife.  Another odd fact; doctors leave bullets in gunshot victums routinely when there is little pain, cutting, or lodging danger to the patient, and often remove the bullet at a later time.  Personal experience. Many shooting victims lived/live full healthy lives with a lead bullet in them.  Why can this be so?  Do your own research, be cautious who you tend to base your fears on.  Thanks.