Recommendations:

1. BE PATIENT
2. There's not much fun in walking up to a dealer and buying a complete set of Ike dollars.  They are not rare, not scarce, and never will be.  Whereas, truly rare, scarce, uncommon coins are just that; rare, scarce and uncommon.  Thus you will look at a lot of coins before you find the ones that you need.  As an exercise check, your pocket change for a 1972 S small date Lincoln cent, a 1964 D "E PLURIDUS UNUM" Jefferson nickel, or the type I and type II 1946 Roosevelt dimes.
3. It's okay to begin collecting everything in sight for a while.  Just don't start with the really expensive stuff.  Collect by type, by issue, by date, proofs, etc., or whatever makes your toe tap.  When I began collecting in 1968 I tried collecting everything, while by buddy was strictly interested in Standing Liberty Quarters.  Wow, the things I learned from him regarding rarity, condition, grading, etc.  So, unless you are incredibly wealthy you will need to narrow your interests after a while and then work on a particular goal.
4. It's okay to switch goals or work on new ones, particularly after you have finished a set or issue.  Sometimes completing a set will entice you to work on upgrading that issue.  Or, you will find a particularly difficult date to locate and thus know to pick up an extra one that you have learned is hard to find.  When I decided to upgrade my Jefferson nickel set with all of the proofs I found that there were a lot of 1942 Proof silver issues for sale but hardly any 1941.  I don't know why, but the 1941 proof was the hardest one to find.
5. BE PATIENT
6. Q. David Bowers, a longtime reputable coin dealer - a true numismatist, has consistently recommended that one buy the book first.  AMEN!  Once you have narrowed your field of interest to a manageable area, buy the reference book.  Buy Breen's book regardless of what you collect (assuming you collect U.S. issues).  Also, keep a current issue of the Guidebook, (Redbook).  Breen's book (you can probably find it for less tha a $100) saved me a bundle when I found an 1857 Flying Eagle cent that had all of Breen's earmarks of a counterfeit. 
7. Subscribe to the Coin Dealer Newsletter.  It has price guides for virtually all U.S. issues and thus you are better calibrated to recognize good value.
8. Don't buy coins with problems.  Period.  See my guide on "No Bargains in Problem Coins."  
9. Buy the best condition that you can afford - within reason.  when a MS 65 is a multiple of a MS 64 then use your best guts-to-brain ratio and decide if the additional appearance/price makes sense.  Some exceptions to this are the real rarities - MS 65 1914 D Lincoln, same for the 1909 S VDB, 1877 Indian cent, 1856 Flying Eagle, Mercury dime overdates, 1916 D mercury.  You get my drift.
10. When you get serious about an goal or set, talk to a local coin dealer that has a shop and let him know in what you are interested.  These dealers are a wealth of information.
11. BE PATIENT
12. Don't be intimidated by the big auction houses and catalogs.  I picked up a gorgeous Barber quarter for my type set.  Got a good deal on it.  Frequently I don't win auctions, but occassionally I do win even when I submit a reasonable bid (sometimes even a bargain bid).  

Suggestions:

• Learn how to grade at the high AU - UNC level for whatever series you collect.  There are screaming bargains there for virtually any series.  A collection, whether a type set or a series like the Indian Cents in AU 55 - AU 58 is a gorgeous set.  And you will still have to plunk down some serious cash for the set definer - the coin that really defines the set, i.e., the 1877, 1908 S and 1909 S in this case with the Indians.
• Keep you collection in a safe deposit box at the bank and let everyone know that you do that.  Do not keep it at home.  You risk too much for the safety of you and the ones that you care about.  It's probably less than $100/year and so worth the cost.
• Give, yes GIVE some inexpensive coins to a young coin collector.  Those wheat cents might be worth 3 cents each, but the delight and encouragement of giving away 30 cents worth of coins to a young collector will be worth the delight for you.
• BE PATIENT