Over a period of 35 years as a coin collector and as an ANA exhibit judge, I have encountered personal situations as well as comments and advice from dealers that I wish I had know earlier.  Alas, I hope that passing this along to collectors, young and old, will help.

Recommendations:

1. BE PATIENT There's not much fun in walking up to a dealer and bu ying a complete set of Ike dollars.  They are not rare, not scarce, and never will be.  So, it's not difficult, much less challanging to put a set together.  Whereas, truly rare, scarce, uncommon coins are just that; rare, scarce and uncommon.  Thus you will look at a lot of coins before ou find the ones that you need.  As an excercise check your pocket change for a 1972 S small date Lincoln cent, a 1964 D E PLURIDUS UNUM Jefferson nickel, or the type I and typeII 1946 Roosevelt dimes.
2. It's okay to begin collecting everything in sight for a while.  Just don't start with the really expensive stuff.  Collect by type, by issue, by date, proofs, etc.   Sticking you toe in the great sea of numismatics serves well to hone your special interests so that you can really focus on your interests later.  When I began collecting I tried everything while by buddy was strictly interested in Standing Liberty Quarters.  Wow, the things I learned from him regarding rarity, condition, grading, etc.  So, unless you are incredibly wealthy you will need to narrow your interests after a while and then work on a particular goal.
3. It's okay to switch goals or work on new ones, particularly after you have finished as set or issue.  Sometimes completing a set will entice you to work on upgrading that issue.  Or you will find a particularly difficult date to find and thus know to pick up an extra one that you have learned is hard to find.  When I decided to upgrade my Jefferson nickel set with all of the proofs I found that there were a lot of 1942 Proof silver issues for sale but hardly any 1940.  I don't know why, but the 1940 proof was the hardest one to find.
4. BE PATIENT
5. Q. David Bowers, a longtime reputable coin dealer - a true numismatist, has consistently recommended that one buy the book first.  AMEN.  Once you have narrowed your field of interest to a manageable area, buy the reference book.  Buy Breen's book regardless of what you collect (assuming you collect U.S. issues).  Also, keep a current issue of the Guidebook, (Redbook).  Breen's book (you can probably find it for less tha a $100) saved me a bundle when I found an 1857 Flying Eagle cent that had all of Breen's earmarks of a counterfeit. 
6. Subscribe to the CoinDealersNewsletter.  It has price guides for virtually all U.S. issues and thus you are better calibrated to recognize good value.
7. Don't buy coins with problems.  Period.  See my guide on "No Bargains in Problem Coins."  The coin will always have these problems.  Corrosion, bent, scratched, badly nicked, nicked in just the wrong place, etc.  Regardless of price (unless free) these coins will always detract from your efforts and collection.
8. Buy the best condition that you can afford - within reason.  when a MS 65 is a multiple of a MS 64 then use your best guts-to-brain ratio and decide if the additional appearance/price makes sense.  Some exceptions to this are the real rarities - MS 65 1914 D Lincoln, same for the 1909 S VDB, 1877 Indian cent, 1856 Flying Eagle, Mercury dime overdates, 1916 D mercury.  You get my drift.
9. When you get serious about an goal or set, talk to a local coin dealer that has a shop and let him know in what you are interested.  These dealers are a wealth of information.
10. BE PATIENT
11. Don't be intimidated by the big auction houses and catalogs.  I picked up a gorgeous Barber quarter for my type set.  Got a good deal on it.  Frequently I don't win the auction, but occassionaly I do when I submit a reasonable bid (sometimes even a bargain bid).  And, you will be put on their mailing list and receive free copies of future auction catalogs.
12. IMPORTANT  Step back and make sure your purchase is sensible.  I ran across an absurd example recently.  There were three listings on eBay - one was a 1964 mint set with all the coins in PCGS 68 or better listed for $125,000(!).  Whereas another listing was an 1873 Proof NGC Proof 63 ULTRA CAMEO Liberty Double Eagle that was listed for $97,000.  Take your pick and then think about whast is likely to be the case in 5 years from now.

Suggestions:

• Learn how to grade at the high AU - UNC level for whatever series you collect (except for proofs of course).  There are screaming bargains there for virtually any series.  A collection, whether a type set or a series like the India Cents in AU 55 - AU 58 is a gorgeous set.  And you will still have to plunk down some serious cash for the set definer - the coin that really defines the set, i.e., the 1877, 1908 S and 1909 S in this case with the Indians.
• Keep you collection in a safe deposit box at the bank and let everyone know that you do that.  Do not keep it at home.  You risk toomuch for the safety of you and the ones that you care about.  It's probably less than $100/year and so worth the cost.
• Give, yes GIVE some inexpensive coins to a young coin collector.  Those wheat cents might be worth 3 cents each, but the delight and encouragement of giving away 30 cents worth of coins to a young collector will be worth the delight for you.
• BE PATIENT