PC noise is typically caused by one problem:  vibration. Every moving part inside a computer vibrates. Hard drives, cooling fans, CPU coolers, power supplies - they all vibrate. If you are interested in building a 'silent' PC or you would like to solve some of your computer's noise problems while cooling your PC properly, here are a few steps you can follow that will provide excellent results.

• Case:  A quality case/tower is essential for providing a quiet environment for a PC. Most major PC manufacturers have really cheapened the quality of their cases, and therefore I always avoid using a case from a company like Dell, HP, etc. Instead, invest in a quality tower, from someone like Antec, Lian Li, or Thermaltake. Sure, you will pay a little extra, but you will get a nice looking case for your computer innards, plus the improved construction will help silence the vibration noise inside. A better quality case can also improve your computer's air flow and cooling as well.
  
• Power Supply:  Again, instead of using a cheap stock power supply, invest in a quality power supply, specifically one that is rated 'silent.' Zalman, Nexus, and Thermaltake all make a series of silent power supplies that I would recommend, but if you trully want a 'silent' power supply, invest in a fan-less power supply. The fans inside a power supply are what usually generate the most noise, therefore a fan-less operation eliminates almost 100% of that.
  
• CPU Cooler:  Two options - Water cooling & Silent CPU Heatsink. Your processor is probably the hottest component in your computer. It must be cooled properly in order to prevent damage. Most stock CPU heatsinks are noisy because the fan that is attached spins super fast and is very small. Instead of using a small fan and heatsink, try a much larger fan, specifically the Zalman series of CPU coolers. Remember, small fans must spin faster to move air at the same rate as a large fan (which can spin at a slower rate). This increased speed leads to a lot of additional noise. If you want the true 'silent' experience, invest in a water cooling system.
• Case Fans:  Cheap case fans are typically small and noisy. When purchasing a PC tower, find one that allows you to install larger cooling fans in the back, preferrably in the 90mm to 120mm range. Smaller fans (60mm-80mm) are much noisier because they must spin faster to move the same amount of air as those larger fans. Large cooling fans are a blessing for cutting down on the noise in a computer.
• Video Card:  Video Cards are a pain in the behind when it comes to noise reduction. Most video cards come with a small heatsink, and a much smaller fan attached. Zalman and Arctic Cooling make a series of VGA coolers that are designed to cool your video card even better than a standard stock heatsink, plus they have much larger fans attached, which provides an excellent solution to noise reduction.
  
• Hard Drive/Optical Drives:  There are a lot more hard drives out there today that are marked as 'silent.' Seagate, for example, makes a series of SATA hard drives that are supposed to be much quieter than their standard line of hard drives. Out of all the disk drives in your PC, the hard drive can often be the noisiest, so invest in a noise-dampened hard drive. DVD writers are hit or miss as far as noise is concerned, but most name-brand DVD writers nowadays are much quieter than they once were. My best advice - check out websites like NewEgg and see what people are saying about the noise in different burners. Also, mounting hard drives, DVD burners, and other optical drives with rubber grommets can reduce the vibration noise inside your case. If your case is large enough, you can even suspend your optical drives with rubber bands, which is great for killing those nagging vibrations.