Obviously, the rarer and more powerful an upgrade is, the less are the chances of picking up one, which is why you should think twice before installing a Superior-rank upgrade. The main source of weapon upgrades are fallen enemies - one, sometimes twice during a single mission an enemy will drop an item which can be picked up, and most often of one of those dropped items there will be a weapon upgrade piece. You can install up to two upgrades on a single weapon, but you can't install two of the same kind - and just like with the case of PCS, once an upgrade is installed you won't be able to get it back it can only be replaced by a new one.
One of the biggest novelties in the second installment of the XCOM franchise is the option to upgrade the weapons of your soldiers, by installing various attachments to them, increasing their effectiveness, or giving them new properties.