Here's a sneaky trick to get you new weapons and also keep your old ones. What you do is drop the weapons of your existing kit on the ground before you talk to the quartermaster and instead of taking them back and issuing you new ones he won't be able to take your previous kit from you so he'll just issue you new kit.
Search in following locations for the hidden Vault-Tec Bobbleheads which will give you either +10 to a skill or +1 to a stat.
Repair - Aerefu: In Evan King's house.
Medicine - Vault 101: Found on your dad's desk when leaving.
Energy Weapons - Ravenhold.
Speech - Paradise Falls: Found at Euology's (Slave Master) house on a table.
Big Guns - Fort Constantine: Found inside a safe in the wall in the basement of the CO Quarters.
Barter - Evergreen Mills: At the back of the Market Bazaar on a shelf in the upper-right side behind the Raider Trader that does not attack.
Explosives - WKML Broadcast Station: Found by turning on your local map and going through the doorway to the sealed cistern.
Lockpick - Bethesda Ruins: Found in Bethesda Offices East, on the top floor near the door to the bridge.
Melee Weapons - Dunwich Building: Found on the floor next to the door just before you leave the Virulent Underchambers.
Science - Vault 106: Found on the second level on a table in the middle of the living quarters.
Small Guns - National Guard Depot: Found by going through the training room and offices to exit from a door to the main room. Then using the power switch on the wall to your left enter the Armory door and search the shelves.
Sneak - Yao Guai Tunnels: Found on the right of the map near some water on top of a metal box in the Den.
Unarmed - Rockopolis: Use your local map and go directly west from Casey Smith's garage and look for tattered banners that are strung up.
Endurance - Deathclaw Sanctuary: Found at the entrance on a table next to a rotting Brahmin corpse.
Luck - Arlington Cemetery North: Found a the house to the north in the basement on a shelf directly below the stairs.
Perception - Republic Of Dave: Found in the Museum of Dave on a bookshelf on the right.
Agility - Greener Pastures Disposal: Found in the small office covered in radiation.
Intelligence - Rivet City: Found on Dr. Li's table in the Science Lab.
Charisma - Vault 108: Found in the Cloning Lab.
Strength - Megaton: In Sherriff Lucas Simm's house.