The Nuclear Option (BoS)

Once you're inside The Institute, talk to Elder Maxson to receive the Fusion Pulse Charge. You have to go follow the quest marker and enter the Old Robotics section of the Institute. Once inside, you can use the old terminal by the stairs to deactivate the turrets. Eliminate the synth resistance downstairs and continue forth.

As you make your way down, you'll stumble upon another Advanced terminal that you can use to deactivate the turrets in the main room. Before heading inside, activate the prototype sentry bot using the old robotics terminal by the doorway. The sentry bot is not a friendly but it will automatically attack anything around it. Destroy the synths and once done, take out the sentry bot as well to get valuable loot. Open the floor hatch then drop down. There's a generator with fusion core here so claim it then continue forth until you reach the door leading to The Institute Bioscience.

Working the way around

Beware of the Power laser turrets on the ceiling as it can cut through your damage resistance fairly easily. Take out the synths and exit the lab to reach the main area of The Institute. Kill the enemies inside then you'll get an update about accessing the terminal in the Director's office to override the lockdown. Access the elevator to reach Father's room.

Before you access the terminal, you'll find him sick in his bed. You can pass two speech checks to get the code from him and help some of the survivors. From the terminal, you have three options: Initiate the evacuation of the residents, initiate the lockdown override, and shutdown the remaining synths. If you still want to work with the Railroad post-mission (especially if you spared them by skipping the quest Tactical Thinking), you have to issue to evacuation order. Failure to do so will make them automatically hostile towards you. Choose whatever option(s) you want then regroup with Maxson in the entrance of Advanced Systems. Enter the area to proceed.

Clear the room then proceed to the reactor area. Kill the last set of enemies here. Two legendary synths will be here as well and they carry the reactor terminal password. Loot the password from their bodies then access the reactor terminal in the walkway above.

Before initiating the shutdown sequence, activate the radiation purge first to remove the radiation. Once the shutdown sequence has been initialized, the reactor's door will open, allowing you to safely place the pulse charge inside. Next, talk to Elder Maxson so you can be teleported out of the reactor chamber.

Destroying the Institute from within

The younger Shaun will be with Ingram. Talk to him and decide whether to leave him or take him with you. Once done, talk to Ingram to be teleported to the detonation site. From there, activate the detonator to trigger the nuke and watch the cutscene. Next, talk to Elder Maxson to complete the mission.