In this room, various notes, among other things, can be found. Do not look at the paintings on the floor yet. Instead, Head to the head of Avery and inspect it for Journal Entry #21. Move close to Sam near this area so you don't miss the opportunity for Optional Conversation 20. (This is NOT the conversation where Sam gives Nate a Libertalia Coin - that convo closes off all possibilities of finding the notes and the optional conversation number 20).
From the entrance, head on, then go left and left again. Look underneath the fallen bookshelf to collect Treasure #80, Emerald Flask.
Now search all drawers, boxes and so forth in the area to collect three notes: Note #11, #12 and #13.
When you've done so, inspect the paintings on the floor and speak with Sam when he calls you for Optional Conversation 21 and to continue with the story.
Sam will call you to move a bookshelf on the right side of the area, revealing a lever, raising the chandelier. This allows you to swing to the chain, and from here you can swing to the paintings, moving to the left to exit the room. Go up the tower's first flight of stairs and head outside. Turn around to spot a rope attachment spot on the far upper left side. You can't immediately swing the rope to it, so climb the crack in the left pillar to go up a little bit first, then swing the rope to the windows and climb up. Now continue to the top of the tower.
As you decend, an action-packed gameplay sequence will follow. Head down as fast as you can. When you find yourself hanging from the rope, quickly climb up, then run and jump at the right times as fast as you can to make the end of the collapsing walkway. Continue through the doors up ahead. When you fall and slide down the slope, don't forget to jump to the ledges in time. Climb up to trigger a scene. Fight your opponent off as best as you can until you're smashed outside. Climb up, around the corner and swing the rope to reach upper ground, from where you can punch down on your target.