Prosperity is your homebase in the game which contains various facilities that will give permanent improvements to your gameplay. Prosperity and its facilities can be upgraded to help you deal with the more dangerous and difficult forces of the Highwaymen. You don't have to visit it to upgrade your facilities since you can just access the Homebase tab in your menu. However, you should pay a visit once in a while to conveniently gather medicinal herbs, talking to Roger to unlock new expeditions, or simply listen to different character interactions, especially when you bring in new companions and specialists.
We have questions and answers related to this topic which may also help you: