Stables
You can store up to five horses at a Stable and they can be summoned from another stable in any region. When you find a stable in a region be sure to add it to your map and if you can locate a Shrine or Tower that is nearby it will be able to act as a waypoint.
Riding Horses
If you ride a horse into smaller animals and enemies it will be killed instantly. Horses cannot gallop through populated areas like towns or villages. If you want to pass through them quickly you will have to dismount and run through on foot and whistle the horse over once you reach the other side.
Wild Horses
Even though you can dismount a wild horse you haven't registered and 'claimed' and continue exploring without it running away it is best to register it at a stable when you get the chance just to be safe. Most horses will stay where you leave them but there will be times they will move on their own accord. For example, when it rains they will seek shelter under trees.
Stealing Horses
An easy way to obtain a relatively tame horse is to steal one from a mounted Bokoblin.