Amount troops to Garrison a building

What influences the amount of troops that can be garrisoned in a building?

This is determined by the Hall of War level of the first player to Garrison the building. Make sure this is your largest player (also combat boosts are influenced) for the maximum number of troops.