The bank, formerly known as Savings Bank of Danbury, plans to build a four-story red brick office building with white trim and a tower entrance at the corner.