Chartered by Congress in 1816, the Second Bank of the United States played a pivotal role in the "bank wars," when President Andrew Jackson contended that the Second Bank was unconstitutional and dangerous to republican ideals. By 1836, all of the Second Bank's funds had been transferred to state banks.

The Second Bank now features a Portrait Gallery with "The People of Independence" exhibit, a  permanent collection of more than 150 portraits of 18th and 19th century political leaders, military officers, explorers and scientists, including many by noted artist Charles Willson Peale.

The building itself is a work of art, and a fine example of Greek Revival architecture. Architect William Strickland looked to the Parthenon in Greece as his muse, borrowing its blank pediments and Doric columns. After the demise of the Bank in 1836, the building changed hands and function, which resulted in changes to its interior, and even some to its exterior.