Investment Securities - Cost and Equity Method Investments (Details) - USD ($)	12 Months Ended
	Dec. 31, 2020	Dec. 31, 2019	Dec. 31, 2018
Investment [Line Items]
Federal home loan bank stock	$ 22,090,000	$ 30,060,000
Federal reserve bank stock	14,653,000	14,653,000
Equity Securities, FV-NI	2,511,000	1,993,000
Equity Securities without Readily Determinable Fair Value, Amount	4,689,000	2,689,000
Alternative Investment	21,522,000	20,411,000
Other Investments and Securities, at Cost	$ 65,465,000	$ 69,806,000
equity investment shares sold	79,697	133,281
Proceeds from Sale of Federal Home Loan Bank Stock	$ 8,000,000.0	$ 13,300,000
equity investment purchased, shares	0	128,553
Gain on equity securities, net	$ 2,182,000	$ 5,118,000	$ 4,616,000
Payments to Acquire Federal Reserve Bank Stock	0	6,428,000	0
Alternative Investment, income recognized	2,400,000	4,800,000	1,400,000
Investments [Member]
Investment [Line Items]
Gain on equity securities, net	$ (239,000)		$ (287,000)
Common Stock [Member]
Investment [Line Items]
Gain on equity securities, net		$ (345,000)