Summary of Significant Accounting Policies - Narrative (Details)	12 Months Ended
	Dec. 31, 2020 USD ($) loan store borrower	Dec. 31, 2019 USD ($) loan	Jan. 01, 2020 USD ($)	Dec. 31, 2018 USD ($)
Property, Plant and Equipment [Line Items]
Number of loans modified in TDR | loan	2	1
Short-term loan modification program, number of loans | loan	36
Short-term loan modification program, outstanding exposure	$ 72,000,000
Short-term loan modification program, outstanding exposure percentage	0.90%
Net loans	$ 7,650,633,000	$ 7,472,090,000
Intangible assets	0	0
Unamortized multifamily FHA MSR's	$ 807,000	310,000
Reasonable and Supportable period (in years)	2 years
Historical loss rate period	12 months
Allowance for credit loss	$ 167,000
Less: allowance for credit losses	(109,579,000)	(73,658,000)	$ (73,658,000)	$ (69,944,000)
Liabilities: Reserve for Unfunded Commitments	(5,498,000)	$ 0	0
Impact of ASC 326 Adoption | Accounting Standards Update 2016-13
Property, Plant and Equipment [Line Items]
Less: allowance for credit losses			10,600,000
Liabilities: Reserve for Unfunded Commitments			$ 4,100,000
PPP loans
Property, Plant and Equipment [Line Items]
Net loans	$ 454,800,000
Number of borrowers (over) | borrower	1,400
90 days or more past due
Property, Plant and Equipment [Line Items]
Number of loans modified in TDR | loan	0
Minimum
Property, Plant and Equipment [Line Items]
Days from commitment to closing	30 days
Maximum
Property, Plant and Equipment [Line Items]
Days from commitment to closing	90 days
Furniture Fixtures and Equipment | Minimum
Property, Plant and Equipment [Line Items]
Property, plant and equipment, useful life	3 years
Furniture Fixtures and Equipment | Maximum
Property, Plant and Equipment [Line Items]
Property, plant and equipment, useful life	7 years
Computer Software and Hardware | Minimum
Property, Plant and Equipment [Line Items]
Property, plant and equipment, useful life	3 years
Computer Software and Hardware | Maximum
Property, Plant and Equipment [Line Items]
Property, plant and equipment, useful life	5 years
Building Improvements | Minimum
Property, Plant and Equipment [Line Items]
Property, plant and equipment, useful life	5 years
Building Improvements | Maximum
Property, Plant and Equipment [Line Items]
Property, plant and equipment, useful life	20 years
Banking Services
Property, Plant and Equipment [Line Items]
Number of stores | store	20
Lending Services
Property, Plant and Equipment [Line Items]
Number of stores | store	6