Summary of Significant Accounting Policies - Impact of Adoption (Details) - USD ($)	Dec. 31, 2020	Jan. 01, 2020	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Allowance for credit losses on debt securities	$ 54,000	$ 58,000
Loans, net of deferred fees	2,625,987,000	31,855,000	$ 2,534,163,000
Allowance for credit losses on off-balance sheet credit exposures	1,078,000	679,000
Allowance for credit losses on loans	44,400,000	31,855,000	23,285,000	$ 27,848,000	$ 19,658,000
Cumulative-effect adjustment as a result of the adoption of ASU 2016-03	577,889,000		576,708,000	367,466,000	271,239,000
Retained earnings	94,899,000		96,741,000
ASU 2016-03 - Topic 326
Allowance for credit losses on loans			(8,570,000)
Cumulative-effect adjustment as a result of the adoption of ASU 2016-03	58,000
ASU 2016-03 - Topic 326 | Restatement Adjustment
Allowance for credit losses on debt securities			58,000
Loans, net of deferred fees			8,570,000
Allowance for credit losses on off-balance sheet credit exposures	(207,000)		(207,000)
Allowance for credit losses on loans			8,570,000
Cumulative-effect adjustment as a result of the adoption of ASU 2016-03			2,359,000
Retained earnings			6,062,000
ASU 2016-03 - Topic 326 | Previously Reported
Loans, net of deferred fees			23,285,000
Allowance for credit losses on off-balance sheet credit exposures			886,000
HCC (Parent)
Cumulative-effect adjustment as a result of the adoption of ASU 2016-03	577,889,000		576,708,000
Commercial
Loans, net of deferred fees			603,345,000
Allowance for credit losses on loans			10,453,000	$ 17,061,000	$ 10,608,000
Commercial
Loans, net of deferred fees	846,386,000	6,790,000	603,345,000
Allowance for credit losses on loans	11,587,000	6,790,000	10,453,000
Commercial | ASU 2016-03 - Topic 326
Allowance for credit losses on loans			3,663,000
Commercial | ASU 2016-03 - Topic 326 | Restatement Adjustment
Loans, net of deferred fees			(3,663,000)
Commercial | ASU 2016-03 - Topic 326 | Previously Reported
Loans, net of deferred fees			10,453,000
Commercial | Commercial
Loans, net of deferred fees	846,386,000		603,345,000
CRE - owner occupied
Loans, net of deferred fees	560,362,000	6,994,000	548,907,000
Allowance for credit losses on loans	8,560,000	6,994,000	3,825,000
CRE - owner occupied | ASU 2016-03 - Topic 326
Allowance for credit losses on loans			(3,169,000)
CRE - owner occupied | ASU 2016-03 - Topic 326 | Restatement Adjustment
Loans, net of deferred fees			3,169,000
CRE - owner occupied | ASU 2016-03 - Topic 326 | Previously Reported
Loans, net of deferred fees			3,825,000
CRE - non-owner occupied
Loans, net of deferred fees	693,103,000	11,672,000	767,821,000
Allowance for credit losses on loans	16,416,000	11,672,000	3,760,000
CRE - non-owner occupied | ASU 2016-03 - Topic 326
Allowance for credit losses on loans			(7,912,000)
CRE - non-owner occupied | ASU 2016-03 - Topic 326 | Restatement Adjustment
Loans, net of deferred fees			7,912,000
CRE - non-owner occupied | ASU 2016-03 - Topic 326 | Previously Reported
Loans, net of deferred fees			3,760,000
Land and construction
Loans, net of deferred fees	144,594,000	1,458,000	147,189,000
Allowance for credit losses on loans	2,509,000	1,458,000	2,621,000
Land and construction | ASU 2016-03 - Topic 326
Allowance for credit losses on loans			1,163,000
Land and construction | ASU 2016-03 - Topic 326 | Restatement Adjustment
Loans, net of deferred fees			(1,163,000)
Land and construction | ASU 2016-03 - Topic 326 | Previously Reported
Loans, net of deferred fees			2,621,000
Home equity
Loans, net of deferred fees	111,885,000	1,321,000	151,775,000
Allowance for credit losses on loans	1,297,000	1,321,000	2,244,000
Home equity | ASU 2016-03 - Topic 326
Allowance for credit losses on loans			923,000
Home equity | ASU 2016-03 - Topic 326 | Restatement Adjustment
Loans, net of deferred fees			(923,000)
Home equity | ASU 2016-03 - Topic 326 | Previously Reported
Loans, net of deferred fees			2,244,000
Multi-family
Loans, net of deferred fees	166,425,000	1,253,000	180,623,000
Allowance for credit losses on loans	2,804,000	1,253,000	57,000
Multi-family | ASU 2016-03 - Topic 326
Allowance for credit losses on loans			(1,196,000)
Multi-family | ASU 2016-03 - Topic 326 | Restatement Adjustment
Loans, net of deferred fees			1,196,000
Multi-family | ASU 2016-03 - Topic 326 | Previously Reported
Loans, net of deferred fees			57,000
Residential mortgages
Loans, net of deferred fees	85,116,000	678,000	100,759,000
Allowance for credit losses on loans	943,000	678,000	243,000
Residential mortgages | ASU 2016-03 - Topic 326
Allowance for credit losses on loans			(435,000)
Residential mortgages | ASU 2016-03 - Topic 326 | Restatement Adjustment
Loans, net of deferred fees			435,000
Residential mortgages | ASU 2016-03 - Topic 326 | Previously Reported
Loans, net of deferred fees			243,000
Consumer and other
Loans, net of deferred fees	18,116,000	1,689,000	33,744,000
Allowance for credit losses on loans	284,000	$ 1,689,000	82,000
Consumer and other | ASU 2016-03 - Topic 326
Allowance for credit losses on loans			(1,607,000)
Consumer and other | ASU 2016-03 - Topic 326 | Restatement Adjustment
Loans, net of deferred fees			1,607,000
Consumer and other | ASU 2016-03 - Topic 326 | Previously Reported
Loans, net of deferred fees			$ 82,000
SBA Paycheck Protection Program ("PPP") loans | Commercial
Loans, net of deferred fees	$ 290,679,000