Loans - Loan Portfolio by Internal Risk Grading by Year of Origination (Details) - USD ($) $ in Thousands	Dec. 31, 2020	Dec. 31, 2019
Financing Receivable, Credit Quality Indicator [Line Items]
2020	$ 1,862,085
2019	3,354,750
2018	2,140,513
2017	1,729,763
2016	1,072,603
Prior	2,152,690
Revolving	912,643
Revolving Converted to Term During the Period	11,386
Total	13,236,433	$ 8,722,311
Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total		8,640,840
Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total		36,722
Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total		44,749
Investor loans secured by real estate
Financing Receivable, Credit Quality Indicator [Line Items]
2020	1,351,350
2019	2,386,517
2018	1,437,278
2017	988,649
2016	674,159
Prior	1,374,935
Revolving	12,877
Revolving Converted to Term During the Period	0
Total	8,225,765	4,153,084
Investor loans secured by real estate | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total		4,147,989
Investor loans secured by real estate | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total		2,151
Investor loans secured by real estate | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total		2,944
Investor loans secured by real estate | CRE non-owner-occupied
Financing Receivable, Credit Quality Indicator [Line Items]
Total	2,675,085	2,070,141
Investor loans secured by real estate | CRE non-owner-occupied | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
2020	265,901
2019	541,994
2018	440,351
2017	287,580
2016	279,238
Prior	791,477
Revolving	11,114
Revolving Converted to Term During the Period	0
Total	2,617,655	2,067,875
Investor loans secured by real estate | CRE non-owner-occupied | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	6,669
2017	437
2016	2,516
Prior	29,738
Revolving	0
Revolving Converted to Term During the Period	0
Total	39,360	1,178
Investor loans secured by real estate | CRE non-owner-occupied | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	9,732
2018	2,045
2017	0
2016	516
Prior	5,218
Revolving	559
Revolving Converted to Term During the Period	0
Total	18,070	1,088
Investor loans secured by real estate | Multifamily
Financing Receivable, Credit Quality Indicator [Line Items]
Total	5,171,356	1,575,726
Investor loans secured by real estate | Multifamily | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
2020	1,027,644
2019	1,677,716
2018	899,123
2017	665,939
2016	354,859
Prior	531,287
Revolving	420
Revolving Converted to Term During the Period	0
Total	5,156,988	1,575,510
Investor loans secured by real estate | Multifamily | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	1,758
2018	2,630
2017	0
2016	8,649
Prior	0
Revolving	0
Revolving Converted to Term During the Period	0
Total	13,037	0
Investor loans secured by real estate | Multifamily | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	0
2017	559
2016	772
Prior	0
Revolving	0
Revolving Converted to Term During the Period	0
Total	1,331	216
Investor loans secured by real estate | Construction and land
Financing Receivable, Credit Quality Indicator [Line Items]
Total	321,993	438,786
Investor loans secured by real estate | Construction and land | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
2020	57,309
2019	144,759
2018	73,313
2017	18,625
2016	20,531
Prior	6,672
Revolving	784
Revolving Converted to Term During the Period	0
Total	321,993	438,769
Investor loans secured by real estate | Construction and land | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total		0
Investor loans secured by real estate | Construction and land | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total		17
Investor loans secured by real estate | SBA secured by real estate
Financing Receivable, Credit Quality Indicator [Line Items]
Total	57,331	68,431
Investor loans secured by real estate | SBA secured by real estate | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	8,306
2018	9,029
2017	13,418
2016	6,305
Prior	7,696
Revolving	0
Revolving Converted to Term During the Period	0
Total	44,754	65,835
Investor loans secured by real estate | SBA secured by real estate | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
2020	496
2019	1,032
2018	1,159
2017	1,000
2016	373
Prior	306
Revolving	0
Revolving Converted to Term During the Period	0
Total	4,366	973
Investor loans secured by real estate | SBA secured by real estate | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	1,220
2018	2,959
2017	1,091
2016	400
Prior	2,541
Revolving	0
Revolving Converted to Term During the Period	0
Total	8,211	1,623
Business loans secured by real estate
Financing Receivable, Credit Quality Indicator [Line Items]
2020	344,058
2019	516,400
2018	422,995
2017	446,733
2016	267,062
Prior	529,565
Revolving	14,518
Revolving Converted to Term During the Period	246
Total	2,541,577	2,288,175
Business loans secured by real estate | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total		2,267,278
Business loans secured by real estate | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total		13,930
Business loans secured by real estate | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total		6,967
Business loans secured by real estate | SBA secured by real estate
Financing Receivable, Credit Quality Indicator [Line Items]
Total	79,595	88,381
Business loans secured by real estate | SBA secured by real estate | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
2020	3,253
2019	7,637
2018	12,608
2017	16,058
2016	8,488
Prior	23,624
Revolving	0
Revolving Converted to Term During the Period	0
Total	71,668	83,106
Business loans secured by real estate | SBA secured by real estate | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	1,200
2017	0
2016	137
Prior	0
Revolving	0
Revolving Converted to Term During the Period	0
Total	1,337	1,842
Business loans secured by real estate | SBA secured by real estate | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	184
2017	1,987
2016	1,376
Prior	3,043
Revolving	0
Revolving Converted to Term During the Period	0
Total	6,590	3,433
Business loans secured by real estate | CRE owner-occupied
Financing Receivable, Credit Quality Indicator [Line Items]
Total	2,114,050	1,846,554
Business loans secured by real estate | CRE owner-occupied | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
2020	293,324
2019	409,758
2018	332,672
2017	327,475
2016	225,098
Prior	469,704
Revolving	14,268
Revolving Converted to Term During the Period	246
Total	2,072,545	1,831,853
Business loans secured by real estate | CRE owner-occupied | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
2020	2,190
2019	15,917
2018	3,802
2017	0
2016	4,153
Prior	201
Revolving	0
Revolving Converted to Term During the Period	0
Total	26,263	11,167
Business loans secured by real estate | CRE owner-occupied | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	3,636
2017	4,214
2016	1,169
Prior	5,973
Revolving	250
Revolving Converted to Term During the Period	0
Total	15,242	3,534
Business loans secured by real estate | Franchise real estate secured
Financing Receivable, Credit Quality Indicator [Line Items]
Total	347,932	353,240
Business loans secured by real estate | Franchise real estate secured | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
2020	44,413
2019	81,438
2018	66,241
2017	96,999
2016	24,673
Prior	27,020
Revolving	0
Revolving Converted to Term During the Period	0
Total	340,784	352,319
Business loans secured by real estate | Franchise real estate secured | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
2020	878
2019	1,650
2018	2,652
2017	0
2016	0
Prior	0
Revolving	0
Revolving Converted to Term During the Period	0
Total	5,180	921
Business loans secured by real estate | Franchise real estate secured | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	0
2017	0
2016	1,968
Prior	0
Revolving	0
Revolving Converted to Term During the Period	0
Total	1,968	0
Commercial loans
Financing Receivable, Credit Quality Indicator [Line Items]
2020	155,831
2019	444,000
2018	266,221
2017	280,317
2016	97,412
Prior	120,099
Revolving	854,568
Revolving Converted to Term During the Period	11,140
Total	2,229,588	1,975,053
Commercial loans | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total		1,920,189
Commercial loans | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total		20,641
Commercial loans | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total		34,223
Commercial loans | Commercial and industrial
Financing Receivable, Credit Quality Indicator [Line Items]
Total	1,768,834	1,393,270
Commercial loans | Commercial and industrial | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
2020	127,082
2019	260,368
2018	159,001
2017	210,163
2016	51,800
Prior	82,291
Revolving	801,752
Revolving Converted to Term During the Period	9,315
Total	1,701,772	1,359,662
Commercial loans | Commercial and industrial | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
2020	735
2019	0
2018	2,331
2017	185
2016	1,320
Prior	243
Revolving	17,890
Revolving Converted to Term During the Period	37
Total	22,741	13,226
Commercial loans | Commercial and industrial | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	3,310
2018	2,737
2017	610
2016	1,333
Prior	2,446
Revolving	32,858
Revolving Converted to Term During the Period	1,027
Total	44,321	20,382
Commercial loans | Franchise non-real estate secured
Financing Receivable, Credit Quality Indicator [Line Items]
Total	444,797	564,357
Commercial loans | Franchise non-real estate secured | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
2020	27,607
2019	164,025
2018	94,494
2017	46,174
2016	40,829
Prior	27,745
Revolving	1,361
Revolving Converted to Term During the Period	502
Total	402,737	546,594
Commercial loans | Franchise non-real estate secured | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	7,267
2018	2,037
2017	230
2016	480
Prior	2,321
Revolving	0
Revolving Converted to Term During the Period	0
Total	12,335	6,930
Commercial loans | Franchise non-real estate secured | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	6,690
2018	3,706
2017	18,425
2016	700
Prior	204
Revolving	0
Revolving Converted to Term During the Period	0
Total	29,725	10,833
Commercial loans | SBA non-real estate secured
Financing Receivable, Credit Quality Indicator [Line Items]
Total	15,957	17,426
Commercial loans | SBA non-real estate secured | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
2020	407
2019	2,257
2018	1,558
2017	2,674
2016	610
Prior	4,449
Revolving	0
Revolving Converted to Term During the Period	259
Total	12,214	13,933
Commercial loans | SBA non-real estate secured | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	0
2017	1,574
2016	0
Prior	0
Revolving	0
Revolving Converted to Term During the Period	0
Total	1,574	485
Commercial loans | SBA non-real estate secured | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	83
2018	357
2017	282
2016	340
Prior	400
Revolving	707
Revolving Converted to Term During the Period	0
Total	2,169	3,008
Retail loans
Financing Receivable, Credit Quality Indicator [Line Items]
2020	10,846
2019	7,833
2018	14,019
2017	14,064
2016	33,970
Prior	128,091
Revolving	30,680
Revolving Converted to Term During the Period	0
Total	239,503	305,999
Retail loans | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total		305,384
Retail loans | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total		0
Retail loans | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total		615
Retail loans | Single family residential
Financing Receivable, Credit Quality Indicator [Line Items]
Total	232,574	255,024
Retail loans | Single family residential | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
2020	10,794
2019	7,714
2018	13,982
2017	14,039
2016	33,968
Prior	124,248
Revolving	27,172
Revolving Converted to Term During the Period	0
Total	231,917	254,463
Retail loans | Single family residential | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total		0
Retail loans | Single family residential | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	0
2018	0
2017	0
2016	0
Prior	657
Revolving	0
Revolving Converted to Term During the Period	0
Total	657	561
Retail loans | Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Total	6,929	50,975
Retail loans | Consumer | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
2020	52
2019	112
2018	37
2017	25
2016	2
Prior	3,145
Revolving	3,508
Revolving Converted to Term During the Period	0
Total	6,881	50,921
Retail loans | Consumer | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total		0
Retail loans | Consumer | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
2020	0
2019	7
2018	0
2017	0
2016	0
Prior	41
Revolving	0
Revolving Converted to Term During the Period	0
Total	$ 48	$ 54