Loans Held for Investment - Internal Risk Grading System under ASC 326 (Details) - USD ($) $ in Thousands	Sep. 30, 2021	Dec. 31, 2020
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	$ 3,483,154	$ 1,862,085
Fiscal Year before Current Fiscal Year	1,875,610	3,354,750
Two Years before Current Fiscal Year	2,573,205	2,140,513
Three Years before Current Fiscal Year	1,527,156	1,729,763
Four Years before Current Fiscal Year	1,244,757	1,072,603
Prior	2,286,095	2,152,690
Revolving	989,785	912,643
Revolving Converted to Term During the Period	3,099	11,386
Total	13,982,861	13,236,433
Investor loans secured by real estate
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	2,346,289	1,351,350
Fiscal Year before Current Fiscal Year	1,406,768	2,386,517
Two Years before Current Fiscal Year	1,874,113	1,437,278
Three Years before Current Fiscal Year	1,026,187	988,649
Four Years before Current Fiscal Year	741,908	674,159
Prior	1,465,337	1,374,935
Revolving	10,390	12,877
Revolving Converted to Term During the Period	0	0
Total	8,870,992	8,225,765
Investor loans secured by real estate | CRE non-owner-occupied
Financing Receivable, Credit Quality Indicator [Line Items]
Total	2,823,065	2,675,085
Investor loans secured by real estate | CRE non-owner-occupied | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	565,605	265,901
Fiscal Year before Current Fiscal Year	268,733	541,994
Two Years before Current Fiscal Year	424,397	440,351
Three Years before Current Fiscal Year	462,395	287,580
Four Years before Current Fiscal Year	219,672	279,238
Prior	823,208	791,477
Revolving	9,599	11,114
Revolving Converted to Term During the Period	0	0
Total	2,773,609	2,617,655
Investor loans secured by real estate | CRE non-owner-occupied | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	0
Two Years before Current Fiscal Year	16,263	6,669
Three Years before Current Fiscal Year	1,943	437
Four Years before Current Fiscal Year	0	2,516
Prior	0	29,738
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	18,206	39,360
Investor loans secured by real estate | CRE non-owner-occupied | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	9,732
Two Years before Current Fiscal Year	25,775	2,045
Three Years before Current Fiscal Year	0	0
Four Years before Current Fiscal Year	0	516
Prior	4,950	5,218
Revolving	525	559
Revolving Converted to Term During the Period	0	0
Total	31,250	18,070
Investor loans secured by real estate | Multifamily
Financing Receivable, Credit Quality Indicator [Line Items]
Total	5,705,666	5,171,356
Investor loans secured by real estate | Multifamily | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	1,694,447	1,027,644
Fiscal Year before Current Fiscal Year	1,012,346	1,677,716
Two Years before Current Fiscal Year	1,348,563	899,123
Three Years before Current Fiscal Year	526,744	665,939
Four Years before Current Fiscal Year	501,849	354,859
Prior	619,628	531,287
Revolving	218	420
Revolving Converted to Term During the Period	0	0
Total	5,703,795	5,156,988
Investor loans secured by real estate | Multifamily | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year		0
Fiscal Year before Current Fiscal Year		1,758
Two Years before Current Fiscal Year		2,630
Three Years before Current Fiscal Year		0
Four Years before Current Fiscal Year		8,649
Prior		0
Revolving		0
Revolving Converted to Term During the Period		0
Total		13,037
Investor loans secured by real estate | Multifamily | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	0
Two Years before Current Fiscal Year	0	0
Three Years before Current Fiscal Year	543	559
Four Years before Current Fiscal Year	556	772
Prior	772	0
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	1,871	1,331
Investor loans secured by real estate | Construction and land
Financing Receivable, Credit Quality Indicator [Line Items]
Total	292,815	321,993
Investor loans secured by real estate | Construction and land | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	86,107	57,309
Fiscal Year before Current Fiscal Year	125,190	144,759
Two Years before Current Fiscal Year	51,733	73,313
Three Years before Current Fiscal Year	22,134	18,625
Four Years before Current Fiscal Year	3,996	20,531
Prior	3,607	6,672
Revolving	48	784
Revolving Converted to Term During the Period	0	0
Total	292,815	321,993
Investor loans secured by real estate | SBA secured by real estate
Financing Receivable, Credit Quality Indicator [Line Items]
Total	49,446	57,331
Investor loans secured by real estate | SBA secured by real estate | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	130	0
Fiscal Year before Current Fiscal Year	499	8,306
Two Years before Current Fiscal Year	7,373	9,029
Three Years before Current Fiscal Year	9,113	13,418
Four Years before Current Fiscal Year	12,665	6,305
Prior	9,722	7,696
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	39,502	44,754
Investor loans secured by real estate | SBA secured by real estate | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	496
Fiscal Year before Current Fiscal Year	0	1,032
Two Years before Current Fiscal Year	0	1,159
Three Years before Current Fiscal Year	962	1,000
Four Years before Current Fiscal Year	0	373
Prior	548	306
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	1,510	4,366
Investor loans secured by real estate | SBA secured by real estate | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	1,220
Two Years before Current Fiscal Year	9	2,959
Three Years before Current Fiscal Year	2,353	1,091
Four Years before Current Fiscal Year	3,170	400
Prior	2,902	2,541
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	8,434	8,211
Business loans secured by real estate
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	737,142	344,058
Fiscal Year before Current Fiscal Year	348,383	516,400
Two Years before Current Fiscal Year	390,418	422,995
Three Years before Current Fiscal Year	285,786	446,733
Four Years before Current Fiscal Year	292,026	267,062
Prior	607,955	529,565
Revolving	4,572	14,518
Revolving Converted to Term During the Period	300	246
Total	2,666,582	2,541,577
Business loans secured by real estate | CRE non-owner-occupied | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year		344,058
Fiscal Year before Current Fiscal Year		516,400
Two Years before Current Fiscal Year		422,995
Three Years before Current Fiscal Year		446,733
Four Years before Current Fiscal Year		267,062
Prior		529,565
Revolving		14,518
Revolving Converted to Term During the Period		246
Total		2,541,577
Business loans secured by real estate | SBA secured by real estate
Financing Receivable, Credit Quality Indicator [Line Items]
Total	69,937	79,595
Business loans secured by real estate | SBA secured by real estate | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	3,789	3,253
Fiscal Year before Current Fiscal Year	2,368	7,637
Two Years before Current Fiscal Year	7,533	12,608
Three Years before Current Fiscal Year	10,498	16,058
Four Years before Current Fiscal Year	11,230	8,488
Prior	25,181	23,624
Revolving	1,361	0
Revolving Converted to Term During the Period	0	0
Total	61,960	71,668
Business loans secured by real estate | SBA secured by real estate | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	0
Two Years before Current Fiscal Year	0	1,200
Three Years before Current Fiscal Year	0	0
Four Years before Current Fiscal Year	0	137
Prior	212	0
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	212	1,337
Business loans secured by real estate | SBA secured by real estate | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	0
Two Years before Current Fiscal Year	0	184
Three Years before Current Fiscal Year	1,326	1,987
Four Years before Current Fiscal Year	3,000	1,376
Prior	3,439	3,043
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	7,765	6,590
Business loans secured by real estate | CRE owner-occupied
Financing Receivable, Credit Quality Indicator [Line Items]
Total	2,242,164	2,114,050
Business loans secured by real estate | CRE owner-occupied | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	635,469	293,324
Fiscal Year before Current Fiscal Year	302,683	409,758
Two Years before Current Fiscal Year	317,035	332,672
Three Years before Current Fiscal Year	221,806	327,475
Four Years before Current Fiscal Year	207,923	225,098
Prior	531,119	469,704
Revolving	3,211	14,268
Revolving Converted to Term During the Period	300	246
Total	2,219,546	2,072,545
Business loans secured by real estate | CRE owner-occupied | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	2,190
Fiscal Year before Current Fiscal Year	0	15,917
Two Years before Current Fiscal Year	6,078	3,802
Three Years before Current Fiscal Year	1,741	0
Four Years before Current Fiscal Year	0	4,153
Prior	0	201
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	7,819	26,263
Business loans secured by real estate | CRE owner-occupied | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	0
Two Years before Current Fiscal Year	0	3,636
Three Years before Current Fiscal Year	6,109	4,214
Four Years before Current Fiscal Year	2,972	1,169
Prior	5,718	5,973
Revolving	0	250
Revolving Converted to Term During the Period	0	0
Total	14,799	15,242
Business loans secured by real estate | Franchise real estate secured
Financing Receivable, Credit Quality Indicator [Line Items]
Total	354,481	347,932
Business loans secured by real estate | Franchise real estate secured | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	97,884	44,413
Fiscal Year before Current Fiscal Year	43,332	81,438
Two Years before Current Fiscal Year	59,772	66,241
Three Years before Current Fiscal Year	44,306	96,999
Four Years before Current Fiscal Year	66,901	24,673
Prior	42,286	27,020
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	354,481	340,784
Business loans secured by real estate | Franchise real estate secured | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year		878
Fiscal Year before Current Fiscal Year		1,650
Two Years before Current Fiscal Year		2,652
Three Years before Current Fiscal Year		0
Four Years before Current Fiscal Year		0
Prior		0
Revolving		0
Revolving Converted to Term During the Period		0
Total		5,180
Business loans secured by real estate | Franchise real estate secured | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year		0
Fiscal Year before Current Fiscal Year		0
Two Years before Current Fiscal Year		0
Three Years before Current Fiscal Year		0
Four Years before Current Fiscal Year		1,968
Prior		0
Revolving		0
Revolving Converted to Term During the Period		0
Total		1,968
Commercial loans
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	379,494	155,831
Fiscal Year before Current Fiscal Year	116,911	444,000
Two Years before Current Fiscal Year	306,644	266,221
Three Years before Current Fiscal Year	213,528	280,317
Four Years before Current Fiscal Year	203,854	97,412
Prior	123,041	120,099
Revolving	948,281	854,568
Revolving Converted to Term During the Period	2,799	11,140
Total	2,294,552	2,229,588
Commercial loans | Commercial and industrial
Financing Receivable, Credit Quality Indicator [Line Items]
Total	1,888,870	1,768,834
Commercial loans | Commercial and industrial | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	260,359	127,082
Fiscal Year before Current Fiscal Year	89,545	260,368
Two Years before Current Fiscal Year	204,683	159,001
Three Years before Current Fiscal Year	139,560	210,163
Four Years before Current Fiscal Year	156,915	51,800
Prior	78,951	82,291
Revolving	906,369	801,752
Revolving Converted to Term During the Period	2,059	9,315
Total	1,838,441	1,701,772
Commercial loans | Commercial and industrial | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	735
Fiscal Year before Current Fiscal Year	0	0
Two Years before Current Fiscal Year	189	2,331
Three Years before Current Fiscal Year	0	185
Four Years before Current Fiscal Year	0	1,320
Prior	0	243
Revolving	14,629	17,890
Revolving Converted to Term During the Period	0	37
Total	14,818	22,741
Commercial loans | Commercial and industrial | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	650	0
Fiscal Year before Current Fiscal Year	1,759	3,310
Two Years before Current Fiscal Year	552	2,737
Three Years before Current Fiscal Year	2,854	610
Four Years before Current Fiscal Year	867	1,333
Prior	1,570	2,446
Revolving	26,619	32,858
Revolving Converted to Term During the Period	740	1,027
Total	35,611	44,321
Commercial loans | Franchise non-real estate secured
Financing Receivable, Credit Quality Indicator [Line Items]
Total	392,950	444,797
Commercial loans | Franchise non-real estate secured | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	118,333	27,607
Fiscal Year before Current Fiscal Year	24,954	164,025
Two Years before Current Fiscal Year	96,784	94,494
Three Years before Current Fiscal Year	65,300	46,174
Four Years before Current Fiscal Year	27,683	40,829
Prior	37,157	27,745
Revolving	0	1,361
Revolving Converted to Term During the Period	0	502
Total	370,211	402,737
Commercial loans | Franchise non-real estate secured | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year		0
Fiscal Year before Current Fiscal Year		7,267
Two Years before Current Fiscal Year		2,037
Three Years before Current Fiscal Year		230
Four Years before Current Fiscal Year		480
Prior		2,321
Revolving		0
Revolving Converted to Term During the Period		0
Total		12,335
Commercial loans | Franchise non-real estate secured | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	6,690
Two Years before Current Fiscal Year	2,236	3,706
Three Years before Current Fiscal Year	4,421	18,425
Four Years before Current Fiscal Year	14,903	700
Prior	1,179	204
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	22,739	29,725
Commercial loans | SBA non-real estate secured
Financing Receivable, Credit Quality Indicator [Line Items]
Total	12,732	15,957
Commercial loans | SBA non-real estate secured | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	152	407
Fiscal Year before Current Fiscal Year	653	2,257
Two Years before Current Fiscal Year	1,424	1,558
Three Years before Current Fiscal Year	938	2,674
Four Years before Current Fiscal Year	3,239	610
Prior	3,567	4,449
Revolving	0	0
Revolving Converted to Term During the Period	0	259
Total	9,973	12,214
Commercial loans | SBA non-real estate secured | Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	0
Two Years before Current Fiscal Year	698	0
Three Years before Current Fiscal Year	117	1,574
Four Years before Current Fiscal Year	0	0
Prior	0	0
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	815	1,574
Commercial loans | SBA non-real estate secured | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	83
Two Years before Current Fiscal Year	78	357
Three Years before Current Fiscal Year	338	282
Four Years before Current Fiscal Year	247	340
Prior	617	400
Revolving	664	707
Revolving Converted to Term During the Period	0	0
Total	1,944	2,169
Retail loans
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	20,229	10,846
Fiscal Year before Current Fiscal Year	3,548	7,833
Two Years before Current Fiscal Year	2,030	14,019
Three Years before Current Fiscal Year	1,655	14,064
Four Years before Current Fiscal Year	6,969	33,970
Prior	89,762	128,091
Revolving	26,542	30,680
Revolving Converted to Term During the Period	0	0
Total	150,735	239,503
Retail loans | Single family residential
Financing Receivable, Credit Quality Indicator [Line Items]
Total	144,309	232,574
Retail loans | Single family residential | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	20,201	10,794
Fiscal Year before Current Fiscal Year	3,518	7,714
Two Years before Current Fiscal Year	1,967	13,982
Three Years before Current Fiscal Year	1,633	14,039
Four Years before Current Fiscal Year	6,954	33,968
Prior	86,711	124,248
Revolving	23,274	27,172
Revolving Converted to Term During the Period	0	0
Total	144,258	231,917
Retail loans | Single family residential | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	0
Two Years before Current Fiscal Year	0	0
Three Years before Current Fiscal Year	0	0
Four Years before Current Fiscal Year	0	0
Prior	51	657
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	51	657
Retail loans | Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Total	6,426	6,929
Retail loans | Consumer | Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	28	52
Fiscal Year before Current Fiscal Year	30	112
Two Years before Current Fiscal Year	57	37
Three Years before Current Fiscal Year	22	25
Four Years before Current Fiscal Year	15	2
Prior	2,964	3,145
Revolving	3,268	3,508
Revolving Converted to Term During the Period	0	0
Total	6,384	6,881
Retail loans | Consumer | Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	7
Two Years before Current Fiscal Year	6	0
Three Years before Current Fiscal Year	0	0
Four Years before Current Fiscal Year	0	0
Prior	36	41
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	$ 42	$ 48