Allowance for Credit Losses - Amortized Cost of Loans by Estimated LTV and FICO Bands (Details) - USD ($) $ in Thousands	Sep. 30, 2021	Dec. 31, 2020
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	$ 3,483,154	$ 1,862,085
Fiscal Year before Current Fiscal Year	1,875,610	3,354,750
Two Years before Current Fiscal Year	2,573,205	2,140,513
Three Years before Current Fiscal Year	1,527,156	1,729,763
Four Years before Current Fiscal Year	1,244,757	1,072,603
Prior	2,286,095	2,152,690
Revolving	989,785	912,643
Revolving Converted to Term During the Period	3,099	11,386
Total	13,982,861	13,236,433
Investor loans secured by real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	2,346,289	1,351,350
Fiscal Year before Current Fiscal Year	1,406,768	2,386,517
Two Years before Current Fiscal Year	1,874,113	1,437,278
Three Years before Current Fiscal Year	1,026,187	988,649
Four Years before Current Fiscal Year	741,908	674,159
Prior	1,465,337	1,374,935
Revolving	10,390	12,877
Revolving Converted to Term During the Period	0	0
Total	8,870,992	8,225,765
Investor loans secured by real estate | CRE non-owner-occupied
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	2,823,065	2,675,085
Investor loans secured by real estate | CRE non-owner-occupied | 55% and below
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	291,909	138,007
Fiscal Year before Current Fiscal Year	142,444	229,272
Two Years before Current Fiscal Year	188,747	182,385
Three Years before Current Fiscal Year	208,825	136,355
Four Years before Current Fiscal Year	127,768	189,848
Prior	589,484	588,230
Revolving	9,599	11,114
Revolving Converted to Term During the Period	0	0
Total	1,558,776	1,475,211
Investor loans secured by real estate | CRE non-owner-occupied | >55-65%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	220,116	101,434
Fiscal Year before Current Fiscal Year	89,488	217,210
Two Years before Current Fiscal Year	193,461	92,015
Three Years before Current Fiscal Year	104,301	130,024
Four Years before Current Fiscal Year	79,960	78,470
Prior	204,683	204,161
Revolving	525	559
Revolving Converted to Term During the Period	0	0
Total	892,534	823,873
Investor loans secured by real estate | CRE non-owner-occupied | >65-75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	53,580	26,460
Fiscal Year before Current Fiscal Year	36,801	102,494
Two Years before Current Fiscal Year	64,165	169,878
Three Years before Current Fiscal Year	146,402	18,876
Four Years before Current Fiscal Year	8,118	13,952
Prior	32,513	29,506
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	341,579	361,166
Investor loans secured by real estate | CRE non-owner-occupied | Greater than 75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	2,750
Two Years before Current Fiscal Year	20,062	4,787
Three Years before Current Fiscal Year	4,810	2,762
Four Years before Current Fiscal Year	3,826	0
Prior	1,478	4,536
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	30,176	14,835
Investor loans secured by real estate | Multifamily
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	5,705,666	5,171,356
Investor loans secured by real estate | Multifamily | 55% and below
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	260,469	218,833
Fiscal Year before Current Fiscal Year	219,510	345,519
Two Years before Current Fiscal Year	314,269	294,464
Three Years before Current Fiscal Year	211,795	233,997
Four Years before Current Fiscal Year	207,913	84,530
Prior	299,265	269,906
Revolving	218	0
Revolving Converted to Term During the Period	0	0
Total	1,513,439	1,447,249
Investor loans secured by real estate | Multifamily | >55-65%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	719,640	381,737
Fiscal Year before Current Fiscal Year	421,994	731,408
Two Years before Current Fiscal Year	580,084	381,282
Three Years before Current Fiscal Year	229,609	215,170
Four Years before Current Fiscal Year	158,927	152,066
Prior	226,977	189,151
Revolving	0	420
Revolving Converted to Term During the Period	0	0
Total	2,337,231	2,051,234
Investor loans secured by real estate | Multifamily | >65-75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	703,894	427,074
Fiscal Year before Current Fiscal Year	367,815	583,078
Two Years before Current Fiscal Year	442,661	215,389
Three Years before Current Fiscal Year	85,883	215,452
Four Years before Current Fiscal Year	133,716	127,684
Prior	85,918	66,457
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	1,819,887	1,635,134
Investor loans secured by real estate | Multifamily | Greater than 75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	10,444	0
Fiscal Year before Current Fiscal Year	3,027	19,469
Two Years before Current Fiscal Year	11,549	10,618
Three Years before Current Fiscal Year	0	1,879
Four Years before Current Fiscal Year	1,849	0
Prior	8,240	5,773
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	35,109	37,739
Investor loans secured by real estate | Construction and land
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	292,815	321,993
Investor loans secured by real estate | Construction and land | 55% and below
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	82,458	57,309
Fiscal Year before Current Fiscal Year	120,912	105,308
Two Years before Current Fiscal Year	42,774	36,068
Three Years before Current Fiscal Year	4,183	18,625
Four Years before Current Fiscal Year	3,996	20,531
Prior	3,607	6,672
Revolving	48	784
Revolving Converted to Term During the Period	0	0
Total	257,978	245,297
Investor loans secured by real estate | Construction and land | >55-65%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	4,278	36,113
Two Years before Current Fiscal Year	6,630	23,770
Three Years before Current Fiscal Year	17,951	0
Four Years before Current Fiscal Year	0	0
Prior	0	0
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	28,859	59,883
Investor loans secured by real estate | Construction and land | >65-75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	3,649	0
Fiscal Year before Current Fiscal Year	0	3,338
Two Years before Current Fiscal Year	2,329	13,475
Three Years before Current Fiscal Year	0	0
Four Years before Current Fiscal Year	0	0
Prior	0	0
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	5,978	16,813
Investor loans secured by real estate | Construction and land | Greater than 75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	0
Two Years before Current Fiscal Year	0	0
Three Years before Current Fiscal Year	0	0
Four Years before Current Fiscal Year	0	0
Prior	0	0
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	0	0
Investor loans secured by real estate | SBA secured by real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	49,446	57,331
Investor loans secured by real estate | SBA secured by real estate | 55% and below
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	2,066
Two Years before Current Fiscal Year	9	649
Three Years before Current Fiscal Year	637	673
Four Years before Current Fiscal Year	695	317
Prior	2,177	778
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	3,518	4,483
Investor loans secured by real estate | SBA secured by real estate | >55-65%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	2,427
Two Years before Current Fiscal Year	2,407	1,639
Three Years before Current Fiscal Year	1,958	4,008
Four Years before Current Fiscal Year	1,910	879
Prior	2,853	4,354
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	9,128	13,307
Investor loans secured by real estate | SBA secured by real estate | >65-75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	130	0
Fiscal Year before Current Fiscal Year	0	3,897
Two Years before Current Fiscal Year	3,870	3,882
Three Years before Current Fiscal Year	4,606	3,482
Four Years before Current Fiscal Year	4,461	4,519
Prior	5,544	1,884
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	18,611	17,664
Investor loans secured by real estate | SBA secured by real estate | Greater than 75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	0	496
Fiscal Year before Current Fiscal Year	499	2,168
Two Years before Current Fiscal Year	1,096	6,977
Three Years before Current Fiscal Year	5,227	7,346
Four Years before Current Fiscal Year	8,769	1,363
Prior	2,598	3,527
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	18,189	21,877
Business loans secured by real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	737,142	344,058
Fiscal Year before Current Fiscal Year	348,383	516,400
Two Years before Current Fiscal Year	390,418	422,995
Three Years before Current Fiscal Year	285,786	446,733
Four Years before Current Fiscal Year	292,026	267,062
Prior	607,955	529,565
Revolving	4,572	14,518
Revolving Converted to Term During the Period	300	246
Total	2,666,582	2,541,577
Business loans secured by real estate | SBA secured by real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	69,937	79,595
Business loans secured by real estate | SBA secured by real estate | 55% and below
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	2,547	1,825
Fiscal Year before Current Fiscal Year	591	1,626
Two Years before Current Fiscal Year	1,897	5,332
Three Years before Current Fiscal Year	1,127	5,495
Four Years before Current Fiscal Year	5,146	3,615
Prior	14,667	13,582
Revolving	1,361	0
Revolving Converted to Term During the Period	0	0
Total	27,336	31,475
Business loans secured by real estate | SBA secured by real estate | >55-65%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	186	246
Fiscal Year before Current Fiscal Year	556	513
Two Years before Current Fiscal Year	199	1,795
Three Years before Current Fiscal Year	1,513	1,094
Four Years before Current Fiscal Year	1,306	3,586
Prior	7,513	5,448
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	11,273	12,682
Business loans secured by real estate | SBA secured by real estate | >65-75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	703	264
Fiscal Year before Current Fiscal Year	328	3,142
Two Years before Current Fiscal Year	3,111	1,515
Three Years before Current Fiscal Year	5,254	3,968
Four Years before Current Fiscal Year	4,104	1,586
Prior	3,586	4,043
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	17,086	14,518
Business loans secured by real estate | SBA secured by real estate | Greater than 75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	353	918
Fiscal Year before Current Fiscal Year	893	2,356
Two Years before Current Fiscal Year	2,326	5,350
Three Years before Current Fiscal Year	3,930	7,488
Four Years before Current Fiscal Year	3,674	1,214
Prior	3,066	3,594
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	14,242	20,920
Business loans secured by real estate | CRE owner-occupied
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	2,242,164	2,114,050
Business loans secured by real estate | CRE owner-occupied | 55% and below
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	296,313	96,803
Fiscal Year before Current Fiscal Year	108,338	160,605
Two Years before Current Fiscal Year	123,527	157,868
Three Years before Current Fiscal Year	100,865	179,791
Four Years before Current Fiscal Year	139,455	131,795
Prior	383,599	328,188
Revolving	3,211	14,518
Revolving Converted to Term During the Period	300	246
Total	1,155,608	1,069,814
Business loans secured by real estate | CRE owner-occupied | >55-65%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	161,368	72,044
Fiscal Year before Current Fiscal Year	73,789	91,028
Two Years before Current Fiscal Year	66,407	98,176
Three Years before Current Fiscal Year	70,311	94,712
Four Years before Current Fiscal Year	46,314	65,120
Prior	99,171	90,548
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	517,360	511,628
Business loans secured by real estate | CRE owner-occupied | >65-75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	126,160	71,692
Fiscal Year before Current Fiscal Year	87,909	152,920
Two Years before Current Fiscal Year	123,495	79,106
Three Years before Current Fiscal Year	52,679	43,832
Four Years before Current Fiscal Year	16,909	31,303
Prior	37,740	31,493
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	444,892	410,346
Business loans secured by real estate | CRE owner-occupied | Greater than 75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	51,628	54,975
Fiscal Year before Current Fiscal Year	32,647	21,122
Two Years before Current Fiscal Year	9,684	4,960
Three Years before Current Fiscal Year	5,801	13,354
Four Years before Current Fiscal Year	8,217	2,202
Prior	16,327	25,649
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	124,304	122,262
Business loans secured by real estate | Franchise real estate secured
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	354,481	347,932
Business loans secured by real estate | Franchise real estate secured | 55% and below
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	17,039	20,801
Fiscal Year before Current Fiscal Year	23,695	10,470
Two Years before Current Fiscal Year	9,052	13,864
Three Years before Current Fiscal Year	12,776	20,956
Four Years before Current Fiscal Year	14,889	9,189
Prior	19,988	16,213
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	97,439	91,493
Business loans secured by real estate | Franchise real estate secured | >55-65%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	39,872	2,689
Fiscal Year before Current Fiscal Year	2,622	9,955
Two Years before Current Fiscal Year	11,055	16,001
Three Years before Current Fiscal Year	10,137	19,102
Four Years before Current Fiscal Year	4,804	6,855
Prior	10,129	2,333
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	78,619	56,935
Business loans secured by real estate | Franchise real estate secured | >65-75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	23,262	19,349
Fiscal Year before Current Fiscal Year	15,150	51,719
Two Years before Current Fiscal Year	36,521	23,258
Three Years before Current Fiscal Year	8,632	9,153
Four Years before Current Fiscal Year	15,563	10,597
Prior	10,951	7,236
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	110,079	121,312
Business loans secured by real estate | Franchise real estate secured | Greater than 75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	17,711	2,452
Fiscal Year before Current Fiscal Year	1,865	10,944
Two Years before Current Fiscal Year	3,144	15,770
Three Years before Current Fiscal Year	12,761	47,788
Four Years before Current Fiscal Year	31,645	0
Prior	1,218	1,238
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	68,344	78,192
Retail loans
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	20,229	10,846
Fiscal Year before Current Fiscal Year	3,548	7,833
Two Years before Current Fiscal Year	2,030	14,019
Three Years before Current Fiscal Year	1,655	14,064
Four Years before Current Fiscal Year	6,969	33,970
Prior	89,762	128,091
Revolving	26,542	30,680
Revolving Converted to Term During the Period	0	0
Total	150,735	239,503
Retail loans | Single family residential
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	144,309	232,574
Retail loans | Single family residential | Greater than 740
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	20,201	10,794
Fiscal Year before Current Fiscal Year	3,518	6,531
Two Years before Current Fiscal Year	1,967	12,679
Three Years before Current Fiscal Year	1,599	8,846
Four Years before Current Fiscal Year	3,319	28,222
Prior	53,609	81,838
Revolving	17,213	19,588
Revolving Converted to Term During the Period	0	0
Total	101,426	168,498
Retail loans | Single family residential | >680 - 740
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	1,183
Two Years before Current Fiscal Year	0	1,303
Three Years before Current Fiscal Year	34	4,732
Four Years before Current Fiscal Year	3,172	2,614
Prior	9,756	15,624
Revolving	5,271	6,685
Revolving Converted to Term During the Period	0	0
Total	18,233	32,141
Retail loans | Single family residential | >580 - 680
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	0
Two Years before Current Fiscal Year	0	0
Three Years before Current Fiscal Year	0	461
Four Years before Current Fiscal Year	453	3,132
Prior	9,301	7,473
Revolving	757	864
Revolving Converted to Term During the Period	0	0
Total	10,511	11,930
Retail loans | Single family residential | Less than 580
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	0
Two Years before Current Fiscal Year	0	0
Three Years before Current Fiscal Year	0	0
Four Years before Current Fiscal Year	10	0
Prior	14,096	19,970
Revolving	33	35
Revolving Converted to Term During the Period	0	0
Total	14,139	20,005
Retail loans | Consumer
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	6,426	6,929
Retail loans | Consumer | Greater than 740
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	27	52
Fiscal Year before Current Fiscal Year	30	69
Two Years before Current Fiscal Year	37	31
Three Years before Current Fiscal Year	22	22
Four Years before Current Fiscal Year	12	1
Prior	2,500	2,609
Revolving	1,325	2,198
Revolving Converted to Term During the Period	0	0
Total	3,953	4,982
Retail loans | Consumer | >680 - 740
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	1	0
Fiscal Year before Current Fiscal Year	0	35
Two Years before Current Fiscal Year	19	6
Three Years before Current Fiscal Year	0	3
Four Years before Current Fiscal Year	3	0
Prior	441	469
Revolving	1,876	1,227
Revolving Converted to Term During the Period	0	0
Total	2,340	1,740
Retail loans | Consumer | >580 - 680
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	15
Two Years before Current Fiscal Year	7	0
Three Years before Current Fiscal Year	0	0
Four Years before Current Fiscal Year	0	1
Prior	59	95
Revolving	49	56
Revolving Converted to Term During the Period	0	0
Total	115	167
Retail loans | Consumer | Less than 580
Financing Receivable, Allowance for Credit Loss [Line Items]
Current Fiscal Year	0	0
Fiscal Year before Current Fiscal Year	0	0
Two Years before Current Fiscal Year	0	0
Three Years before Current Fiscal Year	0	0
Four Years before Current Fiscal Year	0	0
Prior	0	13
Revolving	18	27
Revolving Converted to Term During the Period	0	0
Total	$ 18	$ 40