Allowance for Credit Losses - Amortized Cost of Loans by Estimated LTV and FICO Bands (Details) - USD ($) $ in Thousands	Jun. 30, 2021	Dec. 31, 2020
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	$ 1,932,994	$ 1,862,085
Fiscal year before current fiscal year	1,830,360	3,354,750
Two years before current fiscal year	2,963,529	2,140,513
Three years before current fiscal year	1,819,329	1,729,763
Four years before current fiscal year	1,507,457	1,072,603
Prior	2,649,725	2,152,690
Revolving	889,352	912,643
Revolving Converted to Term During the Period	1,852	11,386
Total	13,594,598	13,236,433
Investor loans secured by real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	1,365,536	1,351,350
Fiscal year before current fiscal year	1,350,055	2,386,517
Two years before current fiscal year	2,137,775	1,437,278
Three years before current fiscal year	1,249,591	988,649
Four years before current fiscal year	870,189	674,159
Prior	1,715,122	1,374,935
Revolving	12,160	12,877
Revolving Converted to Term During the Period	0	0
Total	8,700,428	8,225,765
Investor loans secured by real estate | CRE non-owner-occupied
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	2,810,233	2,675,085
Investor loans secured by real estate | CRE non-owner-occupied | 55% and below
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	174,343	138,007
Fiscal year before current fiscal year	134,995	229,272
Two years before current fiscal year	204,770	182,385
Three years before current fiscal year	225,932	136,355
Four years before current fiscal year	137,797	189,848
Prior	661,868	588,230
Revolving	9,980	11,114
Revolving Converted to Term During the Period	0	0
Total	1,549,685	1,475,211
Investor loans secured by real estate | CRE non-owner-occupied | >55-65%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	84,362	101,434
Fiscal year before current fiscal year	107,085	217,210
Two years before current fiscal year	208,558	92,015
Three years before current fiscal year	105,364	130,024
Four years before current fiscal year	122,170	78,470
Prior	243,022	204,161
Revolving	537	559
Revolving Converted to Term During the Period	0	0
Total	871,098	823,873
Investor loans secured by real estate | CRE non-owner-occupied | >65-75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	42,308	26,460
Fiscal year before current fiscal year	25,566	102,494
Two years before current fiscal year	86,042	169,878
Three years before current fiscal year	147,915	18,876
Four years before current fiscal year	8,567	13,952
Prior	40,446	29,506
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	350,844	361,166
Investor loans secured by real estate | CRE non-owner-occupied | Greater than 75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	0	0
Fiscal year before current fiscal year	0	2,750
Two years before current fiscal year	20,061	4,787
Three years before current fiscal year	12,682	2,762
Four years before current fiscal year	4,373	0
Prior	1,490	4,536
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	38,606	14,835
Investor loans secured by real estate | Multifamily
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	5,539,464	5,171,356
Investor loans secured by real estate | Multifamily | 55% and below
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	118,036	218,833
Fiscal year before current fiscal year	219,481	345,519
Two years before current fiscal year	336,820	294,464
Three years before current fiscal year	243,793	233,997
Four years before current fiscal year	222,718	84,530
Prior	334,898	269,906
Revolving	1,643	0
Revolving Converted to Term During the Period	0	0
Total	1,477,389	1,447,249
Investor loans secured by real estate | Multifamily | >55-65%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	466,963	381,737
Fiscal year before current fiscal year	391,606	731,408
Two years before current fiscal year	673,685	381,282
Three years before current fiscal year	329,299	215,170
Four years before current fiscal year	191,548	152,066
Prior	265,604	189,151
Revolving	0	420
Revolving Converted to Term During the Period	0	0
Total	2,318,705	2,051,234
Investor loans secured by real estate | Multifamily | >65-75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	441,401	427,074
Fiscal year before current fiscal year	373,033	583,078
Two years before current fiscal year	489,754	215,389
Three years before current fiscal year	126,306	215,452
Four years before current fiscal year	154,960	127,684
Prior	119,559	66,457
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	1,705,013	1,635,134
Investor loans secured by real estate | Multifamily | Greater than 75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	1,383	0
Fiscal year before current fiscal year	0	19,469
Two years before current fiscal year	16,338	10,618
Three years before current fiscal year	10,489	1,879
Four years before current fiscal year	1,859	0
Prior	8,288	5,773
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	38,357	37,739
Investor loans secured by real estate | Construction and land
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	297,728	321,993
Investor loans secured by real estate | Construction and land | 55% and below
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	36,740	57,309
Fiscal year before current fiscal year	97,789	105,308
Two years before current fiscal year	73,845	36,068
Three years before current fiscal year	17,816	18,625
Four years before current fiscal year	8,765	20,531
Prior	25,915	6,672
Revolving	0	784
Revolving Converted to Term During the Period	0	0
Total	260,870	245,297
Investor loans secured by real estate | Construction and land | >55-65%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	0	0
Fiscal year before current fiscal year	0	36,113
Two years before current fiscal year	11,429	23,770
Three years before current fiscal year	9,471	0
Four years before current fiscal year	0	0
Prior	0	0
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	20,900	59,883
Investor loans secured by real estate | Construction and land | >65-75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	0	0
Fiscal year before current fiscal year	0	3,338
Two years before current fiscal year	7,911	13,475
Three years before current fiscal year	8,047	0
Four years before current fiscal year	0	0
Prior	0	0
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	15,958	16,813
Investor loans secured by real estate | Construction and land | Greater than 75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	0	0
Fiscal year before current fiscal year	0	0
Two years before current fiscal year	0	0
Three years before current fiscal year	0	0
Four years before current fiscal year	0	0
Prior	0	0
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	0	0
Investor loans secured by real estate | SBA secured by real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	53,003	57,331
Investor loans secured by real estate | SBA secured by real estate | 55% and below
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	0	0
Fiscal year before current fiscal year	0	2,066
Two years before current fiscal year	102	649
Three years before current fiscal year	641	673
Four years before current fiscal year	831	317
Prior	2,303	778
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	3,877	4,483
Investor loans secured by real estate | SBA secured by real estate | >55-65%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	0	0
Fiscal year before current fiscal year	0	2,427
Two years before current fiscal year	2,414	1,639
Three years before current fiscal year	1,965	4,008
Four years before current fiscal year	3,816	879
Prior	2,872	4,354
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	11,067	13,307
Investor loans secured by real estate | SBA secured by real estate | >65-75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	0	0
Fiscal year before current fiscal year	0	3,897
Two years before current fiscal year	3,879	3,882
Three years before current fiscal year	4,624	3,482
Four years before current fiscal year	3,924	4,519
Prior	5,832	1,884
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	18,259	17,664
Investor loans secured by real estate | SBA secured by real estate | Greater than 75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	0	496
Fiscal year before current fiscal year	500	2,168
Two years before current fiscal year	2,167	6,977
Three years before current fiscal year	5,247	7,346
Four years before current fiscal year	8,861	1,363
Prior	3,025	3,527
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	19,800	21,877
Business loans secured by real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	338,745	344,058
Fiscal year before current fiscal year	330,353	516,400
Two years before current fiscal year	440,622	422,995
Three years before current fiscal year	345,111	446,733
Four years before current fiscal year	381,622	267,062
Prior	680,837	529,565
Revolving	3,053	14,518
Revolving Converted to Term During the Period	0	246
Total	2,520,343	2,541,577
Business loans secured by real estate | SBA secured by real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	72,923	79,595
Business loans secured by real estate | SBA secured by real estate | 55% and below
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	1,642	1,825
Fiscal year before current fiscal year	591	1,626
Two years before current fiscal year	1,595	5,332
Three years before current fiscal year	1,033	5,495
Four years before current fiscal year	5,196	3,615
Prior	16,875	13,582
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	26,932	31,475
Business loans secured by real estate | SBA secured by real estate | >55-65%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	37	246
Fiscal year before current fiscal year	1,596	513
Two years before current fiscal year	511	1,795
Three years before current fiscal year	1,733	1,094
Four years before current fiscal year	994	3,586
Prior	7,723	5,448
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	12,594	12,682
Business loans secured by real estate | SBA secured by real estate | >65-75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	643	264
Fiscal year before current fiscal year	329	3,142
Two years before current fiscal year	3,120	1,515
Three years before current fiscal year	5,416	3,968
Four years before current fiscal year	4,008	1,586
Prior	3,244	4,043
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	16,760	14,518
Business loans secured by real estate | SBA secured by real estate | Greater than 75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	180	918
Fiscal year before current fiscal year	907	2,356
Two years before current fiscal year	2,336	5,350
Three years before current fiscal year	4,552	7,488
Four years before current fiscal year	4,638	1,214
Prior	4,024	3,594
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	16,637	20,920
Business loans secured by real estate | CRE owner-occupied
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	2,089,300	2,114,050
Business loans secured by real estate | CRE owner-occupied | 55% and below
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	142,863	96,803
Fiscal year before current fiscal year	90,967	160,605
Two years before current fiscal year	145,590	157,868
Three years before current fiscal year	120,707	179,791
Four years before current fiscal year	160,886	131,795
Prior	409,810	328,188
Revolving	3,053	14,518
Revolving Converted to Term During the Period	0	246
Total	1,073,876	1,069,814
Business loans secured by real estate | CRE owner-occupied | >55-65%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	58,545	72,044
Fiscal year before current fiscal year	64,476	91,028
Two years before current fiscal year	71,835	98,176
Three years before current fiscal year	87,271	94,712
Four years before current fiscal year	90,387	65,120
Prior	118,717	90,548
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	491,231	511,628
Business loans secured by real estate | CRE owner-occupied | >65-75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	53,750	71,692
Fiscal year before current fiscal year	78,611	152,920
Two years before current fiscal year	131,250	79,106
Three years before current fiscal year	63,894	43,832
Four years before current fiscal year	24,207	31,303
Prior	53,691	31,493
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	405,403	410,346
Business loans secured by real estate | CRE owner-occupied | Greater than 75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	21,549	54,975
Fiscal year before current fiscal year	47,614	21,122
Two years before current fiscal year	12,573	4,960
Three years before current fiscal year	5,826	13,354
Four years before current fiscal year	11,615	2,202
Prior	19,613	25,649
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	118,790	122,262
Business loans secured by real estate | Franchise real estate secured
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	358,120	347,932
Business loans secured by real estate | Franchise real estate secured | 55% and below
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	8,599	20,801
Fiscal year before current fiscal year	23,887	10,470
Two years before current fiscal year	9,141	13,864
Three years before current fiscal year	17,024	20,956
Four years before current fiscal year	15,341	9,189
Prior	22,162	16,213
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	96,154	91,493
Business loans secured by real estate | Franchise real estate secured | >55-65%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	27,170	2,689
Fiscal year before current fiscal year	2,644	9,955
Two years before current fiscal year	11,147	16,001
Three years before current fiscal year	10,886	19,102
Four years before current fiscal year	10,729	6,855
Prior	9,804	2,333
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	72,380	56,935
Business loans secured by real estate | Franchise real estate secured | >65-75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	13,812	19,349
Fiscal year before current fiscal year	16,857	51,719
Two years before current fiscal year	40,716	23,258
Three years before current fiscal year	10,181	9,153
Four years before current fiscal year	14,597	10,597
Prior	13,947	7,236
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	110,110	121,312
Business loans secured by real estate | Franchise real estate secured | Greater than 75%
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	9,955	2,452
Fiscal year before current fiscal year	1,874	10,944
Two years before current fiscal year	10,808	15,770
Three years before current fiscal year	16,588	47,788
Four years before current fiscal year	39,024	0
Prior	1,227	1,238
Revolving	0	0
Revolving Converted to Term During the Period	0	0
Total	79,476	78,192
Retail loans
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	13,356	10,846
Fiscal year before current fiscal year	6,810	7,833
Two years before current fiscal year	2,594	14,019
Three years before current fiscal year	2,294	14,064
Four years before current fiscal year	8,808	33,970
Prior	104,123	128,091
Revolving	25,483	30,680
Revolving Converted to Term During the Period	0	0
Total	163,468	239,503
Retail loans | Single family residential
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	157,228	232,574
Retail loans | Single family residential | Greater than 740
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	13,312	10,794
Fiscal year before current fiscal year	6,771	6,531
Two years before current fiscal year	2,524	12,679
Three years before current fiscal year	2,236	8,846
Four years before current fiscal year	4,215	28,222
Prior	64,829	81,838
Revolving	16,092	19,588
Revolving Converted to Term During the Period	0	0
Total	109,979	168,498
Retail loans | Single family residential | >680 - 740
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	0	0
Fiscal year before current fiscal year	0	1,183
Two years before current fiscal year	0	1,303
Three years before current fiscal year	35	4,732
Four years before current fiscal year	4,090	2,614
Prior	11,222	15,624
Revolving	5,524	6,685
Revolving Converted to Term During the Period	0	0
Total	20,871	32,141
Retail loans | Single family residential | >580 - 680
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	0	0
Fiscal year before current fiscal year	0	0
Two years before current fiscal year	0	0
Three years before current fiscal year	0	461
Four years before current fiscal year	484	3,132
Prior	9,425	7,473
Revolving	837	864
Revolving Converted to Term During the Period	0	0
Total	10,746	11,930
Retail loans | Single family residential | Less than 580
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	0	0
Fiscal year before current fiscal year	0	0
Two years before current fiscal year	0	0
Three years before current fiscal year	0	0
Four years before current fiscal year	0	0
Prior	15,598	19,970
Revolving	34	35
Revolving Converted to Term During the Period	0	0
Total	15,632	20,005
Retail loans | Consumer
Financing Receivable, Allowance for Credit Loss [Line Items]
Total	6,240	6,929
Retail loans | Consumer | Greater than 740
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	44	52
Fiscal year before current fiscal year	39	69
Two years before current fiscal year	40	31
Three years before current fiscal year	18	22
Four years before current fiscal year	16	1
Prior	2,526	2,609
Revolving	1,269	2,198
Revolving Converted to Term During the Period	0	0
Total	3,952	4,982
Retail loans | Consumer | >680 - 740
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	0	0
Fiscal year before current fiscal year	0	35
Two years before current fiscal year	20	6
Three years before current fiscal year	5	3
Four years before current fiscal year	3	0
Prior	449	469
Revolving	1,651	1,227
Revolving Converted to Term During the Period	0	0
Total	2,128	1,740
Retail loans | Consumer | >580 - 680
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	0	0
Fiscal year before current fiscal year	0	15
Two years before current fiscal year	10	0
Three years before current fiscal year	0	0
Four years before current fiscal year	0	1
Prior	62	95
Revolving	54	56
Revolving Converted to Term During the Period	0	0
Total	126	167
Retail loans | Consumer | Less than 580
Financing Receivable, Allowance for Credit Loss [Line Items]
Current fiscal year	0	0
Fiscal year before current fiscal year	0	0
Two years before current fiscal year	0	0
Three years before current fiscal year	0	0
Four years before current fiscal year	0	0
Prior	12	13
Revolving	22	27
Revolving Converted to Term During the Period	0	0
Total	$ 34	$ 40