Loans Held for Investment - Delinquencies (Details) - USD ($) $ in Thousands	Jun. 30, 2023	Dec. 31, 2022
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	$ 13,663,412	$ 14,738,224
Basis adjustment associated with fair value hedge
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	13,663,412	14,738,224
Basis adjustment associated with fair value hedge	53,130	61,926
Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	13,632,461	14,694,876
30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	649	20,538
60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	31	185
90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	30,271	22,625
Investor loans secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	8,826,439	9,213,516
Investor loans secured by real estate | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,571,246	2,660,321
Investor loans secured by real estate | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	5,788,030	6,112,026
Investor loans secured by real estate | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	428,287	399,034
Investor loans secured by real estate | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	38,876	42,135
Investor loans secured by real estate | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	8,824,346	9,200,307
Investor loans secured by real estate | Current | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,569,680	2,655,892
Investor loans secured by real estate | Current | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	5,788,030	6,103,246
Investor loans secured by real estate | Current | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	428,287	399,034
Investor loans secured by real estate | Current | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	38,349	42,135
Investor loans secured by real estate | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	527	2,723
Investor loans secured by real estate | 30-59 | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 30-59 | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	2,723
Investor loans secured by real estate | 30-59 | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 30-59 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	527	0
Investor loans secured by real estate | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 60-89 | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 60-89 | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 60-89 | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 60-89 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,566	10,486
Investor loans secured by real estate | 90+ | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,566	4,429
Investor loans secured by real estate | 90+ | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	6,057
Investor loans secured by real estate | 90+ | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 90+ | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Business loans secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,657,344	2,871,588
Business loans secured by real estate | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	57,084	61,368
Business loans secured by real estate | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,281,721	2,432,163
Business loans secured by real estate | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	318,539	378,057
Business loans secured by real estate | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,651,305	2,855,335
Business loans secured by real estate | Current | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	55,778	60,177
Business loans secured by real estate | Current | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,276,988	2,424,174
Business loans secured by real estate | Current | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	318,539	370,984
Business loans secured by real estate | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	8,507
Business loans secured by real estate | 30-59 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Business loans secured by real estate | 30-59 | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	1,434
Business loans secured by real estate | 30-59 | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	7,073
Business loans secured by real estate | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	104
Business loans secured by real estate | 60-89 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	104
Business loans secured by real estate | 60-89 | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Business loans secured by real estate | 60-89 | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Business loans secured by real estate | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	6,039	7,642
Business loans secured by real estate | 90+ | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,306	1,087
Business loans secured by real estate | 90+ | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	4,733	6,555
Business loans secured by real estate | 90+ | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Commercial loans
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,106,395	2,576,839
Commercial loans | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,744,763	2,160,948
Commercial loans | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	351,944	404,791
Commercial loans | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	9,688	11,100
Commercial loans | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,083,576	2,564,012
Commercial loans | Current | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,722,499	2,152,302
Commercial loans | Current | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	351,944	401,199
Commercial loans | Current | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	9,133	10,511
Commercial loans | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	122	8,249
Commercial loans | 30-59 | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	122	4,657
Commercial loans | 30-59 | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	3,592
Commercial loans | 30-59 | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Commercial loans | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	31	81
Commercial loans | 60-89 | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	31	81
Commercial loans | 60-89 | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Commercial loans | 60-89 | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Commercial loans | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	22,666	4,497
Commercial loans | 90+ | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	22,111	3,908
Commercial loans | 90+ | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Commercial loans | 90+ | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	555	589
Retail loans
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	73,234	76,281
Retail loans | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	70,993	72,997
Retail loans | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,241	3,284
Retail loans | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	73,234	75,222
Retail loans | Current | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	70,993	71,940
Retail loans | Current | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,241	3,282
Retail loans | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	1,059
Retail loans | 30-59 | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	1,057
Retail loans | 30-59 | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	2
Retail loans | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 60-89 | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 60-89 | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 90+ | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 90+ | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	$ 0	$ 0