Loans Held for Investment - Delinquencies (Details) - USD ($) $ in Thousands	Mar. 31, 2023	Dec. 31, 2022
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	$ 14,221,789	$ 14,738,224
Basis adjustment associated with fair value hedge
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	14,221,789	14,738,224
Basis adjustment associated with fair value hedge	50,005	61,926
Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	14,200,946	14,694,876
30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	761	20,538
60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,198	185
90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	18,884	22,625
Investor loans secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	9,006,811	9,213,516
Investor loans secured by real estate | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,590,824	2,660,321
Investor loans secured by real estate | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	5,955,239	6,112,026
Investor loans secured by real estate | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	420,079	399,034
Investor loans secured by real estate | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	40,669	42,135
Investor loans secured by real estate | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	8,997,552	9,200,307
Investor loans secured by real estate | Current | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,585,273	2,655,892
Investor loans secured by real estate | Current | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	5,951,531	6,103,246
Investor loans secured by real estate | Current | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	420,079	399,034
Investor loans secured by real estate | Current | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	40,669	42,135
Investor loans secured by real estate | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	6	2,723
Investor loans secured by real estate | 30-59 | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	6	0
Investor loans secured by real estate | 30-59 | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	2,723
Investor loans secured by real estate | 30-59 | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 30-59 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,129	0
Investor loans secured by real estate | 60-89 | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,129	0
Investor loans secured by real estate | 60-89 | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 60-89 | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 60-89 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	8,124	10,486
Investor loans secured by real estate | 90+ | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	4,416	4,429
Investor loans secured by real estate | 90+ | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	3,708	6,057
Investor loans secured by real estate | 90+ | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 90+ | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Business loans secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,774,604	2,871,588
Business loans secured by real estate | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	60,527	61,368
Business loans secured by real estate | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,342,175	2,432,163
Business loans secured by real estate | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	371,902	378,057
Business loans secured by real estate | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,768,344	2,855,335
Business loans secured by real estate | Current | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	59,029	60,177
Business loans secured by real estate | Current | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,337,413	2,424,174
Business loans secured by real estate | Current | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	371,902	370,984
Business loans secured by real estate | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	308	8,507
Business loans secured by real estate | 30-59 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	308	0
Business loans secured by real estate | 30-59 | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	1,434
Business loans secured by real estate | 30-59 | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	7,073
Business loans secured by real estate | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	104
Business loans secured by real estate | 60-89 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	104
Business loans secured by real estate | 60-89 | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Business loans secured by real estate | 60-89 | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Business loans secured by real estate | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	5,952	7,642
Business loans secured by real estate | 90+ | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,190	1,087
Business loans secured by real estate | 90+ | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	4,762	6,555
Business loans secured by real estate | 90+ | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Commercial loans
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,366,287	2,576,839
Commercial loans | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,967,128	2,160,948
Commercial loans | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	388,722	404,791
Commercial loans | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	10,437	11,100
Commercial loans | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,360,963	2,564,012
Commercial loans | Current | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,962,376	2,152,302
Commercial loans | Current | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	388,722	401,199
Commercial loans | Current | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	9,865	10,511
Commercial loans | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	447	8,249
Commercial loans | 30-59 | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	447	4,657
Commercial loans | 30-59 | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	3,592
Commercial loans | 30-59 | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Commercial loans | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	69	81
Commercial loans | 60-89 | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	69	81
Commercial loans | 60-89 | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Commercial loans | 60-89 | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Commercial loans | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	4,808	4,497
Commercial loans | 90+ | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	4,236	3,908
Commercial loans | 90+ | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Commercial loans | 90+ | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	572	589
Retail loans
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	74,087	76,281
Retail loans | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	70,913	72,997
Retail loans | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	3,174	3,284
Retail loans | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	74,087	75,222
Retail loans | Current | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	70,913	71,940
Retail loans | Current | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	3,174	3,282
Retail loans | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	1,059
Retail loans | 30-59 | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	1,057
Retail loans | 30-59 | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	2
Retail loans | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 60-89 | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 60-89 | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 90+ | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 90+ | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	$ 0	$ 0