Loans Held for Investment - Schedule of Delinquencies in the Company's Loan Portfolio (Details) - USD ($) $ in Thousands	Dec. 31, 2023	Dec. 31, 2022
Financing Receivable, Past Due [Line Items]
Loans held for investment	$ 13,289,020	$ 14,676,298
Loans held for investment before basis adjustment	13,318,571	14,738,224
Basis adjustment associated with fair value hedge
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	13,318,571	14,738,224
Basis adjustment associated with fair value hedge	(29,551)	(61,926)
Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	13,308,517	14,694,876
30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,484	20,538
60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,294	185
90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	6,276	22,625
Investor loans secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	8,576,026	9,213,516
Investor loans secured by real estate | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,421,772	2,660,321
Investor loans secured by real estate | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment	5,645,310	6,112,026
Investor loans secured by real estate | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment	472,544	399,034
Investor loans secured by real estate | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	36,400	42,135
Investor loans secured by real estate | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment	8,575,194	9,200,307
Investor loans secured by real estate | Current | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,421,360	2,655,892
Investor loans secured by real estate | Current | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment	5,645,310	6,103,246
Investor loans secured by real estate | Current | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment	472,544	399,034
Investor loans secured by real estate | Current | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	35,980	42,135
Investor loans secured by real estate | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	2,723
Investor loans secured by real estate | 30-59 | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 30-59 | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	2,723
Investor loans secured by real estate | 30-59 | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 30-59 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 60-89 | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 60-89 | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 60-89 | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 60-89 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment	832	10,486
Investor loans secured by real estate | 90+ | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	412	4,429
Investor loans secured by real estate | 90+ | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	6,057
Investor loans secured by real estate | 90+ | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 90+ | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	420	0
Business loans secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,546,589	2,871,588
Business loans secured by real estate | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	50,741	61,368
Business loans secured by real estate | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,191,334	2,432,163
Business loans secured by real estate | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	304,514	378,057
Business loans secured by real estate | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,541,505	2,855,335
Business loans secured by real estate | Current | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	50,604	60,177
Business loans secured by real estate | Current | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,186,679	2,424,174
Business loans secured by real estate | Current | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	304,222	370,984
Business loans secured by real estate | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment	429	8,507
Business loans secured by real estate | 30-59 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	137	0
Business loans secured by real estate | 30-59 | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	1,434
Business loans secured by real estate | 30-59 | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	292	7,073
Business loans secured by real estate | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	104
Business loans secured by real estate | 60-89 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	104
Business loans secured by real estate | 60-89 | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Business loans secured by real estate | 60-89 | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Business loans secured by real estate | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment	4,655	7,642
Business loans secured by real estate | 90+ | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	1,087
Business loans secured by real estate | 90+ | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	4,655	6,555
Business loans secured by real estate | 90+ | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Commercial loans
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,121,255	2,576,839
Commercial loans | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment	1,790,608	2,160,948
Commercial loans | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	319,721	404,791
Commercial loans | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	10,926	11,100
Commercial loans | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,117,136	2,564,012
Commercial loans | Current | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment	1,788,855	2,152,302
Commercial loans | Current | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	318,162	401,199
Commercial loans | Current | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	10,119	10,511
Commercial loans | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,036	8,249
Commercial loans | 30-59 | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment	228	4,657
Commercial loans | 30-59 | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	1,559	3,592
Commercial loans | 30-59 | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	249	0
Commercial loans | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment	1,294	81
Commercial loans | 60-89 | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment	1,294	81
Commercial loans | 60-89 | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Commercial loans | 60-89 | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Commercial loans | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment	789	4,497
Commercial loans | 90+ | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment	231	3,908
Commercial loans | 90+ | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Commercial loans | 90+ | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	558	589
Retail loans
Financing Receivable, Past Due [Line Items]
Loans held for investment	74,701	76,281
Retail loans | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment	72,752	72,997
Retail loans | Consumer loans
Financing Receivable, Past Due [Line Items]
Loans held for investment	1,949	3,284
Retail loans | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment	74,682	75,222
Retail loans | Current | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment	72,733	71,940
Retail loans | Current | Consumer loans
Financing Receivable, Past Due [Line Items]
Loans held for investment	1,949	3,282
Retail loans | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment	19	1,059
Retail loans | 30-59 | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment	19	1,057
Retail loans | 30-59 | Consumer loans
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	2
Retail loans | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Retail loans | 60-89 | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Retail loans | 60-89 | Consumer loans
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Retail loans | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Retail loans | 90+ | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Retail loans | 90+ | Consumer loans
Financing Receivable, Past Due [Line Items]
Loans held for investment	$ 0	$ 0