Loans Held for Investment - Delinquencies (Details) - USD ($) $ in Thousands	Mar. 31, 2022	Dec. 31, 2021
Financing Receivable, Past Due [Line Items]
Loans held for investment	$ 14,733,755	$ 14,295,897
Current
Financing Receivable, Past Due [Line Items]
Loans held for investment	14,690,104	14,276,402
30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment	25,332	1,395
60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment	74	0
90+
Financing Receivable, Past Due [Line Items]
Loans held for investment	18,245	18,100
Investor loans secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	9,162,188	8,987,628
Investor loans secured by real estate | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,774,650	2,771,137
Investor loans secured by real estate | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment	6,041,085	5,891,934
Investor loans secured by real estate | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment	303,811	277,640
Investor loans secured by real estate | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	42,642	46,917
Investor loans secured by real estate | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment	9,150,170	8,975,806
Investor loans secured by real estate | Current | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,762,632	2,760,882
Investor loans secured by real estate | Current | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment	6,041,085	5,890,704
Investor loans secured by real estate | Current | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment	303,811	277,640
Investor loans secured by real estate | Current | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	42,642	46,580
Investor loans secured by real estate | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	1,230
Investor loans secured by real estate | 30-59 | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 30-59 | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	1,230
Investor loans secured by real estate | 30-59 | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 30-59 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 60-89 | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 60-89 | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 60-89 | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 60-89 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment	12,018	10,592
Investor loans secured by real estate | 90+ | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	12,018	10,255
Investor loans secured by real estate | 90+ | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 90+ | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Investor loans secured by real estate | 90+ | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	337
Business loans secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,844,717	2,700,579
Business loans secured by real estate | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	68,466	69,184
Business loans secured by real estate | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,391,984	2,251,014
Business loans secured by real estate | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	384,267	380,381
Business loans secured by real estate | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,839,375	2,695,186
Business loans secured by real estate | Current | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	68,025	68,743
Business loans secured by real estate | Current | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,387,083	2,246,062
Business loans secured by real estate | Current | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	384,267	380,381
Business loans secured by real estate | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Business loans secured by real estate | 30-59 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Business loans secured by real estate | 30-59 | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Business loans secured by real estate | 30-59 | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Business loans secured by real estate | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Business loans secured by real estate | 60-89 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Business loans secured by real estate | 60-89 | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Business loans secured by real estate | 60-89 | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Business loans secured by real estate | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment	5,342	5,393
Business loans secured by real estate | 90+ | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment	441	441
Business loans secured by real estate | 90+ | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment	4,901	4,952
Business loans secured by real estate | 90+ | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Commercial loans
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,641,715	2,506,733
Commercial loans | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,242,632	2,103,112
Commercial loans | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	388,322	392,576
Commercial loans | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	10,761	11,045
Commercial loans | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,615,424	2,504,453
Commercial loans | Current | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment	2,216,983	2,101,558
Commercial loans | Current | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	388,322	392,576
Commercial loans | Current | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	10,119	10,319
Commercial loans | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment	25,332	165
Commercial loans | 30-59 | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment	25,332	92
Commercial loans | 30-59 | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Commercial loans | 30-59 | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	73
Commercial loans | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment	74	0
Commercial loans | 60-89 | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment	74	0
Commercial loans | 60-89 | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Commercial loans | 60-89 | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Commercial loans | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment	885	2,115
Commercial loans | 90+ | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment	243	1,462
Commercial loans | 90+ | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Commercial loans | 90+ | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment	642	653
Retail loans
Financing Receivable, Past Due [Line Items]
Loans held for investment	85,135	100,957
Retail loans | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment	79,978	95,292
Retail loans | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment	5,157	5,665
Retail loans | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment	85,135	100,957
Retail loans | Current | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment	79,978	95,292
Retail loans | Current | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment	5,157	5,665
Retail loans | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Retail loans | 30-59 | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Retail loans | 30-59 | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Retail loans | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Retail loans | 60-89 | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Retail loans | 60-89 | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Retail loans | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Retail loans | 90+ | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment	0	0
Retail loans | 90+ | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment	$ 0	$ 0