Fair Value of Financial Instruments - Quantitative Information about Level 3 Fair Value Measurements (Details) $ in Thousands	Dec. 31, 2020 USD ($)	Dec. 31, 2019 USD ($)
Level 3
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable	$ 13,351,092	$ 8,691,019
Fair Value, Recurring | Level 3 | Black-Scholes option pricing model
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Equity warrants	$ 1,914
Fair Value, Recurring | Level 3 | Black-Scholes option pricing model | Volatility | Minimum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Equity warrants, input	0.3000
Fair Value, Recurring | Level 3 | Black-Scholes option pricing model | Volatility | Maximum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Equity warrants, input	0.3500
Fair Value, Recurring | Level 3 | Black-Scholes option pricing model | Volatility | Weighted Average
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Equity warrants, input	0.3119
Fair Value, Recurring | Level 3 | Black-Scholes option pricing model | Risk free interest rate | Minimum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Equity warrants, input	0.0013
Fair Value, Recurring | Level 3 | Black-Scholes option pricing model | Risk free interest rate | Maximum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Equity warrants, input	0.0036
Fair Value, Recurring | Level 3 | Black-Scholes option pricing model | Risk free interest rate | Weighted Average
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Equity warrants, input	0.0018
Fair Value, Recurring | Level 3 | Black-Scholes option pricing model | Marketability Discount | Minimum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Equity warrants, input	0.0600
Fair Value, Recurring | Level 3 | Black-Scholes option pricing model | Marketability Discount | Maximum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Equity warrants, input	0.1600
Fair Value, Recurring | Level 3 | Black-Scholes option pricing model | Marketability Discount | Weighted Average
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Equity warrants, input	0.1351
Fair Value, Nonrecurring
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable	$ 4,077	2,257
Fair Value, Nonrecurring | Level 3
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable	4,077	2,257
Fair Value, Nonrecurring | Level 3 | Fair value of collateral
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable		2,257
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Investor loans secured by real estate | CRE non-owner-occupied
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable	$ 198	$ 569
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Investor loans secured by real estate | CRE non-owner-occupied | Collateral discount and cost to sell | Minimum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.1000	0.1000
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Investor loans secured by real estate | CRE non-owner-occupied | Collateral discount and cost to sell | Maximum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.1000	0.1000
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Investor loans secured by real estate | CRE non-owner-occupied | Collateral discount and cost to sell | Weighted Average
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.1000	0.1000
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Investor loans secured by real estate | SBA secured by real estate
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable	$ 746	$ 408
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Investor loans secured by real estate | SBA secured by real estate | Collateral discount and cost to sell | Minimum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.1000	0.1000
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Investor loans secured by real estate | SBA secured by real estate | Collateral discount and cost to sell | Maximum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.1000	0.1000
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Investor loans secured by real estate | SBA secured by real estate | Collateral discount and cost to sell | Weighted Average
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.1000	0.1000
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Business loans secured by real estate | SBA secured by real estate
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable	$ 386	$ 140
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Business loans secured by real estate | SBA secured by real estate | Collateral discount and cost to sell | Minimum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.0700	0.0700
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Business loans secured by real estate | SBA secured by real estate | Collateral discount and cost to sell | Maximum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.1000	0.1000
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Business loans secured by real estate | SBA secured by real estate | Collateral discount and cost to sell | Weighted Average
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.0909	0.0781
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Commercial loans | Commercial and industrial
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable	$ 2,040
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Commercial loans | Commercial and industrial | Collateral discount and cost to sell | Minimum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.0700
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Commercial loans | Commercial and industrial | Collateral discount and cost to sell | Maximum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.1000
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Commercial loans | Commercial and industrial | Collateral discount and cost to sell | Weighted Average
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.0906
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Commercial loans | SBA non-real estate secured
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable	$ 707	$ 1,140
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Commercial loans | SBA non-real estate secured | Collateral discount and cost to sell | Minimum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.0700	0.0700
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Commercial loans | SBA non-real estate secured | Collateral discount and cost to sell | Maximum
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.0700	0.6300
Fair Value, Nonrecurring | Level 3 | Fair value of collateral | Commercial loans | SBA non-real estate secured | Collateral discount and cost to sell | Weighted Average
Fair Value Measurement Inputs and Valuation Techniques [Line Items]
Loans receivable, input	0.0700	0.1533