Fair Value Accounting (Fair Value Inputs, Assets, Quantitative Information) (Details) - USD ($) $ in Thousands	3 Months Ended	6 Months Ended
	Jun. 30, 2020	Jun. 30, 2020	Mar. 31, 2020	Dec. 31, 2019	Jun. 30, 2019	Mar. 31, 2019	Dec. 31, 2018
Investment Securities Available for Sale
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Asset measured with unobservable inputs	$ 1,662	$ 1,662	$ 1,562	$ 2,105	$ 2,017	$ 1,981	$ 1,785
Private Equity Investments
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Asset measured with unobservable inputs	698	698	$ 3,255	$ 3,887	$ 9,563	$ 9,481	$ 11,028
Fair Value, Measurements, Recurring Basis | Level 3 | Investment Securities Available for Sale | Discounted cash flow analysis
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Asset measured with unobservable inputs	1,662	$ 1,662
Discount rate		6.51%
Forecasted average Prime reset rate		3.68%
Fair Value, Measurements, Recurring Basis | Level 3 | Private Equity Investments | Individual analysis of each investee company
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Asset measured with unobservable inputs	698	$ 698
Fair Value, Measurements, Recurring Basis | Level 3 | GGL/SBA loans servicing asset | Discounted cash flow analysis
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Asset measured with unobservable inputs	$ 3,019	3,019
Discount rate	12.16%
Prepayment speeds	16.10%
Fair Value, Measurements, Recurring Basis | Level 3 | Earnout liability | Option pricing methods and Monte Carlo simulation
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets, fair value disclosure	$ 15,924	15,924
Fair Value, Measurements, Recurring Basis | Level 3 | Visa derivative liability | Discounted cash flow analysis
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Assets, fair value disclosure	$ 1,755	$ 1,755
Estimated timing of resolution of Covered Litigation and future cumulative deposits to the litigation escrow for settlement of the Covered Litigation		1 year 6 months
Fair Value, Measurements, Nonrecurring Basis | Level 3 | Loans | Third-party appraised value of real estate less estimated selling costs | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		0.00%
Estimated selling costs		0.00%
Fair Value, Measurements, Nonrecurring Basis | Level 3 | Loans | Third-party appraised value of real estate less estimated selling costs | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		43.00%
Estimated selling costs		10.00%
Fair Value, Measurements, Nonrecurring Basis | Level 3 | Loans | Third-party appraised value of real estate less estimated selling costs | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		24.00%
Estimated selling costs		7.00%
Fair Value, Measurements, Nonrecurring Basis | Level 3 | Other real estate | Third-party appraised value of real estate less estimated selling costs | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		0.00%
Estimated selling costs		0.00%
Fair Value, Measurements, Nonrecurring Basis | Level 3 | Other real estate | Third-party appraised value of real estate less estimated selling costs | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		33.00%
Estimated selling costs		10.00%
Fair Value, Measurements, Nonrecurring Basis | Level 3 | Other real estate | Third-party appraised value of real estate less estimated selling costs | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		23.00%
Estimated selling costs		7.00%
Fair Value, Measurements, Nonrecurring Basis | Level 3 | Other assets held for sale | Third-party appraised value less estimated selling costs or BOV | Minimum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		0.00%
Estimated selling costs		0.00%
Fair Value, Measurements, Nonrecurring Basis | Level 3 | Other assets held for sale | Third-party appraised value less estimated selling costs or BOV | Maximum
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		66.00%
Estimated selling costs		10.00%
Fair Value, Measurements, Nonrecurring Basis | Level 3 | Other assets held for sale | Third-party appraised value less estimated selling costs or BOV | Weighted Average
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Discount rate		63.00%
Estimated selling costs		7.00%