Fair Value Measurements - Schedule of Estimated Fair Values of Financial Instruments Not Recorded at Fair Value on Balance Sheet (Details) - USD ($) $ in Thousands	Jun. 30, 2018	Dec. 31, 2017
Estimated fair value | Securities held to maturity
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	$ 66,909	$ 73,458
Estimated fair value | Portfolio loans, net
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	4,252,679	4,096,741
Estimated fair value | State tax credits, held for sale
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	44,770	43,871
Estimated fair value | Deposits
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	4,242,061	4,153,323
Estimated fair value | Subordinated debentures and notes
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	105,889	105,031
Estimated fair value | Federal Home Loan Bank advances
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	361,531	172,893
Estimated fair value | Other Borrowings
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	167,086	253,530
Level 1 | Securities held to maturity
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 1 | Portfolio loans, net
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 1 | State tax credits, held for sale
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 1 | Deposits
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	3,574,581	3,577,641
Level 1 | Subordinated debentures and notes
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 1 | Federal Home Loan Bank advances
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 1 | Other Borrowings
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 2 | Securities held to maturity
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	66,909	73,458
Level 2 | Portfolio loans, net
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 2 | State tax credits, held for sale
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 2 | Deposits
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 2 | Subordinated debentures and notes
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	105,889	105,031
Level 2 | Federal Home Loan Bank advances
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	361,531	172,893
Level 2 | Other Borrowings
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	167,086	253,530
Level 3 | Securities held to maturity
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 3 | Portfolio loans, net
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	4,252,679	4,096,741
Level 3 | State tax credits, held for sale
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	44,770	43,871
Level 3 | Deposits
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	667,480	575,682
Level 3 | Subordinated debentures and notes
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 3 | Federal Home Loan Bank advances
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 3 | Other Borrowings
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	$ 0	$ 0