Fair Value Measurements - Schedule of Estimated Fair Values of Financial Instruments Not Recorded at Fair Value on Balance Sheet (Details) - USD ($) $ in Thousands	Dec. 31, 2016	Dec. 31, 2015
Estimated fair value | Securities held to maturity
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	$ 79,639	$ 43,441
Estimated fair value | Portfolio loans, net
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	3,125,701	2,782,704
Estimated fair value | State tax credits, held for sale
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	37,679	43,647
Estimated fair value | Deposits
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	3,232,414	2,784,654
Estimated fair value | Subordinated debentures
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	86,052	35,432
Estimated fair value | Federal Home Loan Bank advances
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value		109,994
Estimated fair value | Other Borrowings
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	276,905	270,286
Level 1 | Securities held to maturity
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 1 | Portfolio loans, net
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 1 | State tax credits, held for sale
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 1 | Deposits
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	2,760,202	2,428,403
Level 1 | Subordinated debentures
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 1 | Federal Home Loan Bank advances
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value		0
Level 1 | Other Borrowings
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 2 | Securities held to maturity
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	79,639	43,441
Level 2 | Portfolio loans, net
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 2 | State tax credits, held for sale
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 2 | Deposits
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 2 | Subordinated debentures
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	86,052	35,432
Level 2 | Federal Home Loan Bank advances
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value		109,994
Level 2 | Other Borrowings
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	276,905	270,286
Level 3 | Securities held to maturity
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 3 | Portfolio loans, net
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	3,125,701	2,782,704
Level 3 | State tax credits, held for sale
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	37,679	43,647
Level 3 | Deposits
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	472,212	356,251
Level 3 | Subordinated debentures
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 3 | Federal Home Loan Bank advances
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value		0
Level 3 | Other Borrowings
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	$ 0	$ 0