Fair Value Measurements - Schedule of Estimated Fair Values of Financial Instruments Not Recorded at Fair Value on Balance Sheet (Details) - USD ($) $ in Thousands	Jun. 30, 2015	Dec. 31, 2014
Estimated fair value | Securities held to maturity
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	$ 44,135	$ 45,795
Estimated fair value | Portfolio loans, net
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	2,581,224	2,482,700
Estimated fair value | State tax credits, held for sale
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	36,584	31,281
Estimated fair value | Deposits
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	2,693,784	2,494,624
Estimated fair value | Subordinated debentures
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	34,529	34,124
Estimated fair value | Federal Home Loan Bank advances
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	72,998	144,000
Estimated fair value | Other Borrowings
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	188,533	239,950
Level 1 | Securities held to maturity
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 1 | Portfolio loans, net
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 1 | State tax credits, held for sale
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 1 | Deposits
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	2,180,628	1,986,158
Level 1 | Subordinated debentures
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 1 | Federal Home Loan Bank advances
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 1 | Other Borrowings
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 2 | Securities held to maturity
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	44,135	45,795
Level 2 | Portfolio loans, net
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 2 | State tax credits, held for sale
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 2 | Deposits
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 2 | Subordinated debentures
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	34,529	34,124
Level 2 | Federal Home Loan Bank advances
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	72,998	144,000
Level 2 | Other Borrowings
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	188,533	239,950
Level 3 | Securities held to maturity
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	0	0
Level 3 | Portfolio loans, net
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	2,581,224	2,482,700
Level 3 | State tax credits, held for sale
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial assets not on condensed consolidated balance sheets at fair value	36,584	31,281
Level 3 | Deposits
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	513,156	508,466
Level 3 | Subordinated debentures
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 3 | Federal Home Loan Bank advances
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	0	0
Level 3 | Other Borrowings
Fair Value, Off-balance Sheet Risks, Disclosure Information [Line Items]
Financial liabilities not on condensed consolidated balance sheets at fair value	$ 0	$ 0