Fair Value of Financial Instruments - Fair Value Estimates (Details) - USD ($) $ in Thousands	Mar. 31, 2018	Dec. 31, 2017
Assets:
Investments held-to-maturity	$ 23,915	$ 18,082
Investment securities available-for-sale, at fair value	863,243	787,429
Accrued interest receivable	27,073	27,053
Level 1
Assets:
Cash and cash equivalents	128,996	200,064
Investments held-to-maturity	0	0
Investment securities available-for-sale, at fair value	0	0
Loans held for sale	0	0
Loans held for investment, net	0	0
Derivative asset	0	0
Accrued interest receivable	27,073	27,053
Liabilities:
Deposit accounts	5,074,330	5,001,053
FHLB advances	0	0
Other borrowings	0	0
Subordinated debentures	0	0
Derivative liability	0	0
Accrued interest payable	1,638	2,131
Level 1 | Interest-bearing time deposits with financial institutions
Assets:
Cash and cash equivalents	3,693	3,693
Level 2
Assets:
Cash and cash equivalents	0	0
Investments held-to-maturity	23,915	18,082
Investment securities available-for-sale, at fair value	863,243	787,429
Loans held for sale	31,255	23,524
Loans held for investment, net	0	0
Derivative asset	2,717	1,135
Accrued interest receivable	0	0
Liabilities:
Deposit accounts	1,109,351	1,074,564
FHLB advances	438,664	489,823
Other borrowings	43,991	46,373
Subordinated debentures	115,544	115,159
Derivative liability	2,717	1,135
Accrued interest payable	0	0
Level 2 | Interest-bearing time deposits with financial institutions
Assets:
Cash and cash equivalents	0	0
Level 3
Assets:
Cash and cash equivalents	0	0
Investments held-to-maturity	0	0
Investment securities available-for-sale, at fair value	0	0
Loans held for sale	0	0
Loans held for investment, net	6,254,567	6,269,366
Derivative asset	0	0
Accrued interest receivable	0	0
Liabilities:
Deposit accounts	0	0
FHLB advances	0	0
Other borrowings	0	0
Subordinated debentures	0	0
Derivative liability	0	0
Accrued interest payable	0	0
Level 3 | Interest-bearing time deposits with financial institutions
Assets:
Cash and cash equivalents	0	0
Carrying Amount
Assets:
Cash and cash equivalents	128,996	200,064
Investments held-to-maturity	24,559	18,291
Investment securities available-for-sale, at fair value	863,243	787,429
FHLB, FRB and other stock	82,115	65,881
Loans held for sale	29,034	23,426
Loans held for investment, net	6,241,841	6,167,532
Derivative asset	2,717	1,135
Accrued interest receivable	27,073	27,053
Liabilities:
Deposit accounts	6,192,273	6,085,868
FHLB advances	439,647	490,148
Other borrowings	43,876	46,139
Subordinated debentures	105,188	105,123
Derivative liability	2,717	1,135
Accrued interest payable	1,638	2,131
Carrying Amount | Interest-bearing time deposits with financial institutions
Assets:
Cash and cash equivalents	3,693	3,693
Estimated Fair Value
Assets:
Cash and cash equivalents	128,996	200,064
Investments held-to-maturity	23,915	18,082
Investment securities available-for-sale, at fair value	863,243	787,429
Loans held for sale	31,255	23,524
Loans held for investment, net	6,254,567	6,269,366
Derivative asset	2,717	1,135
Accrued interest receivable	27,073	27,053
Liabilities:
Deposit accounts	6,183,681	6,075,617
FHLB advances	438,664	489,823
Other borrowings	43,991	46,373
Subordinated debentures	115,544	115,159
Derivative liability	2,717	1,135
Accrued interest payable	1,638	2,131
Estimated Fair Value | Interest-bearing time deposits with financial institutions
Assets:
Cash and cash equivalents	$ 3,693	$ 3,693