Fair Value of Financial Instruments - Fair Value Estimates (Details) - USD ($) $ in Thousands	Mar. 31, 2017	Dec. 31, 2016
Assets:
Investments held-to-maturity	$ 8,147	$ 8,461
Investment securities available-for-sale, at fair value	435,408	380,963
Accrued interest receivable	13,366	13,145
Level 1
Assets:
Cash and cash equivalents	100,513	156,857
Investments held-to-maturity	0	0
Investment securities available-for-sale, at fair value	0	0
Loans held for sale, net	0	0
Loans held for investment, net	0	0
Accrued interest receivable	13,366	13,145
Liabilities:
Deposit accounts	2,439,584	2,330,579
FHLB advances	0	0
Other borrowings	0	0
Subordinated debentures	0	0
Accrued interest payable	286	263
Level 2
Assets:
Cash and cash equivalents	0	0
Investments held-to-maturity	8,147	8,461
Investment securities available-for-sale, at fair value	435,408	380,963
Loans held for sale, net	11,969	8,405
Loans held for investment, net	0	0
Accrued interest receivable	0	0
Liabilities:
Deposit accounts	597,500	573,467
FHLB advances	259,917	277,935
Other borrowings	52,126	50,905
Subordinated debentures	70,104	69,982
Accrued interest payable	0	0
Level 3
Assets:
Cash and cash equivalents	0	0
Investments held-to-maturity	0
Investment securities available-for-sale, at fair value	0	0
Loans held for sale, net	0	0
Loans held for investment, net	3,347,355	3,211,154
Accrued interest receivable	0	0
Liabilities:
Deposit accounts	0	0
FHLB advances	0	0
Other borrowings	0	0
Subordinated debentures	0	0
Accrued interest payable	0	0
Carrying Amount
Assets:
Cash and cash equivalents	100,513	156,857
Investments held-to-maturity	8,272	8,565
Investment securities available-for-sale, at fair value	435,408	380,963
Federal Reserve Bank and FHLB stock, at cost	37,811	37,304
Loans held for sale, net	11,090	7,711
Loans held for investment, net	3,362,622	3,220,317
Accrued interest receivable	13,366	13,145
Liabilities:
Deposit accounts	3,297,073	3,145,581
FHLB advances	260,000	278,000
Other borrowings	51,363	49,971
Subordinated debentures	69,413	69,383
Accrued interest payable	286	263
Estimated Fair Value
Assets:
Cash and cash equivalents	100,513	156,857
Investments held-to-maturity	8,147	8,461
Investment securities available-for-sale, at fair value	435,408	380,963
Loans held for sale, net	11,969	8,405
Loans held for investment, net	3,347,355	3,211,154
Accrued interest receivable	13,366	13,145
Liabilities:
Deposit accounts	3,037,084	2,904,046
FHLB advances	259,917	277,935
Other borrowings	52,126	50,905
Subordinated debentures	70,104	69,982
Accrued interest payable	286	263
Interest-bearing time deposits with financial institutions | Level 1
Assets:
Cash and cash equivalents	3,944	3,944
Interest-bearing time deposits with financial institutions | Level 2
Assets:
Cash and cash equivalents	0	0
Interest-bearing time deposits with financial institutions | Level 3
Assets:
Cash and cash equivalents	0	0
Interest-bearing time deposits with financial institutions | Carrying Amount
Assets:
Cash and cash equivalents	3,944	3,944
Interest-bearing time deposits with financial institutions | Estimated Fair Value
Assets:
Cash and cash equivalents	$ 3,944	$ 3,944