Fair Value of Financial Instruments - Fair value estimates (Details) - USD ($) $ in Thousands	Dec. 31, 2016	Dec. 31, 2015
Assets:
Investments held to maturity	$ 8,461	$ 9,572
Investment securities available-for-sale	380,963	280,273
Accrued interest receivable	13,145	9,315
Liabilities:
FHLB advances	278,000	148,000
Level 1
Assets:
Cash and cash equivalents	156,857	78,417
Investments held to maturity	0	0
Investment securities available-for-sale	0	0
Loans held for sale	0	0
Loans held for investment, net	0	0
Accrued interest receivable	13,145	9,315
Liabilities:
Deposit accounts	2,330,579	1,674,148
FHLB advances	0	0
Other borrowings	0	0
Subordinated debentures	0	0
Accrued interest payable	263	206
Level 2
Assets:
Cash and cash equivalents	0	0
Investments held to maturity	8,461	9,572
Investment securities available-for-sale	380,963	280,273
Loans held for sale	8,405	9,507
Loans held for investment, net	0	0
Accrued interest receivable	0	0
Liabilities:
Deposit accounts	573,467	521,291
FHLB advances	277,935	148,036
Other borrowings	50,905	49,156
Subordinated debentures	69,982	68,675
Accrued interest payable	0	0
Level 3
Assets:
Cash and cash equivalents	0	0
Investment securities available-for-sale	0	0
Loans held for sale	0	0
Loans held for investment, net	3,211,154	2,244,936
Accrued interest receivable	0	0
Liabilities:
Deposit accounts	0	0
FHLB advances	0	0
Other borrowings	0	0
Subordinated debentures	0	0
Accrued interest payable	0	0
Carrying Amount
Assets:
Cash and cash equivalents	156,857	78,417
Investments held to maturity	8,565	9,642
Investment securities available-for-sale	380,963	280,273
FHLB, FRB and other stock	37,304	22,292
Loans held for sale	7,711	8,565
Loans held for investment, net	3,220,317	2,236,998
Accrued interest receivable	13,145	9,315
Liabilities:
Deposit accounts	3,145,485	2,195,123
FHLB advances	278,000	148,000
Other borrowings	49,971	48,125
Subordinated debentures	69,383	69,263
Accrued interest payable	263	206
Estimated Fair Value
Assets:
Cash and cash equivalents	156,857	78,417
Investments held to maturity	8,461	9,572
Investment securities available-for-sale	380,963	280,273
Loans held for sale	8,405	9,507
Loans held for investment, net	3,211,154	2,244,936
Accrued interest receivable	13,145	9,315
Liabilities:
Deposit accounts	2,904,046	2,195,439
FHLB advances	277,935	148,036
Other borrowings	50,905	49,156
Subordinated debentures	69,982	68,675
Accrued interest payable	263	206
Bank Time Deposits | Level 1
Assets:
Cash and cash equivalents	3,944	1,972
Bank Time Deposits | Level 2
Assets:
Cash and cash equivalents	0	0
Bank Time Deposits | Level 3
Assets:
Cash and cash equivalents	0	0
Bank Time Deposits | Carrying Amount
Assets:
Cash and cash equivalents	3,944	1,972
Bank Time Deposits | Estimated Fair Value
Assets:
Cash and cash equivalents	$ 3,944	$ 1,972