Portfolio Loans - Summary of Allowance for Loan Losses by Portfolio Class and Category (Details) - Portfolio loans, net - USD ($) $ in Thousands	3 Months Ended
	Jun. 30, 2016	Mar. 31, 2016	Dec. 31, 2015
Summary of Activity in the Allowance for Loan Losses by Portfolio Class and Category Based on Impairment Method [Line Items]
Individually evaluated for impairment	$ 4,677		$ 2,335
Allowance for Loan Losses [Roll Forward]
Balance at December 31, 2015	34,373	$ 33,441
Provision (provision reversal) charged to expense	716	833
Losses charged off	(163)	(73)
Recoveries	572	172
Balance at March 31, 2016	35,498	34,373
Commercial and industrial
Summary of Activity in the Allowance for Loan Losses by Portfolio Class and Category Based on Impairment Method [Line Items]
Individually evaluated for impairment	2,716		1,953
Allowance for Loan Losses [Roll Forward]
Balance at December 31, 2015	23,161	22,056
Provision (provision reversal) charged to expense	302	1,120
Losses charged off	(157)	(68)
Recoveries	502	53
Balance at March 31, 2016	23,808	23,161
CRE - owner occupied
Summary of Activity in the Allowance for Loan Losses by Portfolio Class and Category Based on Impairment Method [Line Items]
Individually evaluated for impairment	0		6
Allowance for Loan Losses [Roll Forward]
Balance at December 31, 2015	3,117	2,969
Provision (provision reversal) charged to expense	541	80
Losses charged off	0	0
Recoveries	15	68
Balance at March 31, 2016	2,591	3,117
Commercial Real Estate Investor Owned
Summary of Activity in the Allowance for Loan Losses by Portfolio Class and Category Based on Impairment Method [Line Items]
Individually evaluated for impairment	0		0
Allowance for Loan Losses [Roll Forward]
Balance at December 31, 2015	3,375	3,484
Provision (provision reversal) charged to expense	27	(116)
Losses charged off	0	0
Recoveries	8	7
Balance at March 31, 2016	3,356	3,375
Construction and land development
Summary of Activity in the Allowance for Loan Losses by Portfolio Class and Category Based on Impairment Method [Line Items]
Individually evaluated for impairment	31		369
Allowance for Loan Losses [Roll Forward]
Balance at December 31, 2015	1,645	1,704
Provision (provision reversal) charged to expense	434	(65)
Losses charged off	0	0
Recoveries	8	6
Balance at March 31, 2016	1,219	1,645
Residential real estate
Summary of Activity in the Allowance for Loan Losses by Portfolio Class and Category Based on Impairment Method [Line Items]
Individually evaluated for impairment	3		7
Allowance for Loan Losses [Roll Forward]
Balance at December 31, 2015	1,841	1,796
Provision (provision reversal) charged to expense	(80)	11
Losses charged off	0	0
Recoveries	36	34
Balance at March 31, 2016	1,797	1,841
Consumer and other
Summary of Activity in the Allowance for Loan Losses by Portfolio Class and Category Based on Impairment Method [Line Items]
Individually evaluated for impairment	1,927		$ 0
Allowance for Loan Losses [Roll Forward]
Balance at December 31, 2015	1,234	1,432
Provision (provision reversal) charged to expense	1,496	(197)
Losses charged off	(6)	(5)
Recoveries	3	4
Balance at March 31, 2016	$ 2,727	$ 1,234