Business Segment Information - Banking Segment (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2017	Sep. 30, 2016	Sep. 30, 2017	Sep. 30, 2016	Dec. 31, 2016
Segment Reporting Information [Line Items]
Total interest income	$ 45,137	$ 33,471	$ 125,007	$ 86,718
Total interest expense	5,625	3,053	15,119	7,904
Net interest income	39,512	30,418	109,888	78,814
Provision for loan losses	572	2,819	9,697	4,247
Net interest income after provision for loan losses	38,940	27,599	100,191	74,567
Gain on sale of subsidiary			20,860
Other noninterest income			15,798
Noninterest income	4,171	6,099	36,658	14,748
Noninterest expense	28,225	25,792	90,383	66,201
Operating income (loss)	14,886	7,906	46,466	23,114
Total assets	2,906,161		2,906,161		$ 2,641,067
Gross loans	2,425,463		2,425,463		2,027,624
Operating Segments | Banking
Segment Reporting Information [Line Items]
Total interest income	32,973	24,863	93,204	62,399
Intersegment interest allocations	2,193	1,251	5,211	3,351
Total interest expense	4,294	2,671	11,177	6,908
Net interest income	30,872	23,443	87,238	58,842
Provision for loan losses	(69)	2,861	7,571	4,128
Net interest income after provision for loan losses	30,941	20,582	79,667	54,714
Other noninterest income			10,604
Noninterest income	3,498	3,147		5,984
Noninterest expense	21,984	19,000	65,171	45,987
Operating income (loss)	12,455	4,729	25,100	14,711
Total assets	2,852,651		2,852,651		2,588,509
Gross loans	2,331,514		2,331,514		1,961,552
Operating Segments | Factoring
Segment Reporting Information [Line Items]
Total interest income	11,736	8,465	30,828	23,589
Intersegment interest allocations	(2,193)	(1,251)	(5,211)	(3,351)
Net interest income	9,543	7,214	25,617	20,238
Provision for loan losses	649	33	2,042	118
Net interest income after provision for loan losses	8,894	7,181	23,575	20,120
Other noninterest income			2,203
Noninterest income	774	680		1,622
Noninterest expense	5,600	4,984	16,677	14,519
Operating income (loss)	4,068	2,877	9,101	7,223
Total assets	327,194		327,194		223,994
Gross loans	315,742		315,742		212,784
Operating Segments | Asset Management
Segment Reporting Information [Line Items]
Total interest income		61	3	125
Net interest income		61	3	125
Net interest income after provision for loan losses		61	3	125
Other noninterest income			1,717
Noninterest income		1,534		4,819
Noninterest expense		1,259	1,456	3,818
Operating income (loss)		336	264	1,126
Total assets					4,879
Operating Segments | Corporate
Segment Reporting Information [Line Items]
Total interest income	428	82	972	605
Total interest expense	1,331	382	3,942	996
Net interest income	(903)	(300)	(2,970)	(391)
Provision for loan losses	(8)	(75)	84	1
Net interest income after provision for loan losses	(895)	(225)	(3,054)	(392)
Gain on sale of subsidiary			20,860
Other noninterest income			1,274
Noninterest income	(101)	738		2,323
Noninterest expense	641	549	7,079	1,877
Operating income (loss)	(1,637)	$ (36)	12,001	$ 54
Total assets	493,335		493,335		391,745
Gross loans	12,737		12,737		1,866
Eliminations
Segment Reporting Information [Line Items]
Total assets	(767,019)		(767,019)		(568,060)
Gross loans	$ (234,530)		$ (234,530)		$ (148,578)