Loans and Credit Quality (Details 5) (NonCovered [Member], USD $) In Thousands, unless otherwise specified	Jun. 30, 2014	Dec. 31, 2013
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, Related Allowance	$ 15,335	$ 7,600
Impaired Financing Receivable, Recorded Investment	130,543	88,422
Impaired Financing Receivable, Unpaid Principal Balance	161,035	113,813
Hospitality [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	1,974	5,717
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	1,973	6,215
Impaired Financing Receivable, Related Allowance	117	198
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	6,552	3,013
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	7,593	3,385
SBA 504 Loans [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	0	1,642
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	0	1,643
Impaired Financing Receivable, Related Allowance	0	230
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	8,032	2,602
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	10,787	3,646
Real Estate Mortgage Other Receivable [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	14,329	15,937
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	14,446	16,571
Impaired Financing Receivable, Related Allowance	2,128	1,760
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	31,400	33,365
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	43,753	46,062
Residential Real Estate Construction Financing Receivable [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	771	778
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	770	778
Impaired Financing Receivable, Related Allowance	150	168
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	542	0
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	550	0
Commercial Real Estate Portfolio Segment [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	435	1,250
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	432	1,250
Impaired Financing Receivable, Related Allowance	88	1
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	11,049	5,484
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	15,575	9,923
Collateralized Loan Obligations [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	11,958	4,377
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	12,389	4,692
Impaired Financing Receivable, Related Allowance	10,380	4,270
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	3,188	6,700
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	4,656	9,924
Unsecured Commercial Member [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	701	801
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	712	829
Impaired Financing Receivable, Related Allowance	414	375
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	304	179
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	424	247
Asset Based [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	269	1,070
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	0	1,070
Impaired Financing Receivable, Related Allowance	269	180
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	4,604	0
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	5,315	0
Cash Flow [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	0	0
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	0	0
Impaired Financing Receivable, Related Allowance	0	0
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	15,792	0
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	17,525	0
Equipment Finance Commercial Financing Receivable [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	1,941	0
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	1,941	0
Impaired Financing Receivable, Related Allowance	1,468	0
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	8,635	632
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	12,700	632
SBA 7(a) Financing Receivable [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	0	1,136
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	0	1,136
Impaired Financing Receivable, Related Allowance	0	178
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	4,096	3,037
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	5,304	4,945
Consumer Portfolio Segment [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	3,651	424
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	3,751	471
Impaired Financing Receivable, Related Allowance	321	240
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	320	278
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	439	394
Impaired Financing Receivable, Recorded Investment	3,971	702
Impaired Financing Receivable, Unpaid Principal Balance	4,190	865
Mortgage Loans on Real Estate [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, Related Allowance	2,245	2,188
Impaired Financing Receivable, Recorded Investment	62,287	62,276
Impaired Financing Receivable, Unpaid Principal Balance	78,552	77,522
Real Estate Construction [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, Related Allowance	238	169
Impaired Financing Receivable, Recorded Investment	12,797	7,512
Impaired Financing Receivable, Unpaid Principal Balance	17,327	11,951
Commercial Portfolio Segment [Member]
Financing Receivable, Impaired [Line Items]
Impaired Financing Receivable, Related Allowance	12,531	5,003
Impaired Financing Receivable, Recorded Investment	51,488	17,932
Impaired Financing Receivable, Unpaid Principal Balance	$ 60,966	$ 23,475