Loans and Credit Quality (Details 7) (NonCovered [Member], USD $) In Thousands, unless otherwise specified	3 Months Ended				6 Months Ended
	Jun. 30, 2014		Jun. 30, 2013		Jun. 30, 2014		Jun. 30, 2013
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts	6		11		19		17
Financing Receivable, Modifications, Pre-Modification Recorded Investment	$ 1,400		$ 6,099		$ 12,284		$ 19,717
Financing Receivable, Modifications, Post-Modification Recorded Investment	1,400		6,099		12,284		19,717
Financing Receivable, Modifications, Subsequent Default, Number of Contracts	0		4		2		5
Financing Receivable, Modifications, Subsequent Default, Recorded Investment	0	[1]	2,138	[1],[2]	427	[1],[3]	3,344	[1],[4]
Consumer Portfolio Segment [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts	0		1		1		1
Financing Receivable, Modifications, Pre-Modification Recorded Investment	0		14		124		14
Financing Receivable, Modifications, Post-Modification Recorded Investment	0		14		124		14
Collateralized Loan Obligations [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts	2		3		6		4
Financing Receivable, Modifications, Pre-Modification Recorded Investment	59		3,518		3,346		3,913
Financing Receivable, Modifications, Post-Modification Recorded Investment	59		3,518		3,346		3,913
Financing Receivable, Modifications, Subsequent Default, Number of Contracts	0		3		2		3
Financing Receivable, Modifications, Subsequent Default, Recorded Investment	0	[1]	788	[1]	427	[1]	788	[1]
Unsecured Commercial Member [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts	0		2		2		2
Financing Receivable, Modifications, Pre-Modification Recorded Investment	0		398		38		398
Financing Receivable, Modifications, Post-Modification Recorded Investment	0		398		38		398
Asset Based [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts	0		1		0		1
Financing Receivable, Modifications, Pre-Modification Recorded Investment	0		2,032		0		2,032
Financing Receivable, Modifications, Post-Modification Recorded Investment	0		2,032		0		2,032
SBA 7(a) Financing Receivable [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts	0		4		0		4
Financing Receivable, Modifications, Pre-Modification Recorded Investment	0		137		0		137
Financing Receivable, Modifications, Post-Modification Recorded Investment	0		137		0		137
Commercial Real Estate Construction Financing Receivable [Member] | Real Estate Construction [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts					2
Financing Receivable, Modifications, Pre-Modification Recorded Investment					4,920
Financing Receivable, Modifications, Post-Modification Recorded Investment					4,920
Real Estate Mortgage Other Receivable [Member] | Mortgage Loans on Real Estate [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts	4		0		8		5
Financing Receivable, Modifications, Pre-Modification Recorded Investment	1,341		0		3,856		13,223
Financing Receivable, Modifications, Post-Modification Recorded Investment	1,341		0		3,856		13,223
Financing Receivable, Modifications, Subsequent Default, Number of Contracts	0		1		0		2
Financing Receivable, Modifications, Subsequent Default, Recorded Investment	$ 0	[1]	$ 1,350	[1]	$ 0	[1]	$ 2,556	[1]

[1] (1) The population of defaulted restructured loans for the period indicated includes only those loans restructured during the preceding 12-month period. The table excludes defaulted troubled restructurings in those classes for which the recorded investment was zero at the end of the period.

[2] (2) Represents the balance at June 30, 2013, and is net of charge-offs of $1.1 million.

[3] (2) Represents the balance at June 30, 2014, and is net of charge-offs of $0.2 million

[4] (3) Represents the balance at June 30, 2013, and is net of charge-offs of $1.1 million