Loans and Credit Quality (Details 7) (NonCovered [Member], USD $) In Thousands, unless otherwise specified	3 Months Ended				9 Months Ended
	Sep. 30, 2014		Sep. 30, 2013		Sep. 30, 2014		Sep. 30, 2013
Financing Receivable, Modifications [Line Items]
Financing Receivables, Impaired, Troubled Debt Restructuring, Write-down	$ 236		$ 0		$ 236		$ 1,100
Financing Receivable, Modifications, Number of Contracts	25		11		44		29
Financing Receivable, Modifications, Pre-Modification Recorded Investment	13,736		5,340		26,020		23,439
Financing Receivable, Modifications, Post-Modification Recorded Investment	13,736		5,340		26,020		23,439
Financing Receivable, Modifications, Subsequent Default, Number of Contracts	3		2		5		4
Financing Receivable, Modifications, Subsequent Default, Recorded Investment	1,227	[1],[2]	70	[1],[3]	1,358	[1],[4]	1,824	[1],[5]
Consumer Portfolio Segment [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts	3		0		4		0
Financing Receivable, Modifications, Pre-Modification Recorded Investment	183		0		307		0
Financing Receivable, Modifications, Post-Modification Recorded Investment	183		0		307		0
Financing Receivable, Modifications, Subsequent Default, Number of Contracts					1		0
Financing Receivable, Modifications, Subsequent Default, Recorded Investment					109	[1]	0	[1]
Collateralized Loan Obligations [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts	9		8		15		7
Financing Receivable, Modifications, Pre-Modification Recorded Investment	1,507		4,824		4,853		4,553
Financing Receivable, Modifications, Post-Modification Recorded Investment	1,507		4,824		4,853		4,553
Financing Receivable, Modifications, Subsequent Default, Number of Contracts					1		1
Financing Receivable, Modifications, Subsequent Default, Recorded Investment					22	[1]	427	[1]
Unsecured Commercial Member [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts	3		2		5		5
Financing Receivable, Modifications, Pre-Modification Recorded Investment	173		499		211		502
Financing Receivable, Modifications, Post-Modification Recorded Investment	173		499		211		502
Financing Receivable, Modifications, Subsequent Default, Number of Contracts	2		2		2		2
Financing Receivable, Modifications, Subsequent Default, Recorded Investment	37	[1]	70	[1]	37	[1]	70	[1]
Asset Based [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts	0		1		0		1
Financing Receivable, Modifications, Pre-Modification Recorded Investment	0		17		0		2,032
Financing Receivable, Modifications, Post-Modification Recorded Investment	0		17		0		2,032
SBA Financing Receivable [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts	2		0		2		4
Financing Receivable, Modifications, Pre-Modification Recorded Investment	1,313		0		1,313		137
Financing Receivable, Modifications, Post-Modification Recorded Investment	1,313		0		1,313		137
Commercial Real Estate Construction Financing Receivable [Member] | Real Estate Construction [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts					2		0
Financing Receivable, Modifications, Pre-Modification Recorded Investment					4,920		0
Financing Receivable, Modifications, Post-Modification Recorded Investment					4,920		0
Financing Receivable, Modifications, Subsequent Default, Number of Contracts	1		0		1		0
Financing Receivable, Modifications, Subsequent Default, Recorded Investment	1,190	[1]	0	[1]	1,190	[1]	0	[1]
SBA Loans [Member] | Mortgage Loans on Real Estate [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts	1		0		1		0
Financing Receivable, Modifications, Pre-Modification Recorded Investment	64		0		64		0
Financing Receivable, Modifications, Post-Modification Recorded Investment	64		0		64		0
Real Estate Mortgage Other Receivable [Member] | Mortgage Loans on Real Estate [Member]
Financing Receivable, Modifications [Line Items]
Financing Receivable, Modifications, Number of Contracts	7		0		15		12
Financing Receivable, Modifications, Pre-Modification Recorded Investment	10,496		0		14,352		16,215
Financing Receivable, Modifications, Post-Modification Recorded Investment	10,496		0		14,352		16,215
Financing Receivable, Modifications, Subsequent Default, Number of Contracts					0		1
Financing Receivable, Modifications, Subsequent Default, Recorded Investment					$ 0	[1]	$ 1,327	[1]

[1] (1) The population of defaulted restructured loans for the period indicated includes only those loans restructured during the preceding 12-month period. The table excludes defaulted troubled restructurings in those classes for which the recorded investment was zero at the end of the period.

[2] (2) Represents the balance at September 30, 2014, and is net of charge-offs of $0.2 million.

[3] (3)Represents the balance at September 30, 2013 and there were no charge-offs.

[4] (2) Represents the balance at September 30, 2014, and is net of charge-offs of $0.2 million

[5] (3) Represents the balance at September 30, 2013, and is net of charge-offs of $1.1 million