Loans and Credit Quality (Details 4) (USD $) In Thousands, unless otherwise specified	Jun. 30, 2014	Dec. 31, 2013	Jun. 30, 2013
Delinquent loans in loan portfolio
30-89 Days Past Due	$ 9,000	$ 4,000
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)		3,929,556
Mortgage Loans on Real Estate [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	5,601,020	2,795,998
Real Estate Other Class [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)		2,196,388
SBA 504 Loans [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)		45,166
Hospitality [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)		180,553
Real Estate Construction [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	310,534	219,517
Commercial Real Estate Construction Financing Receivable [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)		148,898
Residential Real Estate Construction Financing Receivable [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)		58,881
Commercial Portfolio Segment [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	5,225,694	1,242,750
SBA 7(a) Financing Receivable [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)		28,641
Equipment Finance Commercial Financing Receivable [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)		273,483
Cash Flow [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)		0
Asset Based [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)		202,428
Unsecured Commercial Member [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)		153,752
Collateralized Loan Obligations [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)		586,481
Consumer Portfolio Segment [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	63,276	55,070
NonCovered [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	12,121	23,897
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	22,280	5,233
Financing Receivable, Recorded Investment, Past Due	34,401	29,130
Current	10,757,268	3,900,426
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	10,791,669	3,929,556	3,926,230
NonCovered [Member] | Mortgage Loans on Real Estate [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	5,099	15,590
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	14,691	2,406
Financing Receivable, Recorded Investment, Past Due	19,790	17,996
Current	5,216,662	2,404,111
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	5,236,452	2,422,107	2,559,639
NonCovered [Member] | Real Estate Other Class [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	4,340,748	2,196,388
NonCovered [Member] | SBA 504 Loans [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	346,398	45,166
NonCovered [Member] | Hospitality [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	549,306	180,553
NonCovered [Member] | Real Estate Construction [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	0	0
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	2,029	2,013
Financing Receivable, Recorded Investment, Past Due	2,029	2,013
Current	296,448	205,766
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	298,477	207,779	197,776
NonCovered [Member] | Commercial Real Estate Construction Financing Receivable [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	226,558	148,898
NonCovered [Member] | Residential Real Estate Construction Financing Receivable [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	71,919	58,881
NonCovered [Member] | Commercial Portfolio Segment [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	6,894	4,988
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	2,325	814
Financing Receivable, Recorded Investment, Past Due	9,219	5,802
Current	5,184,516	1,238,983
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	5,193,735	1,244,785	1,140,219
NonCovered [Member] | SBA 7(a) Financing Receivable [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	42,333	28,641
NonCovered [Member] | Equipment Finance Commercial Financing Receivable [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	932,554	273,483
NonCovered [Member] | Cash Flow [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	2,138,917	0
NonCovered [Member] | Asset Based [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	1,488,212	202,428
NonCovered [Member] | Unsecured Commercial Member [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	145,497	153,752
NonCovered [Member] | Collateralized Loan Obligations [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	446,222	586,481
NonCovered [Member] | Consumer Portfolio Segment [Member]
Delinquent loans in loan portfolio
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)		54,885	28,596
Consumer Financing Receivable [Member] | NonCovered [Member] | Consumer Portfolio Segment [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	128	3,319
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	3,235	0
Financing Receivable, Recorded Investment, Past Due	3,363	3,319
Current	59,642	51,566
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	63,005	54,885
Collateralized Loan Obligations [Member] | NonCovered [Member] | Commercial Portfolio Segment [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	6,606	473
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	182	259
Financing Receivable, Recorded Investment, Past Due	6,788	732
Current	439,434	585,749
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	446,222	586,481
Unsecured Commercial Member [Member] | NonCovered [Member] | Commercial Portfolio Segment [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	145	83
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	18	68
Financing Receivable, Recorded Investment, Past Due	163	151
Current	145,334	153,601
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	145,497	153,752
Asset Based [Member] | NonCovered [Member] | Commercial Portfolio Segment [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	0	0
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	0	0
Financing Receivable, Recorded Investment, Past Due	0	0
Current	1,488,212	202,428
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	1,488,212	202,428
Cash Flow [Member] | NonCovered [Member] | Commercial Portfolio Segment [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	0
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	0
Financing Receivable, Recorded Investment, Past Due	0
Current	2,138,917
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	2,138,917
Equipment Finance Commercial Financing Receivable [Member] | NonCovered [Member] | Commercial Portfolio Segment [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	0	2,662
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	1,941	244
Financing Receivable, Recorded Investment, Past Due	1,941	2,906
Current	930,613	270,577
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	932,554	273,483
SBA 7(a) Financing Receivable [Member] | NonCovered [Member] | Commercial Portfolio Segment [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	143	1,770
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	184	243
Financing Receivable, Recorded Investment, Past Due	327	2,013
Current	42,006	26,628
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	42,333	28,641
Residential Real Estate Construction Financing Receivable [Member] | NonCovered [Member] | Real Estate Construction [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	0	0
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	542	0
Financing Receivable, Recorded Investment, Past Due	542	0
Current	71,377	58,881
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	71,919	58,881
Commercial Real Estate Construction Financing Receivable [Member] | NonCovered [Member] | Real Estate Construction [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	0	0
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	1,487	2,013
Financing Receivable, Recorded Investment, Past Due	1,487	2,013
Current	225,071	146,885
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	226,558	148,898
Hospitality [Member] | NonCovered [Member] | Mortgage Loans on Real Estate [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	0	0
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	0	0
Financing Receivable, Recorded Investment, Past Due	0	0
Current	549,306	180,553
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	549,306	180,553
SBA 504 Loans [Member] | NonCovered [Member] | Mortgage Loans on Real Estate [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	3,125	2,564
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	1,788	0
Financing Receivable, Recorded Investment, Past Due	4,913	2,564
Current	341,485	42,602
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	346,398	45,166
Real Estate Other Class [Member] | NonCovered [Member] | Mortgage Loans on Real Estate [Member]
Delinquent loans in loan portfolio
30-89 Days Past Due	1,974	13,026
Financing Receivable, Recorded Investment, Equal to Greater than 90 Days Past Due	12,903	2,406
Financing Receivable, Recorded Investment, Past Due	14,877	15,432
Current	4,325,871	2,180,956
Loans held for investment, net (including $0 and $340.0 million, respectively, of loans that can only be used to settle obligations of consolidated VIEs)	$ 4,340,748	$ 2,196,388