Valuation (Fair Value Measurement Using Significant Unobservable Inputs) (Details) - Level 3 [Member] - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2018	Mar. 31, 2017
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	$ 661,588	$ 523,837
Assets, Accreted Discounts/Amortized Premiums	(6,329)	(4,228)
Assets, Realized Gain (Loss)	4,342	743
Assets, Change In Net Unrealized Gain/(Loss)	9,128	10,387
Assets, Purchases	174,297	206,874
Assets, Sales	(90,246)	(106,310)
Assets, Transfers into Level 3	11,561	3,473
Assets, Transfers out of Level 3	(5,618)	(5,414)
Assets, Ending Balance	758,723	629,362
Liabilities, Begining Balance	(125,105)	(505)
Liabilities, Accreted Discounts/Amortized Premiums	0	0
Liabilities, Realized Gain/(Loss)	0	(652)
Liabilities, Change In Net Unrealized Gain/(Loss)	784	714
Liabilities Purchases/Payments	10,546	301
Liabilities, Sales/Issuance	0	(76)
Liabilities, Transfers into Level 3	0	0
Liabilities, Transfers out of Level 3	0	0
Liabilities, Ending Balance	(113,775)	(218)
Other secured borrowings [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Liabilities, Begining Balance	(125,105)
Liabilities, Accreted Discounts/Amortized Premiums	0
Liabilities, Realized Gain/(Loss)	0
Liabilities, Change In Net Unrealized Gain/(Loss)	784
Liabilities, Payments	10,546
Liabilities, Issuances	0
Liabilities, Transfers into Level 3	0
Liabilities, Transfers out of Level 3	0
Liabilities, Ending Balance	(113,775)
Investments [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	658,448	518,250
Assets, Accreted Discounts/Amortized Premiums	(6,329)	(4,228)
Assets, Realized Gain (Loss)	4,256	649
Assets, Change In Net Unrealized Gain/(Loss)	9,199	10,046
Assets, Purchases	174,273	206,815
Assets, Sales	(90,136)	(106,057)
Assets, Transfers into Level 3	11,561	3,473
Assets, Transfers out of Level 3	(5,618)	(5,414)
Assets, Ending Balance	755,654	623,534
Fair Value, Assets Measured on Recurring Basis, Change in Unrealized Gain (Loss)	8,600	6,900
Investments [Member] | Agency residential mortgage-backed securities [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	6,173	29,622
Assets, Accreted Discounts/Amortized Premiums	(600)	(2,592)
Assets, Realized Gain (Loss)	39	(35)
Assets, Change In Net Unrealized Gain/(Loss)	264	228
Assets, Purchases	1,101	2,202
Assets, Sales	(388)	0
Assets, Transfers into Level 3	0	0
Assets, Transfers out of Level 3	(461)	0
Assets, Ending Balance	6,128	29,425
Investments [Member] | Private Label Residential Mortgage-Backed Securities [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	101,297	90,083
Assets, Accreted Discounts/Amortized Premiums	106	1,478
Assets, Realized Gain (Loss)	2,288	(703)
Assets, Change In Net Unrealized Gain/(Loss)	293	2,893
Assets, Purchases	20,660	8,049
Assets, Sales	(21,574)	(19,527)
Assets, Transfers into Level 3	11,561	3,473
Assets, Transfers out of Level 3	(2,769)	(5,414)
Assets, Ending Balance	111,862	80,332
Investments [Member] | Private Label Commercial Mortgage-Backed Securities [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	12,347	43,268
Assets, Accreted Discounts/Amortized Premiums	(183)	356
Assets, Realized Gain (Loss)	1,554	342
Assets, Change In Net Unrealized Gain/(Loss)	121	1,411
Assets, Purchases	9,624	0
Assets, Sales	(7,366)	(4,077)
Assets, Transfers into Level 3	0	0
Assets, Transfers out of Level 3	(2,388)	0
Assets, Ending Balance	13,709	41,300
Investments [Member] | Commercial Mortgage Loans [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	108,301	61,129
Assets, Accreted Discounts/Amortized Premiums	618	530
Assets, Realized Gain (Loss)	330	337
Assets, Change In Net Unrealized Gain/(Loss)	(161)	1,206
Assets, Purchases	3,988	23,045
Assets, Sales	(3,782)	(23,739)
Assets, Transfers into Level 3	0	0
Assets, Transfers out of Level 3	0	0
Assets, Ending Balance	109,294	62,508
Investments [Member] | Residential Mortgage Loans [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	182,472	84,290
Assets, Accreted Discounts/Amortized Premiums	(715)	140
Assets, Realized Gain (Loss)	(54)	(51)
Assets, Change In Net Unrealized Gain/(Loss)	(653)	404
Assets, Purchases	73,040	34,138
Assets, Sales	(13,309)	(6,271)
Assets, Transfers into Level 3	0	0
Assets, Transfers out of Level 3	0	0
Assets, Ending Balance	240,781	112,650
Investments [Member] | Collateralized Loan Obligations [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	24,911	44,956
Assets, Accreted Discounts/Amortized Premiums	455	(1,183)
Assets, Realized Gain (Loss)	2	920
Assets, Change In Net Unrealized Gain/(Loss)	226	2,532
Assets, Purchases	10,095	38,712
Assets, Sales	(8,210)	(15,376)
Assets, Transfers into Level 3	0	0
Assets, Transfers out of Level 3	0	0
Assets, Ending Balance	27,479	70,561
Investments [Member] | Consumer loans and asset-backed securities backed by consumer loans [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	135,258	107,157	[1]
Assets, Accreted Discounts/Amortized Premiums	(5,896)	(3,057)	[1]
Assets, Realized Gain (Loss)	501	(552)	[1]
Assets, Change In Net Unrealized Gain/(Loss)	3,804	837	[1]
Assets, Purchases	42,133	24,418	[1]
Assets, Sales	(27,378)	(20,961)	[1]
Assets, Transfers into Level 3	0	0	[1]
Assets, Transfers out of Level 3	0	0	[1]
Assets, Ending Balance	148,422	107,842	[1]
Investments [Member] | Corporate Debt [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	23,947	25,004
Assets, Accreted Discounts/Amortized Premiums	(114)	100
Assets, Realized Gain (Loss)	52	331
Assets, Change In Net Unrealized Gain/(Loss)	364	158
Assets, Purchases	456	47,094
Assets, Sales	(6,705)	(13,078)
Assets, Transfers into Level 3	0	0
Assets, Transfers out of Level 3	0	0
Assets, Ending Balance	18,000	59,609
Investments [Member] | Real Estate Owned [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	26,277	3,349
Assets, Accreted Discounts/Amortized Premiums	0	0
Assets, Realized Gain (Loss)	(456)	60
Assets, Change In Net Unrealized Gain/(Loss)	615	106
Assets, Purchases	4,098	24,157
Assets, Sales	(1,424)	(2,282)
Assets, Transfers into Level 3	0	0
Assets, Transfers out of Level 3	0	0
Assets, Ending Balance	29,110	25,390
Investments [Member] | Corporate Equity Investments [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	37,465	29,392	[1]
Assets, Accreted Discounts/Amortized Premiums	0	0	[1]
Assets, Realized Gain (Loss)	0	0	[1]
Assets, Change In Net Unrealized Gain/(Loss)	4,326	271	[1]
Assets, Purchases	9,078	5,000	[1]
Assets, Sales	0	(746)	[1]
Assets, Transfers into Level 3	0	0	[1]
Assets, Transfers out of Level 3	0	0	[1]
Assets, Ending Balance	50,869	33,917	[1]
Financial Derivatives - Assets [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	3,140	5,587
Assets, Accreted Discounts/Amortized Premiums	0	0
Assets, Realized Gain (Loss)	86	94
Assets, Change In Net Unrealized Gain/(Loss)	(71)	341
Assets, Purchases	24	59
Assets, Sales	(110)	(253)
Assets, Transfers into Level 3	0	0
Assets, Transfers out of Level 3	0	0
Assets, Ending Balance	3,069	5,828
Fair Value, Assets Measured on Recurring Basis, Change in Unrealized Gain (Loss)	(100)	500
Financial Derivatives - Assets [Member] | Total Return Swaps [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance		155
Assets, Accreted Discounts/Amortized Premiums		0
Assets, Realized Gain (Loss)		157
Assets, Change In Net Unrealized Gain/(Loss)		(155)
Assets, Purchases		0
Assets, Sales		(157)
Assets, Transfers into Level 3		0
Assets, Transfers out of Level 3		0
Assets, Ending Balance		0
Financial Derivatives - Assets [Member] | Warrant [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance		106
Assets, Accreted Discounts/Amortized Premiums		0
Assets, Realized Gain (Loss)		(100)
Assets, Change In Net Unrealized Gain/(Loss)		(6)
Assets, Purchases		0
Assets, Sales		0
Assets, Transfers into Level 3		0
Assets, Transfers out of Level 3		0
Assets, Ending Balance		0
Financial Derivatives - Assets [Member] | Credit Default Swaps On Asset Backed Securities [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Assets, Beginning Balance	3,140	5,326
Assets, Accreted Discounts/Amortized Premiums	0	0
Assets, Realized Gain (Loss)	86	37
Assets, Change In Net Unrealized Gain/(Loss)	(71)	502
Assets, Purchases	24	59
Assets, Sales	(110)	(96)
Assets, Transfers into Level 3	0	0
Assets, Transfers out of Level 3	0	0
Assets, Ending Balance	3,069	5,828
Financial Derivatives - Liabilities [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Fair Value, Liabilities Measured on Recurring Basis, Change in Unrealized Gain (Loss)		500
Financial Derivatives - Liabilities [Member] | Total Return Swaps [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Liabilities, Begining Balance		(249)
Liabilities, Accreted Discounts/Amortized Premiums		0
Liabilities, Realized Gain/(Loss)		(206)
Liabilities, Change In Net Unrealized Gain/(Loss)		248
Liabilities, Purchases		283
Liabilities, Sales		(76)
Liabilities, Transfers into Level 3		0
Liabilities, Transfers out of Level 3		0
Liabilities, Ending Balance		0
Financial Derivatives - Liabilities [Member] | Credit Default Swaps On Asset Backed Securities [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Liabilities, Begining Balance		(256)
Liabilities, Accreted Discounts/Amortized Premiums		0
Liabilities, Realized Gain/(Loss)		(446)
Liabilities, Change In Net Unrealized Gain/(Loss)		466
Liabilities, Purchases		18
Liabilities, Sales		0
Liabilities, Transfers into Level 3		0
Liabilities, Transfers out of Level 3		0
Liabilities, Ending Balance		$ (218)
Other secured borrowings [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Fair Value, Assets Measured on Recurring Basis, Change in Unrealized Gain (Loss)	$ 800

[1] Conformed to current period presentation.