Related Party Transactions (Details) - USD ($) $ in Thousands		3 Months Ended			9 Months Ended
		Sep. 30, 2017		Sep. 30, 2016	Sep. 30, 2017		Sep. 30, 2016	Dec. 31, 2016
Related party transactions [Line Items]
Annual base management fee percentage					1.50%
Management fee description					The Operating Partnership pays the Manager 1.50% per annum of total equity of the Operating Partnership calculated in accordance with U.S. GAAP as of the end of each fiscal quarter (before deductions for base management fees and incentive fees payable with respect to such fiscal quarter), provided that total equity is adjusted to exclude one-time events pursuant to changes in U.S. GAAP, as well as non-cash charges after discussion between the Manager and the Company's independent directors, and approval by a majority of the Company's independent directors in the case of non-cash charges.
Management Fee Expense	[1]	$ 2,161		$ 2,485	$ 6,942		$ 7,649
Incentive fee rate					25.00%
Incentive fees arrangement					The Manager is entitled to receive a quarterly incentive fee equal to the positive excess, if any, of (i) the product of (A) 25% and (B) the excess of (1) Adjusted Net Income (described below) for the Incentive Calculation Period (which means such fiscal quarter and the immediately preceding three fiscal quarters) over (2) the sum of the Hurdle Amounts (described below) for the Incentive Calculation Period, over (ii) the sum of the incentive fees already paid or payable for each fiscal quarter in the Incentive Calculation Period preceding such fiscal quarter. For purposes of calculating the incentive fee, "Adjusted Net Income" for the Incentive Calculation Period means the net increase in equity from operations of the Operating Partnership, after all base management fees but before any incentive fees for such period, and excluding any non-cash equity compensation expenses for such period, as reduced by any Loss Carryforward (as described below) as of the end of the fiscal quarter preceding the Incentive Calculation Period. For purposes of calculating the incentive fee, the "Loss Carryforward" as of the end of any fiscal quarter is calculated by determining the excess, if any, of (1) the Loss Carryforward as of the end of the immediately preceding fiscal quarter over (2) the Company's net increase in equity from operations (expressed as a positive number) or net decrease in equity from operations (expressed as a negative number) of the Operating Partnership for such fiscal quarter. As of September 30, 2017, there was no Loss Carryforward. For purposes of calculating the incentive fee, the "Hurdle Amount" means, with respect to any fiscal quarter, the product of (i) one-fourth of the greater of (A) 9% and (B) 3% plus the 10-year U.S. Treasury rate for such fiscal quarter, (ii) the sum of (A) the weighted average gross proceeds per share of all common share and OP Unit issuances since inception of the Company and up to the end of such fiscal quarter, with each issuance weighted by both the number of shares and OP Units issued in such issuance and the number of days that such issued shares and OP Units were outstanding during such fiscal quarter, using a first-in first-out basis of accounting (i.e. attributing any share and OP Unit repurchases to the earliest issuances first) and (B) the result obtained by dividing (I) retained earnings attributable to common shares and OP Units at the beginning of such fiscal quarter by (II) the average number of common shares and OP Units outstanding for each day during such fiscal quarter, (iii) the sum of the average number of common shares, LTIP Units, and OP Units outstanding for each day during such fiscal quarter. For purposes of determining the Hurdle Amount, issuances of common shares and OP Units (a) as equity incentive awards, (b) to the Manager as part of its base management fee or incentive fee and (c) to the Manager or any of its affiliates in privately negotiated transactions, are excluded from the calculation. The payment of the incentive fee will be in a combination of common shares and cash, provided that at least 10% of any quarterly payment will be made in common shares.
Termination Fee, Description					The Management Agreement requires the Company to pay a termination fee to the Manager in the event of (1) the Company's termination or non-renewal of the Management Agreement without cause or (2) the Company's termination of the Management Agreement based on unsatisfactory performance by the Manager that is materially detrimental to the Company (3) the Manager's termination of the Management Agreement upon a default by the Company in the performance of any material term of the Management Agreement. Such termination fee will be equal to the amount of three times the sum of (i) the average annual Quarterly Base Management Fee Amounts paid or payable with respect to the two 12-month periods ending on the last day of the latest fiscal quarter completed on or prior to the date of the notice of termination or non-renewal and (ii) the average annual Quarterly Incentive Fee Amounts paid or payable with respect to the two 12-month periods ending on the last day of the latest fiscal quarter completed on or prior to the date of the notice of termination or non-renewal.
Incentive fee hurdle rate fixed					9.00%
Incentive fee hurdle rate floating					3.00%
Minimum percentage of incentive fee to be paid in share					10.00%
Expense Reimbursement Period					60 days
Expense Reimbursement - Manager					$ 4,100		5,200
Warehouse facility, monetary amount		$ 5,000			$ 5,000
Line of Credit Facility, Interest Rate at Period End		15.00%			15.00%
Advances made under warehouse facility		$ 0			$ 0
Fair Value		1,756,432			1,756,432			$ 1,505,026
Purchase commitment principal balance					68,400
Estimated remaining contingent purchase obligations		173,500			173,500
Other secured borrowings		89,646			89,646			24,086
Non-controlling interests		14,089			14,089			7,116
Outstanding debt of related party trust		89,600			89,600
Aggregate unpaid principal balance of loans sold					$ 124,000
Company's share of loans sold					51.00%
Co-participants share of loans sold					49.00%
Senior notes issued in securitization transaction					$ 87,000
Subordinated notes issued in securitization transaction					$ 18,700
Company's percentage ownership of jointly owned entity					51.00%
Affiliated co-participants percentage ownership of jointly owned entity					49.00%
Fair value repurchase agreements	[2]	193,070			$ 193,070			184,819
Investments held with dealers as collateral		1,160,000			1,160,000			1,150,000
Notes issued by CLO securitization					$ 373,600
Percentage of unsecured subordinated notes purchased					25.00%
Principal amount of notes purchased from related party CLO securitization					$ 36,600
Cost of notes purchased from related party CLO securitization					$ 35,900
Rate of return required					12.00%
Management fee rebate	[1]	172		$ 0	$ 172		$ 0
Minimum economic interest required to be retained					5.00%
Economic interest retained					6.00%
The Company [Member]
Related party transactions [Line Items]
Corporate loans transferred into CLO securitization					$ 62,000
Affiliated Entity [Member]
Related party transactions [Line Items]
Corporate loans transferred into CLO securitization					141,700
Secured senior notes [Member]
Related party transactions [Line Items]
Notes issued by CLO securitization					187,500
Secured subordinated notes [Member]
Related party transactions [Line Items]
Notes issued by CLO securitization					97,300
Unsecured subordinated notes [Member]
Related party transactions [Line Items]
Notes issued by CLO securitization					88,800
Secured borrowing facility collateralized by commercial mortgage loans and REO [Member]
Related party transactions [Line Items]
Other secured borrowings		$ 42,700			$ 42,700			24,100
Related Party CLO securitization [Member]
Related party transactions [Line Items]
Percentage non-controlling ownership of related party		25.00%			25.00%
Corporate Debt [Member]
Related party transactions [Line Items]
Fair Value		$ 89,308	[3],[4]		$ 89,308	[3],[4]		80,095	[5],[6]
Repurchase Agreements [Member] | Acquiror Repurchase Agreement [Member]
Related party transactions [Line Items]
Fair value repurchase agreements		5,700			5,700
Investments held with dealers as collateral	[5],[7],[8]	9,400			9,400
Residential mortgage loans and REO [Member]
Related party transactions [Line Items]
Fair value of loans held in related party trust		163,600			163,600			86,000
Commercial mortgage loans and REO [Member]
Related party transactions [Line Items]
Fair Value	[3],[4],[9],[10]	27,900			27,900
Consumer loans and asset-backed securities backed by consumer loans [Member]
Related party transactions [Line Items]
Fair Value		116,340	[3],[4],[10]		116,340	[3],[4],[10]		107,157	[5],[6],[11],[12]
Fair value of loans held in related party trust		96,100			96,100			43,200
Collateralized Loan Obligations [Member]
Related party transactions [Line Items]
Fair Value		135,064	[3],[4]		135,064	[3],[4]		44,956	[5],[6]
Collateralized Loan Obligations [Member] | Related Party CLO securitization [Member]
Related party transactions [Line Items]
Fair Value	[3],[4],[13]	29,800			29,800
Corporate Equity Investments [Member]
Related party transactions [Line Items]
Fair Value		38,117	[3],[4]		38,117	[3],[4]		33,788	[5],[6],[11]
Corporate Equity Investments [Member] | Related Party CLO securitization [Member]
Related party transactions [Line Items]
Fair Value	[3],[4],[14]	5,600			5,600
Related Party 1 [Member]
Related party transactions [Line Items]
Fair value of loans held in related party trust		9,700			9,700			$ 7,600
Non-controlling interest of joint venture partner [Member] | Unrelated third party joint venture interest [Member] | Commercial mortgage loans and REO [Member]
Related party transactions [Line Items]
Non-controlling interests		1,800			1,800
Non-controlling interest of joint venture partner [Member] | Related party joint venture interest [Member] | Commercial mortgage loans and REO [Member]
Related party transactions [Line Items]
Non-controlling interests		$ 5,200			$ 5,200

[1] See Note 7 for further details on management fee rebates.

[2] In the Company's Consolidated Statement of Assets, Liabilities, and Equity, all balances associated with repurchase agreements, reverse repurchase agreements, and financial derivatives are presented on a gross basis.

[3] See Note 2 and Note 3 in Notes to Consolidated Financial Statements.

[4] The table below shows the ratings on the Company's long investments from Moody's, Standard and Poor's, or Fitch, as well as the Company's long investments that were unrated but guaranteed by the Federal National Mortgage Association, the Federal Home Loan Mortgage Corporation, or the Government National Mortgage Association. Ratings tend to be a lagging credit indicator; as a result, the credit quality of the Company's long investment holdings may be lower than the credit quality implied based on the ratings listed below. In situations where an investment has a split rating, the lowest provided rating is used. The ratings descriptions include ratings qualified with a "+," "-," "1," "2," or "3."Rating Description Percent of EquityUnrated but Agency-Guaranteed 148.83%A/A/A 0.05%Aaa/AAA/AAA 0.24%Baa/BBB/BBB 2.59%Ba/BB/BB or below 45.48%Unrated 81.73%

[5] See Note 2 and Note 3 in Notes to Consolidated Financial Statements.

[6] The table below shows the Company's long investment ratings from Moody's, Standard and Poor's, or Fitch, as well as the Company's long investments that were unrated but guaranteed by the Federal National Mortgage Association, the Federal Home Loan Mortgage Corporation, or the Government National Mortgage Association. Ratings tend to be a lagging credit indicator; as a result, the credit quality of the Company's long investment holdings may be lower than the credit quality implied based on the ratings listed below. In situations where an investment has a split rating, the lowest provided rating is used. The ratings descriptions include ratings qualified with a "+," "-," "1," "2," or "3."Rating Description Percent of EquityUnrated but Agency-Guaranteed 139.27%Aaa/AAA/AAA 0.84%Aa/AA/AA 0.03%A/A/A 0.05%Baa/BBB/BBB 2.60%Ba/BB/BB or below 30.24%Unrated 60.39%

[7] In general, securities received pursuant to repurchase agreements were delivered to counterparties in short sale transactions.

[8] Repurchase agreement is between the Company and CILO 2016-LD1 Holdings LLC, an entity in which the Company has a beneficial interest and is co-owned by an affiliate of Ellington Management Group, L.L.C. CILO 2016-LD1 Holdings LLC owns subordinated notes issued by, as well as trust certificates representing ownership of, a securitization trust. See Note 7 to the Notes to Consolidated Financial Statements.

[9] Includes non-performing commercial loans in the amount of $6.6 million whereby principal and/or interest is past due and a maturity date is not applicable.

[10] Loans and real estate owned are beneficially owned by the Company through participation certificates in the various trusts that hold such investments. See Note 7 to the Notes to the Consolidated Financial Statements.

[11] Conformed to current period presentation.

[12] Loans and real estate owned are beneficially owned by the Company through participation certificates in the various trusts that hold such investments. See Note 7 to the Notes to Consolidated Financial Statements.

[13] Includes investment in collateralized loan obligation notes in the amount of $29.8 million that were issued and are managed by related parties of the Company. See Note 7 to the Notes to the Consolidated Financial Statements.

[14] Represents the Company's investment in a related party. See Note 7 to the Notes to the Consolidated Financial Statements.