Borrowings under Repurchase Agreements and Securitized Debt (Details) $ in Thousands		3 Months Ended	12 Months Ended
		Mar. 31, 2016 USD ($) counterparties	Dec. 31, 2015 USD ($) counterparties
Repurchase Agreements and Securitized Debt [Line Items]
Number of Counterparties with Outstanding Reverse Repurchase Agreements | counterparties		18	18
Concentration Risk, Reverse Repurchase Agreements		17.00%	15.00%
Number Counterparties With Concentration Risk, Reverse Repurchase Agreements | counterparties		1	1
Forward Starting Reverse Repurchase Agreements		$ 8,100	$ 4,400
Investments held with dealers as collateral		1,330,000	1,350,000
Pledged Financial Instruments, Not Separately Reported, Securities For Repurchase Agreements Unsettled		27,900	16,800
Securitized Debt		23,238	$ 0
Loans and REO pledged as collateral		$ 39,100
Investment Interest Rate Range Start		3.25%
Minimum [Member]
Repurchase Agreements and Securitized Debt [Line Items]
Reverse Repurchase Agreements Maturity		30 days
Reverse Repurchase Agreements Remaining Terms		1 day	4 days
Reverse Repurchase Agreements Interest Rate		0.10%	0.37%
Maximum [Member]
Repurchase Agreements and Securitized Debt [Line Items]
Reverse Repurchase Agreements Maturity		180 days
Reverse Repurchase Agreements Remaining Terms		575 days	666 days
Reverse Repurchase Agreements Interest Rate		3.50%	2.92%
Reverse Repurchase Agreements [Member]
Repurchase Agreements and Securitized Debt [Line Items]
Cash collateral posted for securities sold under agreements to repurchase	[1],[2]	$ 34,491	$ 25,542
Security Owned and Pledged as Collateral, Fair Value		$ 2,700	$ 5,100

[1] For the purpose of this presentation, for each row the total amount of financial instruments transferred or pledged and cash collateral (received) or pledged may not exceed the applicable gross amount of assets or (liabilities) as presented here. Therefore, the Company has reduced the amount of financial instruments transferred or pledged as collateral related to the Company's reverse repurchase agreements and cash collateral pledged on the Company's financial derivative liabilities. Total financial instruments transferred or pledged as collateral on the Company's reverse repurchase agreements as of March 31, 2016 and December 31, 2015 were $1.34 billion and $1.36 billion, respectively. As of March 31, 2016 and December 31, 2015, total cash collateral on financial derivative assets excludes excess net cash collateral pledged of $16.3 million and $4.1 million, respectively. As of March 31, 2016 and December 31, 2015, total cash collateral on financial derivative liabilities excludes excess cash collateral pledged of $35.1 million and $45.8 million, respectively.

[2] When collateral is pledged to or pledged by a counterparty, it is often pledged or posted with respect to all positions with such counterparty, and in such cases such collateral cannot be specifically identified as relating to a specific asset or liability. As a result, in preparing the above tables, the Company has made assumptions in allocating pledged or posted collateral among the various rows.