Fair Value of Financial Instruments (Details 2) (USD $) In Thousands, unless otherwise specified	Jun. 30, 2014		Dec. 31, 2013
Financial Assets:
Agency MBS	$ 6,549,451		$ 6,519,221
Securities obtained and pledged as collateral	446,375		383,743
Restricted cash	54,430		37,520
Linked Transactions	106,859		28,181
Swaps	2,755		13,000
Financial Liabilities:
Repurchase agreements	8,384,101	[1]	8,339,297	[1]
Securitized debt	214,048	[2]	366,205	[2]
Obligation to return securities obtained as collateral	446,375		383,743
Senior Notes	100,000		100,000
Swaps	57,426	[3]	28,217	[3]
Carrying Value
Financial Assets:
Agency MBS	6,549,451		6,519,221
Non-Agency MBS, including MBS transferred to consolidated VIEs	5,001,337		4,852,137
Securities obtained and pledged as collateral	446,375		383,743
Cash and cash equivalents	343,719		565,370
Restricted cash	54,430		37,520
Linked Transactions	106,859		28,181
Swaps	2,755		13,000
Residential whole loans	59,746		0
Financial Liabilities:
Repurchase agreements	8,384,101		8,339,297
Securitized debt	214,048		366,205
Obligation to return securities obtained as collateral	446,375		383,743
Senior Notes	100,000		100,000
Swaps	57,426		28,217
Estimated Fair Value
Financial Assets:
Agency MBS	6,549,451		6,519,221
Non-Agency MBS, including MBS transferred to consolidated VIEs	5,001,337		4,852,137
Securities obtained and pledged as collateral	446,375		383,743
Cash and cash equivalents	343,719		565,370
Restricted cash	54,430		37,520
Linked Transactions	106,859		28,181
Swaps	2,755		13,000
Residential whole loans	59,746		0
Financial Liabilities:
Repurchase agreements	8,383,492		8,339,071
Securitized debt	214,821		366,767
Obligation to return securities obtained as collateral	446,375		383,743
Senior Notes	104,720		98,000
Swaps	$ 57,426		$ 28,217

[1] The fair value of securities pledged against the Company’s repurchase agreements was $10.220 billion and $10.116 billion at June 30, 2014 and December 31, 2013, respectively.

[2] Securitized Debt represents third-party liabilities of consolidated VIEs and excludes liabilities of the VIEs acquired by the Company that eliminate on consolidation. The third-party beneficial interest holders in the VIEs have no recourse to the general credit of the Company. (See Notes 10 and 15 for further discussion.)

[3] The fair value of securities pledged against the Company’s Swaps was $75.1 million and $73.9 million at June 30, 2014 and December 31, 2013, respectively.