Other Assets - Derivative Instruments (Details 2) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2016 USD ($) derivative	Sep. 30, 2015 USD ($) derivative	Sep. 30, 2016 USD ($) derivative	Sep. 30, 2015 USD ($) derivative	Sep. 30, 2016 USD ($) derivative	Mar. 31, 2016 USD ($)	Dec. 31, 2015 USD ($)
Derivative [Line Items]
Number of swaps in default or early termination that would trigger counterparty to terminate all swaps | derivative			1
Assets Pledged					$ 10,698,988		$ 11,410,397
Number of terminated derivatives | derivative	0	0	0	0
Net unrealized gains (losses)	$ (131,971)	$ (64,571)	$ (69,399)	$ (59,062)	(109,202)		(69,399)
Interest expense attributable to Swaps	48,167	43,703	143,487	130,492
AOCI from derivative hedging instruments:
Balance at beginning of period	(131,971)	(64,571)	(69,399)	(59,062)
Unrealized gain/(loss) on Swaps, net	22,769	(40,884)	(39,803)	(46,393)
Balance at end of period	(109,202)	(105,455)	(109,202)	(105,455)
Interest Rate Contract
Derivative [Line Items]
Aggregate fair value of assets needed to immediately settle					110,400
Associated accrued interest payable					1,100
Assets Pledged					149,956		109,142
Interest Rate Contract | Agency MBS, at fair value
Derivative [Line Items]
Assets Pledged					34,469		38,569
Interest Rate Contract | Restricted cash
Derivative [Line Items]
Assets Pledged					115,487		70,573
Swaps, at fair value
Derivative [Line Items]
Aggregate notional amount of derivatives					$ 3,000,000		$ 3,050,000
Weighted average fixed-pay rate					1.82%		1.82%
Weighted Average Variable Interest Rate					0.53%		0.34%
Interest expense attributable to Swaps	$ 10,170	$ 12,711	$ 31,279	$ 41,266
Weighted average Swap rate paid	1.82%	1.82%	1.82%	1.83%
Weighted average Swap rate received	0.49%	0.19%	0.45%	0.18%
Swaps, at fair value | Within 30 days
Derivative [Line Items]
Aggregate notional amount of derivatives					$ 0		$ 50,000
Weighted average fixed-pay rate					0.00%		2.13%
Weighted Average Variable Interest Rate					0.00%		0.42%
Swaps, at fair value | Over 30 days to 3 months
Derivative [Line Items]
Aggregate notional amount of derivatives						$ 100,000	$ 0
Weighted average fixed-pay rate						0.48%	0.00%
Weighted Average Variable Interest Rate						0.53%	0.00%
Swaps, at fair value | Over 3 months to 6 months
Derivative [Line Items]
Aggregate notional amount of derivatives					$ 50,000		$ 0
Weighted average fixed-pay rate					0.67%		0.00%
Weighted Average Variable Interest Rate					0.52%		0.00%
Swaps, at fair value | Over 6 months to 12 months
Derivative [Line Items]
Aggregate notional amount of derivatives					$ 300,000		$ 100,000
Weighted average fixed-pay rate					0.57%		0.48%
Weighted Average Variable Interest Rate					0.52%		0.32%
Swaps, at fair value | Over 12 months to 24 months
Derivative [Line Items]
Aggregate notional amount of derivatives					$ 550,000		$ 350,000
Weighted average fixed-pay rate					1.49%		0.58%
Weighted Average Variable Interest Rate					0.53%		0.27%
Swaps, at fair value | Over 24 months to 36 months
Derivative [Line Items]
Aggregate notional amount of derivatives					$ 200,000		$ 550,000
Weighted average fixed-pay rate					1.71%		1.49%
Weighted Average Variable Interest Rate					0.54%		0.32%
Swaps, at fair value | Over 36 months to 48 months
Derivative [Line Items]
Aggregate notional amount of derivatives					$ 1,500,000		$ 200,000
Weighted average fixed-pay rate					2.22%		1.71%
Weighted Average Variable Interest Rate					0.53%		0.42%
Swaps, at fair value | Over 48 months to 60 months
Derivative [Line Items]
Aggregate notional amount of derivatives					$ 200,000		$ 1,500,000
Weighted average fixed-pay rate					2.20%		2.22%
Weighted Average Variable Interest Rate					0.53%		0.36%
Swaps, at fair value | Over 60 months to 72 months
Derivative [Line Items]
Aggregate notional amount of derivatives					$ 0		$ 200,000
Weighted average fixed-pay rate					0.00%		2.20%
Weighted Average Variable Interest Rate					0.00%		0.30%
Swaps, at fair value | Over 72 months to 84 months
Derivative [Line Items]
Aggregate notional amount of derivatives						$ 100,000	$ 0
Weighted average fixed-pay rate						2.75%	0.00%
Weighted Average Variable Interest Rate						0.53%	0.00%
Swaps, at fair value | Over 84 Months
Derivative [Line Items]
Number of instruments held | derivative					1
Aggregate notional amount of derivatives					$ 0		$ 100,000
Weighted average fixed-pay rate					0.00%		2.75%
Weighted Average Variable Interest Rate					0.00%		0.40%
Swaps, at fair value | Hedging
Derivative [Line Items]
Aggregate notional amount of derivatives					$ 3,000,000
Net unrealized gains (losses)	$ (109,200)		$ (109,200)		$ (109,200)
Average maturity term of swaps			37 months
AOCI from derivative hedging instruments:
Balance at end of period	(109,200)		$ (109,200)
Swaps, at fair value | Minimum | LIBOR
Derivative [Line Items]
Derivative, variable interest rate, term			1 month
Swaps, at fair value | Maximum | Hedging
Derivative [Line Items]
Maximum maturity term of swaps			83 months
Swaps, at fair value | Maximum | LIBOR
Derivative [Line Items]
Derivative, variable interest rate, term			3 months
New Swaps
Derivative [Line Items]
Aggregate notional amount	$ 0		$ 0
Weighted average fixed-pay rate					0.00%
Number of new Swaps | derivative	0		0
Swaps amortized expired
Derivative [Line Items]
Aggregate notional amount	$ 0		$ 50,000
Swaps amortized expired | QTD
Derivative [Line Items]
Weighted average fixed-pay rate					0.00%
Swaps amortized expired | YTD
Derivative [Line Items]
Weighted average fixed-pay rate					2.13%