Fair Value of Financial Instruments Recorded basis and Fair Value (Details) - USD ($) $ in Thousands		Mar. 31, 2016	Dec. 31, 2015
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Derivative assets		$ 14,845	$ 7,835
Derivative liabilities		90,487	41,205
Recorded Basis [Member]
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Mortgage-backed Securities Available-for-sale, Fair Value Disclosure		3,328,752	3,493,701
Mortgage loans held for investment, net, Fair Value Disclosure		23,271	24,145
Investment in FHLB stock, Fair Value Disclosure		11,475	11,475
Derivative assets		14,845	7,835
Repurchase agreements, Fair Value Disclosure		2,722,019	2,589,420
FHLB advances, Fair Value Disclosure		263,000	520,000
Non-recourse collateralized financing, Fair Value		8,140	8,442
Derivative liabilities		90,487	41,205
Fair Value, Fair Value Disclosure [Member]
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Mortgage-backed Securities Available-for-sale, Fair Value Disclosure		3,328,752	3,493,701
Mortgage loans held for investment, net, Fair Value Disclosure	[1]	19,902	20,849
Investment in FHLB stock, Fair Value Disclosure	[2]	11,475	11,475
Derivative assets		14,845	7,835
Repurchase agreements, Fair Value Disclosure	[2]	2,722,019	2,589,420
FHLB advances, Fair Value Disclosure	[2]	263,000	520,000
Non-recourse collateralized financing, Fair Value	[1]	7,813	8,102
Derivative liabilities		$ 90,487	$ 41,205

[1] The Company determines the fair value of its mortgage loans held for investment, net and its non-recourse collateralized financing using internally developed cash flow models with inputs similar to those used to estimate the fair value of the Company's Level 3 non-Agency MBS.

[2] The carrying value of repurchase agreements and FHLB advances generally approximates fair value due to their short term maturities