Mortgage Backed Securities Mortgage backed securities designated as AFS (Details) (USD $) In Thousands, unless otherwise specified	9 Months Ended		12 Months Ended
	Sep. 30, 2014		Dec. 31, 2013
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	$ 2,781,833		$ 3,254,946
Available-for-sale MBS, Unamortized Premium (Discount)	811,251		758,950
Available-for-sale MBS, Amortized Cost	3,593,084		4,013,896
Available-for-sale Debt Securities, Accumulated Gross Unrealized Gain, before Tax	54,491		45,114
Available-for-sale MBS, Gross Unrealized Loss	(32,182)		(69,566)
Available-for-sale MBS, Fair Value	3,615,393		3,989,444
Agency MBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	2,505,715		2,865,398
Available-for-sale MBS, Unamortized Premium (Discount)	540,739		627,047
Available-for-sale MBS, Amortized Cost	3,046,454		3,492,445
Available-for-sale Debt Securities, Accumulated Gross Unrealized Gain, before Tax	32,340		26,792
Available-for-sale MBS, Gross Unrealized Loss	(31,442)		(63,976)
Available-for-sale MBS, Fair Value	3,047,352		3,455,261
Agency MBS [Member] | Residential Mortgage Backed Securities [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	2,200,149		2,591,568
Available-for-sale MBS, Unamortized Premium (Discount)	120,667		154,220
Available-for-sale MBS, Amortized Cost	2,320,816		2,745,788
Available-for-sale Debt Securities, Accumulated Gross Unrealized Gain, before Tax	7,593		6,104
Available-for-sale MBS, Gross Unrealized Loss	(31,105)		(59,742)
Available-for-sale MBS, Fair Value	2,297,304		2,692,150
Available-for-sale MBS, Weighted Average Coupon	3.14%		3.22%
Agency MBS [Member] | Commercial Mortgage Backed Securities [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	305,566		273,830
Available-for-sale MBS, Unamortized Premium (Discount)	19,186		19,061
Available-for-sale MBS, Amortized Cost	324,752		292,891	[1]
Available-for-sale Debt Securities, Accumulated Gross Unrealized Gain, before Tax	13,803		10,793
Available-for-sale MBS, Gross Unrealized Loss	(224)		(900)
Available-for-sale MBS, Fair Value	338,331		302,784	[1]
Available-for-sale MBS, Weighted Average Coupon	5.22%		5.07%
Agency MBS [Member] | CMBS IO [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	0	[2]	0	[3]
Available-for-sale MBS, Unamortized Premium (Discount)	400,886		453,766
Available-for-sale MBS, Amortized Cost	400,886		453,766
Available-for-sale Debt Securities, Accumulated Gross Unrealized Gain, before Tax	10,944		9,895
Available-for-sale MBS, Gross Unrealized Loss	(113)		(3,334)
Available-for-sale MBS, Fair Value	411,717		460,327
Available-for-sale MBS, Weighted Average Coupon	0.83%		0.83%
Notional balance for interest only securities	9,461,823		10,160,502
Non-Agency MBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	276,118		389,548
Available-for-sale MBS, Unamortized Premium (Discount)	270,512		131,903
Available-for-sale MBS, Amortized Cost	546,630		521,451
Available-for-sale Debt Securities, Accumulated Gross Unrealized Gain, before Tax	22,151		18,322
Available-for-sale MBS, Gross Unrealized Loss	(740)		(5,590)
Available-for-sale MBS, Fair Value	568,041		534,183
Non-Agency MBS [Member] | Residential Mortgage Backed Securities [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	9,522		13,845
Available-for-sale MBS, Unamortized Premium (Discount)	(4)		(338)
Available-for-sale MBS, Amortized Cost	9,518		13,507
Available-for-sale Debt Securities, Accumulated Gross Unrealized Gain, before Tax	118		338
Available-for-sale MBS, Gross Unrealized Loss	(81)		(80)
Available-for-sale MBS, Fair Value	9,555		13,765
Available-for-sale MBS, Weighted Average Coupon	4.20%		4.61%
Non-Agency MBS [Member] | Commercial Mortgage Backed Securities [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	266,596		375,703
Available-for-sale MBS, Unamortized Premium (Discount)	(12,374)		(18,277)
Available-for-sale MBS, Amortized Cost	254,222		357,426
Available-for-sale Debt Securities, Accumulated Gross Unrealized Gain, before Tax	16,397		15,366
Available-for-sale MBS, Gross Unrealized Loss	(346)		(3,511)
Available-for-sale MBS, Fair Value	270,273		369,281
Available-for-sale MBS, Weighted Average Coupon	4.89%		5.10%
Non-Agency MBS [Member] | CMBS IO [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	0	[2]	0	[3]
Available-for-sale MBS, Unamortized Premium (Discount)	282,890		150,518
Available-for-sale MBS, Amortized Cost	282,890		150,518
Available-for-sale Debt Securities, Accumulated Gross Unrealized Gain, before Tax	5,636		2,618
Available-for-sale MBS, Gross Unrealized Loss	(313)		(1,999)
Available-for-sale MBS, Fair Value	288,213		151,137
Available-for-sale MBS, Weighted Average Coupon	0.69%		0.66%
Notional balance for interest only securities	7,246,832		4,274,957
Agency MBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	155,501		1,912,937
Available-for-sale MBS, Continuous Unrealized Loss Position, Less than 12 Months, Gross Unrealized Losses	(1,142)		(43,543)
Available-for-sale MBS, Number of Securities in Continuous Unrealized Loss Position for Less than One Year	25		150
Available-for-sale MBS, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	1,701,638		670,402
Available-for-sale MBS, Continuous Unrealized Loss Position, 12 Months or Longer, Gross Unrealized Losses	(30,299)		(20,433)
Available-for-sale MBS, Number of Securities in Continuous Unrealized Loss Position for Greater than or Equal to One Year	129		67
Non-Agency MBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	74,239		162,558
Available-for-sale MBS, Continuous Unrealized Loss Position, Less than 12 Months, Gross Unrealized Losses	(427)		(5,435)
Available-for-sale MBS, Number of Securities in Continuous Unrealized Loss Position for Less than One Year	17		39
Available-for-sale MBS, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	31,348		6,310
Available-for-sale MBS, Continuous Unrealized Loss Position, 12 Months or Longer, Gross Unrealized Losses	$ (313)		$ (155)
Available-for-sale MBS, Number of Securities in Continuous Unrealized Loss Position for Greater than or Equal to One Year	8		6

[1] (1) As of December 31, 2013, the Company had Agency CMBS with an amortized cost of $26,920 and fair value of $28,717 which were designated as trading securities and are not included in this table. The Company changed the designation of these Agency CMBS to AFS during the three months ended June 30, 2014. Changes in the fair value of these MBS while they were designated as trading were recognized in net income within "fair value adjustments, net". Changes in the fair value of these MBS, which are now designated as AFS, are recognized in "other comprehensive income". As of September 30, 2014, the Company does not have any MBS designated as trading.

[2] (1)The notional balance for Agency CMBS IO and non-Agency CMBS IO was $9,461,823 and $7,246,832, respectively, as of September 30, 2014. December 31, 2013 Par Net Premium (Discount) Amortized Cost Gross Unrealized Gain Gross Unrealized Loss Fair Value WACAgency: RMBS$2,591,568 $154,220 $2,745,788 $6,104 $(59,742) $2,692,150 3.22%CMBS (1)273,830 19,061 292,891 10,793 (900) 302,784 5.07%CMBS IO (2)— 453,766 453,766 9,895 (3,334) 460,327 0.83%Total Agency AFS:2,865,398 627,047 3,492,445 26,792 (63,976) 3,455,261 Non-Agency: RMBS13,845 (338) 13,507 338 (80) 13,765 4.61%CMBS375,703 (18,277) 357,426 15,366 (3,511) 369,281 5.10%CMBS IO (2)— 150,518 150,518 2,618 (1,999) 151,137 0.66%Total non-Agency AFS:389,548 131,903 521,451 18,322 (5,590) 534,183 Total AFS securities$3,254,946 $758,950 $4,013,896 $45,114 $(69,566) $3,989,444

[3] (2) The notional balance for the Agency CMBS IO and non-Agency CMBS IO was $10,160,502 and $4,274,957, respectively, as of December 31, 2013.