MBS and CRT Securities (Details 2)		3 Months Ended
	Dec. 31, 2014 USD ($)	Mar. 31, 2016 USD ($) Security	Mar. 31, 2015 USD ($)	Dec. 31, 2015 USD ($)
Fair Value
Less than 12 Months		$ 2,784,300,000
12 Months or more		1,747,548,000
Total		4,531,848,000
Unrealized Losses
Less than 12 Months		28,501,000
12 Months or more		23,856,000
Total		$ 52,357,000
Number of Securities
Less than 12 months (in security) | Security		217
12 months or more (in security) | Security		247
Unrealized loss on investment securities		$ (52,357,000)		$ (71,929,000)
Total OTTI losses		0	$ (395,000)
OTTI reclassified from OCI		0	(12,000)
Net Impairment Losses Recognized in Earnings		0	(407,000)
Roll-forward of the credit loss component of OTTI
Credit loss component of OTTI at beginning of period		36,820,000	36,115,000
Additions for credit related OTTI not previously recognized		0	395,000
Subsequent additional credit related OTTI recorded		0	12,000
Credit loss component of OTTI at end of period	$ 36,115,000	36,820,000	36,522,000
Changes in the components of the purchase discount on Non-Agency MBS
Cumulative effect adjustment on adoption of revised accounting standard for repurchase agreement financing	4,500,000	0	4,537,000
AOCI from AFS Securities:
Unrealized gain on AFS securities at beginning of period		585,250,000	813,515,000
Cumulative effect adjustment on adoption of revised accounting standard for repurchase agreement financing	4,500,000	0	4,537,000
Reclassification adjustment for MBS sales included in net income		(10,485,000)	(6,423,000)
Reclassification adjustment for OTTI included in net income		0	(407,000)
Change in AOCI from AFS securities		(56,099,000)	36,742,000
Balance at end of period	813,515,000	529,151,000	850,257,000
Total MBS
Fair Value
Less than 12 Months		2,688,898,000
12 Months or more		1,742,960,000
Total		4,431,858,000
Unrealized Losses
Less than 12 Months		27,430,000
12 Months or more		23,443,000
Total		$ 50,873,000
Number of Securities
Less than 12 months (in security) | Security		194
12 months or more (in security) | Security		246
Unrealized loss on investment securities		$ (50,873,000)		(68,782,000)
Agency Mortgage Backed Securities
Fair Value
Less than 12 Months		570,899,000
12 Months or more		1,494,324,000
Total		2,065,223,000
Unrealized Losses
Less than 12 Months		4,508,000
12 Months or more		15,329,000
Total		$ 19,837,000
Number of Securities
Less than 12 months (in security) | Security		96
12 months or more (in security) | Security		221
Unrealized loss on investment securities		$ (19,837,000)		(40,425,000)
AOCI from AFS Securities:
Unrealized (loss)/gain on AFS securities, net		13,226,000	12,885,000
MBS and CRT Securities Interest Income
Coupon interest		32,132,000	40,743,000
Effective yield adjustment		(8,135,000)	(9,070,000)
Interest income		23,997,000	31,673,000
Agency Mortgage Backed Securities | Fannie Mae
Fair Value
Less than 12 Months		566,376,000
12 Months or more		976,991,000
Total		1,543,367,000
Unrealized Losses
Less than 12 Months		4,507,000
12 Months or more		10,348,000
Total		$ 14,855,000
Number of Securities
Less than 12 months (in security) | Security		95
12 months or more (in security) | Security		129
Unrealized loss on investment securities		$ (14,855,000)		(25,830,000)
Agency Mortgage Backed Securities | Freddie Mac
Fair Value
Less than 12 Months		4,523,000
12 Months or more		517,333,000
Total		521,856,000
Unrealized Losses
Less than 12 Months		1,000
12 Months or more		4,981,000
Total		$ 4,982,000
Number of Securities
Less than 12 months (in security) | Security		1
12 months or more (in security) | Security		92
Unrealized loss on investment securities		$ (4,982,000)		(14,595,000)
Non-Agency MBS
Fair Value
Less than 12 Months		2,117,999,000
12 Months or more		248,636,000
Total		2,366,635,000
Unrealized Losses
Less than 12 Months		22,922,000
12 Months or more		8,114,000
Total		$ 31,036,000
Number of Securities
Less than 12 months (in security) | Security		98
12 months or more (in security) | Security		25
Unrealized loss on investment securities		$ (31,036,000)		(28,357,000)
Credit-related OTTI losses		0
AOCI from AFS Securities:
Unrealized (loss)/gain on AFS securities, net		(58,840,000)	26,150,000
Sales of MBS
Proceeds from sale of MBS categorized as available-for-sale		32,000,000	10,800,000
Gains on sales of MBS		9,700,000	6,400,000
Non-Agency MBS | Discount Designated as Credit Reserve and OTTI
Changes in the components of the purchase discount on Non-Agency MBS
Balance at beginning of period		(787,541,000)	(900,557,000)
Cumulative effect adjustment on adoption of revised accounting standard for repurchase agreement financing		0	(15,543,000)
Accretion of discount		0	0
Realized credit losses		18,050,000	19,624,000
Purchases		(4,294,000)	(34,000)
Sales		12,020,000	1,049,000
Net impairment losses recognized in earnings		0	(407,000)
Transfers/release of credit reserve		4,201,000	22,335,000
Balance at end of period	(900,557,000)	(757,564,000)	(873,533,000)
AOCI from AFS Securities:
Cumulative effect adjustment on adoption of revised accounting standard for repurchase agreement financing		0	(15,543,000)
Non-Agency MBS | Accretable Discount
Changes in the components of the purchase discount on Non-Agency MBS
Balance at beginning of period		(312,182,000)	(399,564,000)
Cumulative effect adjustment on adoption of revised accounting standard for repurchase agreement financing		0	1,832,000
Accretion of discount		21,406,000	24,800,000
Realized credit losses		0	0
Purchases		1,606,000	(3,410,000)
Sales		12,040,000	9,969,000
Net impairment losses recognized in earnings		0	0
Transfers/release of credit reserve		(4,201,000)	(22,335,000)
Balance at end of period	$ (399,564,000)	(281,331,000)	(388,708,000)
AOCI from AFS Securities:
Cumulative effect adjustment on adoption of revised accounting standard for repurchase agreement financing		0	1,832,000
Non-Agency MBS | Expected to Recover Par
Fair Value
Less than 12 Months		1,841,079,000
12 Months or more		181,507,000
Total		2,022,586,000
Unrealized Losses
Less than 12 Months		15,689,000
12 Months or more		3,324,000
Total		$ 19,013,000
Number of Securities
Less than 12 months (in security) | Security		55
12 months or more (in security) | Security		10
Unrealized loss on investment securities		$ (19,013,000)		(20,143,000)
Non-Agency MBS | Expected to Recover Less Than Par
Fair Value
Less than 12 Months		276,920,000
12 Months or more		67,129,000
Total		344,049,000
Unrealized Losses
Less than 12 Months		7,233,000
12 Months or more		4,790,000
Total		$ 12,023,000
Number of Securities
Less than 12 months (in security) | Security		43
12 months or more (in security) | Security		15
Unrealized loss on investment securities		$ (12,023,000)		(8,214,000)
Legacy Non-Agency MBS
Number of Securities
Unrealized loss on investment securities		(13,200,000)
MBS and CRT Securities Interest Income
Coupon interest		40,309,000	49,356,000
Effective yield adjustment		19,913,000	24,406,000
Interest income		60,222,000	73,762,000
RPL/NPL MBS
Number of Securities
Unrealized loss on investment securities		(17,800,000)
MBS and CRT Securities Interest Income
Coupon interest		24,370,000	19,185,000
Effective yield adjustment		1,560,000	344,000
Interest income		25,930,000	19,529,000
CRT securities
Fair Value
Less than 12 Months		95,402,000
12 Months or more		4,588,000
Total		99,990,000
Unrealized Losses
Less than 12 Months		1,071,000
12 Months or more		413,000
Total		$ 1,484,000
Number of Securities
Less than 12 months (in security) | Security		23
12 months or more (in security) | Security		1
Unrealized loss on investment securities		$ (1,484,000)		(3,147,000)
MBS and CRT Securities Interest Income
Coupon interest		2,356,000	1,202,000
Effective yield adjustment		336,000	158,000
Interest income		2,692,000	$ 1,360,000
CRT securities | Fair value option elected, debt securities
Fair Value
Total		24,700,000
Number of Securities
Unrealized loss on investment securities		$ (132,000)		$ (555,000)