MBS, CRT Securities and MSR Related Assets (Unrealized Losses) (Details) $ in Thousands	Sep. 30, 2018 USD ($) security	Dec. 31, 2017 USD ($)
Fair Value
Less than 12 Months	$ 1,602,941
12 Months or more	1,140,526
Total	2,743,467
Unrealized Losses
Less than 12 Months	12,385
12 Months or more	55,861
Total	$ 68,246
Number of Securities
Less than 12 months (in security) | security	167
12 months or more (in security) | security	361
Available-for-sale securities	$ 6,779,045	$ 7,023,050
Gross Unrealized Losses	68,246	43,613
Agency MBS
Fair Value
Less than 12 Months	1,107,546
12 Months or more	1,136,649
Total	2,244,195
Unrealized Losses
Less than 12 Months	10,313
12 Months or more	55,795
Total	$ 66,108
Number of Securities
Less than 12 months (in security) | security	142
12 months or more (in security) | security	354
Available-for-sale securities	$ 2,905,490	2,824,681
Gross Unrealized Losses	66,108	43,137
Agency MBS | Fannie Mae
Fair Value
Less than 12 Months	438,980
12 Months or more	838,566
Total	1,277,546
Unrealized Losses
Less than 12 Months	4,293
12 Months or more	35,918
Total	$ 40,211
Number of Securities
Less than 12 months (in security) | security	105
12 months or more (in security) | security	251
Available-for-sale securities	$ 1,896,739	2,246,600
Gross Unrealized Losses	40,211	28,341
Agency MBS | Freddie Mac
Fair Value
Less than 12 Months	668,566
12 Months or more	298,083
Total	966,649
Unrealized Losses
Less than 12 Months	6,020
12 Months or more	19,877
Total	$ 25,897
Number of Securities
Less than 12 months (in security) | security	37
12 months or more (in security) | security	103
Available-for-sale securities	$ 1,003,583	571,748
Gross Unrealized Losses	25,897	14,796
Non-Agency MBS
Fair Value
Less than 12 Months	479,641
12 Months or more	3,877
Total	483,518
Unrealized Losses
Less than 12 Months	2,068
12 Months or more	66
Total	$ 2,134
Number of Securities
Less than 12 months (in security) | security	21
12 months or more (in security) | security	7
Available-for-sale securities	$ 3,334,610	3,533,966
Gross Unrealized Losses	2,134	453
Non-Agency MBS | Expected to Recover Par
Fair Value
Less than 12 Months	435,011
12 Months or more	3,877
Total	438,888
Unrealized Losses
Less than 12 Months	1,849
12 Months or more	66
Total	$ 1,915
Number of Securities
Less than 12 months (in security) | security	15
12 months or more (in security) | security	7
Available-for-sale securities	$ 1,343,272	1,132,205
Gross Unrealized Losses	1,915	434
Non-Agency MBS | Expected to Recover Less Than Par
Fair Value
Less than 12 Months	44,630
12 Months or more	0
Total	44,630
Unrealized Losses
Less than 12 Months	219
12 Months or more	0
Total	$ 219
Number of Securities
Less than 12 months (in security) | security	6
12 months or more (in security) | security	0
Available-for-sale securities	$ 1,991,338	2,401,761
Gross Unrealized Losses	219	19
Total MBS
Fair Value
Less than 12 Months	1,587,187
12 Months or more	1,140,526
Total	2,727,713
Unrealized Losses
Less than 12 Months	12,381
12 Months or more	55,861
Total	$ 68,242
Number of Securities
Less than 12 months (in security) | security	163
12 months or more (in security) | security	361
Available-for-sale securities	$ 6,240,100	6,358,647
Gross Unrealized Losses	68,242	43,590
CRT securities
Fair Value
Less than 12 Months	15,754
12 Months or more	0
Total	15,754
Unrealized Losses
Less than 12 Months	4
12 Months or more	0
Total	$ 4
Number of Securities
Less than 12 months (in security) | security	4
12 months or more (in security) | security	0
Available-for-sale securities	$ 538,945	664,403
Gross Unrealized Losses	4	$ 23
Agency MBS, Fair Value Option
Number of Securities
Available-for-sale securities	746,700
Gross Unrealized Losses	$ 5,500