MBS, CRT Securities and MSR Related Assets (Purchase Discounts) (Details) - Non-Agency MBS - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2018	Mar. 31, 2017
Discount Designated as Credit Reserve and OTTI
Changes in the components of the purchase discount on Non-Agency MBS
Balance at beginning of period	$ (593,227)	$ (694,241)
Accretion of discount	0	0
Realized credit losses	8,447	12,324
Purchases	(535)	0
Sales	5,592	19,741
Net impairment losses recognized in earnings	0	(414)
Transfers/release of credit reserve	7,143	9,253
Balance at end of period	(572,580)	(653,337)
Accretable Discount
Changes in the components of the purchase discount on Non-Agency MBS
Balance at beginning of period	(215,325)	(278,191)
Accretion of discount	17,216	21,616
Realized credit losses	0	0
Purchases	488	0
Sales	5,105	(3,897)
Net impairment losses recognized in earnings	0	0
Transfers/release of credit reserve	(7,143)	(9,253)
Balance at end of period	$ (199,659)	$ (269,725)