MBS, CRT Securities and MSR Related Assets (Purchase Discounts) (Details) - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2018	Jun. 30, 2017	Jun. 30, 2018	Jun. 30, 2017
Changes in the components of the purchase discount on Non-Agency MBS
Realized credit losses	$ 0	$ 618	$ 0	$ 1,032
Discount Designated as Credit Reserve and OTTI | Non-Agency MBS
Changes in the components of the purchase discount on Non-Agency MBS
Balance at beginning of period	(572,580)	(653,337)	(593,227)	(694,241)
Impact of RMBS Issuer Settlement	0	0	0	0
Accretion of discount	0	0	0	0
Realized credit losses	10,954	13,139	19,401	25,463
Purchases	0	(484)	(535)	(484)
Sales	0	5,037	5,592	24,778
Net impairment losses recognized in earnings	0	(618)	0	(1,032)
Transfers/release of credit reserve	8,030	9,765	15,173	19,018
Balance at end of period	(553,596)	(626,498)	(553,596)	(626,498)
Accretable Discount | Non-Agency MBS
Changes in the components of the purchase discount on Non-Agency MBS
Balance at beginning of period	(199,659)	(269,724)	(215,325)	(278,191)
Impact of RMBS Issuer Settlement	(12,089)	0	(12,089)	0
Accretion of discount	17,530	20,223	34,746	41,840
Realized credit losses	0	0	0	0
Purchases	0	(1,520)	488	(1,520)
Sales	0	2,819	5,105	(1,078)
Net impairment losses recognized in earnings	0	0	0	0
Transfers/release of credit reserve	(8,030)	(9,765)	(15,173)	(19,018)
Balance at end of period	(202,248)	$ (257,967)	$ (202,248)	$ (257,967)
Jp Morgan Chase
Changes in the components of the purchase discount on Non-Agency MBS
Proceeds from Legal Settlements	$ 12,100