MBS, CRT Securities and MSR Related Assets (MSR Related Assets - Narrative) (Details) - USD ($) $ in Thousands	3 Months Ended		6 Months Ended		12 Months Ended
	Jun. 30, 2018	Jun. 30, 2017	Jun. 30, 2018	Jun. 30, 2017	Dec. 31, 2017	Mar. 31, 2018
Term notes backed by MSR related collateral
Debt Securities, Available-for-sale [Line Items]
Term notes backed by MSR related collateral	$ 381,390		$ 381,390		$ 381,804
Amortized costs	380,400		380,400		381,000
Gross unrealized gains	1,000		$ 1,000		$ 804
Weighted average yield			5.56%		5.80%
Weighted average to maturity			4 years 1 month 6 days		3 years 5 months 4 days
Corporate Loan
Debt Securities, Available-for-sale [Line Items]
Commitment to lend	130,000		$ 130,000
Amount drawn	124,200		124,200			$ 124,200
Interest income	1,200	$ 2,000	3,700	$ 3,700
Discount accretion	1,100	$ 73	$ 1,200	$ 135
Commitment fee income	$ 136