Residential Whole Loans (Purchased Credit Impaired Loans - Narrative) (Details) - Purchased credit impaired loans - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2019	Jun. 30, 2018	Jun. 30, 2019	Jun. 30, 2018	Mar. 31, 2019	Dec. 31, 2018	Mar. 31, 2018	Dec. 31, 2017
Financing Receivable, Impaired [Line Items]
Allowance for loan losses	$ 1,536	$ 297	$ 1,536	$ 297	$ 1,151	$ 968	$ 380	$ 330
Provisions/(reversal of provisions) for loan losses	$ 385		$ 568
Reversal of provision for loan losses		$ (83)		$ (33)