Residential Whole Loans (Purchased Credit Impaired Loans) (Details) $ in Thousands	Sep. 30, 2018 USD ($)
Accounts, Notes, Loans and Financing Receivable [Line Items]
Purchase credit impaired loans transaction closed in current period which were purchased in prior periods	$ 54,900
Purchased credit impaired loans | Residential whole loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Contractually required principal and interest	154,911
Contractual cash flows not expected to be collected (non-accretable yield)	(15,378)
Expected cash flows to be collected	139,533
Interest component of expected cash flows (accretable yield)	(41,947)
Fair value at the date of acquisition	$ 97,586