Use of Special Purpose Entities and Variable Interest Entities (Loan Securitization Transactions) (Details) - USD ($) $ in Thousands	1 Months Ended	12 Months Ended
	Dec. 31, 2017	Dec. 31, 2019	Dec. 31, 2018
Variable Interest Entity [Line Items]
Securitized debt		$ 570,952	$ 684,420
Asset-backed Securities, Securitized Loans and Receivables
Variable Interest Entity [Line Items]
Unpaid principal balance residential whole loans sold		1,290,029	1,290,029
Securitized debt		$ 570,952	$ 684,420
Weighted average interest rate		3.68%	3.66%
Weighted average contractual maturity of Senior Bonds		30 years	31 years
Cash received		$ 802,815	$ 802,815
Debt issuance cost		$ 2,900	3,800
Debt instrument, basis spread on variable rate		3.00%
Debt instrument, coupon step-up period	36 months
Asset-backed Securities, Securitized Loans and Receivables | Senior Notes
Variable Interest Entity [Line Items]
Face amount		$ 802,817	802,817
Proceeds from senior bonds sold with step-up feature		493,200	582,800
Asset-backed Securities, Securitized Loans and Receivables | Rated and Non-Rated Certificates
Variable Interest Entity [Line Items]
Face amount		$ 275,174	$ 275,174