Residential Mortgage Securities and MSR-Related Assets (MSR Related Assets - Narrative) (Details) - USD ($) $ in Thousands	3 Months Ended	9 Months Ended
	Mar. 31, 2020	Sep. 30, 2020	Dec. 31, 2019
Term notes backed by MSR-related collateral
Debt Securities, Available-for-sale [Line Items]
Term notes backed by MSR related collateral		$ 234,091	$ 1,157,463
Amortized costs		184,500	1,200,000
Gross unrealized gains		$ 49,500	$ 5,200
Weighted average yield	4.75%	12.10%
Weighted average to maturity	5 years 3 months 18 days	9 years 6 months
Proceeds from sale of MSR		$ 711,700
Realized losses		28,700
Impairment		280,800
Corporate Loan
Debt Securities, Available-for-sale [Line Items]
Commitment to lend		32,600
Amount drawn		$ 18,100
Interest coupon rate		5.52%
Debt term		11 months
Minimum | Corporate Loan
Debt Securities, Available-for-sale [Line Items]
Commitment fee on undrawn amount		0.25%
Maximum | Corporate Loan
Debt Securities, Available-for-sale [Line Items]
Commitment fee on undrawn amount		1.00%