Residential Whole Loans (Fair Value Components of Net Income) (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2021	Sep. 30, 2020	Sep. 30, 2021	Sep. 30, 2020
Receivables [Abstract]
Net unrealized gains/(losses)	$ 20,494	$ 58,863	$ 58,807	$ (13,683)
Other income	1,321	1,453	518	3,601
Total	$ 21,815	$ 60,316	$ 59,325	$ (10,082)