Securitization Transactions (Schedule of Residential Loan Securitizations) (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2022	Mar. 31, 2021
Securitization Transactions [Line Items]
Unrealized gain (loss) on Other secured borrowings, at fair value	$ 55,600	$ 1,200
Non-QM loan securitization | February 2021
Securitization Transactions [Line Items]
Total Face Amount of Certificates Issued	251,771
Purchase of certificates	1,600
Non-QM loan securitization | November 2019 [Member]
Securitization Transactions [Line Items]
Total Face Amount of Certificates Issued	$ 267,255
Economic interest retained	6.40%
Purchase of certificates	$ 1,700
Non-QM loan securitization | June 2020 [Member]
Securitization Transactions [Line Items]
Total Face Amount of Certificates Issued	259,273
Purchase of certificates	1,900
Non-QM loan securitization | November 2020
Securitization Transactions [Line Items]
Total Face Amount of Certificates Issued	219,732
Purchase of certificates	1,400
Non-QM loan securitization | June 2021
Securitization Transactions [Line Items]
Total Face Amount of Certificates Issued	331,777
Purchase of certificates	2,100
Non-QM loan securitization | October 2021
Securitization Transactions [Line Items]
Total Face Amount of Certificates Issued	257,645
Purchase of certificates	1,800
Non-QM loan securitization | February 2022
Securitization Transactions [Line Items]
Total Face Amount of Certificates Issued	417,188
Purchase of certificates	3,600
The Company [Member] | Non-QM loan securitization | February 2021
Securitization Transactions [Line Items]
Principal Balance of Loans Transferred to the Depositor	251,771
The Company [Member] | Non-QM loan securitization | November 2019 [Member]
Securitization Transactions [Line Items]
Principal Balance of Loans Transferred to the Depositor	267,255
The Company [Member] | Non-QM loan securitization | June 2020 [Member]
Securitization Transactions [Line Items]
Principal Balance of Loans Transferred to the Depositor	259,273
The Company [Member] | Non-QM loan securitization | November 2020
Securitization Transactions [Line Items]
Principal Balance of Loans Transferred to the Depositor	219,732
The Company [Member] | Non-QM loan securitization | June 2021
Securitization Transactions [Line Items]
Principal Balance of Loans Transferred to the Depositor	331,777
The Company [Member] | Non-QM loan securitization | October 2021
Securitization Transactions [Line Items]
Principal Balance of Loans Transferred to the Depositor	257,645
The Company [Member] | Non-QM loan securitization | February 2022
Securitization Transactions [Line Items]
Principal Balance of Loans Transferred to the Depositor	$ 417,188
Retained, non-risk retention [Member] | Non-QM loan securitization | February 2021
Securitization Transactions [Line Items]
Economic interest retained	0.40%
Retained, non-risk retention [Member] | Non-QM loan securitization | June 2020 [Member]
Securitization Transactions [Line Items]
Economic interest retained	6.40%
Retained, non-risk retention [Member] | Non-QM loan securitization | November 2020
Securitization Transactions [Line Items]
Economic interest retained	2.40%
Retained, non-risk retention [Member] | Non-QM loan securitization | October 2021
Securitization Transactions [Line Items]
Economic interest retained	2.00%
Retained, risk retention rules [Member] | Non-QM loan securitization | February 2021
Securitization Transactions [Line Items]
Economic interest retained	5.40%
Retained, risk retention rules [Member] | Non-QM loan securitization | June 2020 [Member]
Securitization Transactions [Line Items]
Economic interest retained	5.10%
Retained, risk retention rules [Member] | Non-QM loan securitization | November 2020
Securitization Transactions [Line Items]
Economic interest retained	5.20%
Retained, risk retention rules [Member] | Non-QM loan securitization | June 2021
Securitization Transactions [Line Items]
Economic interest retained	6.50%
Retained, risk retention rules [Member] | Non-QM loan securitization | October 2021
Securitization Transactions [Line Items]
Economic interest retained	6.30%
Retained, risk retention rules [Member] | Non-QM loan securitization | February 2022
Securitization Transactions [Line Items]
Economic interest retained	5.30%