Segment Reporting Operating Profit and Loss by Segment (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2022	Dec. 31, 2021		Dec. 31, 2020
Segment Reporting Information [Line Items]
Interest income	$ 282,218	$ 175,505		$ 173,531
Interest expense	(141,777)	(44,030)		(61,665)
Noninterest Income	(80,447)	19,562	[1]	(65,207)	[1]
Operating Expenses	(84,965)	(65,441)		(44,838)
Income (Loss) from Continuing Operations before Equity Method Investments, Income Taxes, Noncontrolling Interest	(24,971)	85,596		1,821
Income Tax Expense (Benefit)	(17,716)	3,144		11,377
Earnings (losses) from investments in unconsolidated entities	(63,614)	58,104		37,933
Net Income (Loss), Including Portion Attributable to Noncontrolling Interest	(70,869)	140,556		28,377
Net income (loss) attributable to non-controlling interests	(822)	7,093		3,369
Dividends on preferred stock	15,292	8,117		7,763
Net income (loss) attributable to common stockholders	(85,339)	125,346		17,245
Depreciation	330
Investment Portfolio Segment
Segment Reporting Information [Line Items]
Interest income	277,141	175,419		173,081
Interest expense	(124,308)	(38,990)		(56,625)
Noninterest Income	(118,797)	21,393		116,456
Operating Expenses	(29,532)	(21,638)		(19,668)
Income (Loss) from Continuing Operations before Equity Method Investments, Income Taxes, Noncontrolling Interest	4,504	136,184		24,115
Income Tax Expense (Benefit)	0	0		0
Earnings (losses) from investments in unconsolidated entities	(63,614)	58,104		37,933
Net Income (Loss), Including Portion Attributable to Noncontrolling Interest	(59,110)	194,288		62,048
Net income (loss) attributable to non-controlling interests	212	5,294		3,357
Dividends on preferred stock	0	0		0
Net income (loss) attributable to common stockholders	(59,322)	188,994		58,691
Depreciation	0
Longbridge Segment
Segment Reporting Information [Line Items]
Interest income	2,859
Interest expense	(4,628)
Noninterest Income	39,935
Operating Expenses	(23,674)
Income (Loss) from Continuing Operations before Equity Method Investments, Income Taxes, Noncontrolling Interest	14,492
Income Tax Expense (Benefit)	0
Earnings (losses) from investments in unconsolidated entities	0
Net Income (Loss), Including Portion Attributable to Noncontrolling Interest	14,492
Net income (loss) attributable to non-controlling interests	32
Dividends on preferred stock	0
Net income (loss) attributable to common stockholders	14,460
Depreciation	330
Corporate and Other
Segment Reporting Information [Line Items]
Interest income	2,218	86		450
Interest expense	(12,841)	(5,040)		(5,040)
Noninterest Income	(1,585)	(1,831)		(4,590)
Operating Expenses	(31,759)	(43,803)		(25,170)
Income (Loss) from Continuing Operations before Equity Method Investments, Income Taxes, Noncontrolling Interest	(43,967)	(50,588)		(22,294)
Income Tax Expense (Benefit)	(17,716)	3,144		11,377
Earnings (losses) from investments in unconsolidated entities	0	0		0
Net Income (Loss), Including Portion Attributable to Noncontrolling Interest	(26,251)	(53,732)		(33,671)
Net income (loss) attributable to non-controlling interests	(1,066)	1,799		12
Dividends on preferred stock	15,292	8,117		7,763
Net income (loss) attributable to common stockholders	(40,477)	$ (63,648)		$ (41,446)
Depreciation	$ 0

[1] Conformed to current period presentation.