LOANS HELD FOR INVESTMENT, NET - Allowance for Credit Losses Activity (Details) - USD ($)	3 Months Ended
	Mar. 31, 2023	Mar. 31, 2022	Dec. 31, 2022
Allowance for loan losses [Roll Forward]
Allowance for credit losses, Beginning balance	$ 83,507,000
Recovery for credit losses	(3,648,000)	$ (1,592,000)
Allowance for credit losses, Ending balance	78,335,000
Non-accrual with No Allowance	5,637,000		$ 6,075,000
Non-accrual with Allowance	25,907,000		28,162,000
Reserve	19,721,000		22,389,000
Interest income on non-accrual loans held for investment
Interest income on non-accrual loans	0	0
Total real estate loans
Allowance for loan losses [Roll Forward]
Allowance for credit losses, Beginning balance	43,381,000	47,771,000
Recovery for credit losses	(2,763,000)	(2,781,000)
Recoveries		2,000
Allowance for credit losses, Ending balance	40,618,000	44,992,000
One-to-four family residential and cooperative/condominium apartment
Allowance for loan losses [Roll Forward]
Allowance for credit losses, Beginning balance	5,969,000	5,932,000
Recovery for credit losses	43,000	(1,404,000)
Allowance for credit losses, Ending balance	6,012,000	4,528,000
Non-accrual with Allowance	2,808,000		3,203,000
Reserve	138,000		181,000
Multifamily residential and residential mixed-use
Allowance for loan losses [Roll Forward]
Allowance for credit losses, Beginning balance	8,360,000	7,816,000
Recovery for credit losses	(747,000)	(757,000)
Recoveries		2,000
Allowance for credit losses, Ending balance	7,613,000	7,061,000
Commercial real estate ("CRE")
Allowance for loan losses [Roll Forward]
Allowance for credit losses, Beginning balance	27,329,000	29,166,000
Recovery for credit losses	(2,851,000)	(521,000)
Allowance for credit losses, Ending balance	24,478,000	28,645,000
Non-accrual with No Allowance	4,659,000		4,915,000
Non-accrual with Allowance	3,409,000		3,417,000
Reserve	1,010,000		1,424,000
Acquisition, development, and construction
Allowance for loan losses [Roll Forward]
Allowance for credit losses, Beginning balance	1,723,000	4,857,000
Recovery for credit losses	792,000	(99,000)
Allowance for credit losses, Ending balance	2,515,000	4,758,000
Non-accrual with No Allowance			657,000
Non-accrual with Allowance	657,000
Reserve	305,000
Commercial and Industrial ("C&I") Loans
Allowance for loan losses [Roll Forward]
Allowance for credit losses, Beginning balance	39,853,000	35,331,000
Recovery for credit losses	(834,000)	1,516,000
Charge-offs	(2,029,000)	(2,635,000)
Recoveries	487,000	51,000
Allowance for credit losses, Ending balance	37,477,000	34,263,000
Non-accrual with No Allowance	978,000		503,000
Non-accrual with Allowance	18,934,000		21,443,000
Reserve	18,169,000		20,685,000
Other Loans
Allowance for loan losses [Roll Forward]
Allowance for credit losses, Beginning balance	273,000	751,000
Recovery for credit losses	(34,000)	(389,000)
Charge-offs	(1,000)	(3,000)
Recoveries	2,000	1,000
Allowance for credit losses, Ending balance	240,000	360,000
Non-accrual with Allowance	99,000		99,000
Reserve	99,000		$ 99,000
Includes other loans
Allowance for loan losses [Roll Forward]
Allowance for credit losses, Beginning balance	83,507,000	83,853,000
Recovery for credit losses	(3,631,000)	(1,654,000)
Charge-offs	(2,030,000)	(2,638,000)
Recoveries	489,000	54,000
Allowance for credit losses, Ending balance	$ 78,335,000	$ 79,615,000