Loans - Net Loans Held-for-Investment (Details) - USD ($) $ in Thousands	Jun. 30, 2024	Mar. 31, 2024	Dec. 31, 2023	Jun. 30, 2023	Mar. 31, 2023	Dec. 31, 2022
Loans and Leases Receivable Disclosure [Line Items]
Total loans held-for-investment	$ 4,091,220		$ 4,203,654
Allowance for credit losses	(34,780)	$ (37,039)	(37,535)	$ (41,154)	$ (41,436)	$ (42,617)
Net loans held-for-investment	4,056,440		4,166,119
Financial Asset, Other than Financial Asset Acquired with Credit Deterioration
Loans and Leases Receivable Disclosure [Line Items]
Total loans held-for-investment	4,081,876		4,193,755
Allowance for credit losses	(31,835)	(33,974)	(34,450)	(37,428)	(37,674)	(38,734)
PCD loans
Loans and Leases Receivable Disclosure [Line Items]
Total loans held-for-investment	9,344		9,899
Allowance for credit losses	(2,945)	(3,065)	(3,085)	(3,726)	(3,762)	(3,883)
Real estate loans: | Financial Asset, Other than Financial Asset Acquired with Credit Deterioration
Loans and Leases Receivable Disclosure [Line Items]
Loans	3,913,766		4,035,902
Total loans held-for-investment	3,913,766		4,035,902
Real estate loans: | Financial Asset, Other than Financial Asset Acquired with Credit Deterioration | Multifamily
Loans and Leases Receivable Disclosure [Line Items]
Loans	2,665,202		2,750,996
Total loans held-for-investment	2,665,202		2,750,996
Real estate loans: | Financial Asset, Other than Financial Asset Acquired with Credit Deterioration | Commercial mortgage
Loans and Leases Receivable Disclosure [Line Items]
Loans	896,157		929,595
Total loans held-for-investment	896,157		929,595
Real estate loans: | Financial Asset, Other than Financial Asset Acquired with Credit Deterioration | One-to-four family residential mortgage
Loans and Leases Receivable Disclosure [Line Items]
Loans	151,948		160,824
Total loans held-for-investment	151,948		160,824
Allowance for credit losses	(2,292)	(2,776)	(3,285)	(3,814)	(3,587)	(3,936)
Real estate loans: | Financial Asset, Other than Financial Asset Acquired with Credit Deterioration | Home equity and lines of credit
Loans and Leases Receivable Disclosure [Line Items]
Loans	167,852		163,520
Total loans held-for-investment	167,852		163,520
Allowance for credit losses	(2,562)	(2,299)	(1,705)	(1,278)	(1,398)	(866)
Real estate loans: | Financial Asset, Other than Financial Asset Acquired with Credit Deterioration | Construction and land
Loans and Leases Receivable Disclosure [Line Items]
Loans	32,607		30,967
Total loans held-for-investment	32,607		30,967
Allowance for credit losses	(108)	(122)	(149)	(301)	(286)	(324)
Commercial and industrial and other loans | Financial Asset, Other than Financial Asset Acquired with Credit Deterioration
Loans and Leases Receivable Disclosure [Line Items]
Loans	168,110		157,853
Commercial and industrial loans | Financial Asset, Other than Financial Asset Acquired with Credit Deterioration
Loans and Leases Receivable Disclosure [Line Items]
Loans	165,788		155,268
Total loans held-for-investment	165,788		155,268
Allowance for credit losses	(7,564)	(7,507)	(6,050)	(7,295)	(4,366)	(4,114)
Other loans | Financial Asset, Other than Financial Asset Acquired with Credit Deterioration
Loans and Leases Receivable Disclosure [Line Items]
Loans	2,322		2,585
Total loans held-for-investment	2,322		2,585
Allowance for credit losses	$ (3)	$ (4)	$ (6)	$ (7)	$ (7)	$ (9)