Loans - Narrative (Details)	12 Months Ended
	Dec. 31, 2023 USD ($) payment PSARate qtr	Dec. 31, 2022 USD ($) PSARate	Dec. 31, 2021 USD ($)
Accounts, Notes, Loans and Financing Receivable [Line Items]
Deferred loan costs	$ 25,300,000	$ 26,600,000
Proceeds from sale of loans held-for-investment	56,561,000	63,564,000	$ 10,032,000
Gain (loss) on sale of loans	7,436,000	6,431,000	35,880,000
Accrued Interest	300,000	200,000	200,000
Interest income	0	0	0
Troubled loan modification	0
Troubled loan modification, subsequent default	$ 0
Past due period for modified loans	90 days
Number of consecutive payments for mortgage loans missed | payment	4
Custodial deposit accounts	$ 28,600,000	$ 28,400,000
Maximum number of consecutive quarterly losses recorded for profitability requirement | qtr	4
Percentage of decline in net worth during the period	30.00%
Maximum percentage of decline in net worth for one consecutive quarter	25.00%
Maximum percentage of decline in net worth for two consecutive quarters	40.00%
Average coupon rate	3.89%	3.60%
Average servicing fee	0.26%	0.26%
Average discount rate	10.25%	10.12%
Average PSA rate | PSARate	142	133
Fixed Rate Residential Mortgage
Accounts, Notes, Loans and Financing Receivable [Line Items]
Proceeds from sale of loans held-for-investment	$ 56,700,000	$ 63,000,000
Gain (loss) on sale of loans	$ (140,000)	$ 550,000
Mortgage | Residential Adjustable Rate Mortgage
Accounts, Notes, Loans and Financing Receivable [Line Items]
Proceeds from sale of loans held-for-investment			9,600,000
Gain (loss) on sale of loans			$ 450,000