Loans and Allowance for Credit Losses - Balance and activity in the Allowance for credit losses (Details) - USD ($)	12 Months Ended
	Jun. 30, 2023	Jun. 30, 2022	Jun. 30, 2021	Jun. 30, 2020
Allowance for credit losses:
Balance, beginning of period			$ 25,139,000
Balance, beginning of period	$ 33,192,000	$ 33,222,000
Initial ACL on PCD loans	17,100,000	1,500,000	1,000,000.0
Allowance for credit losses	14,131,000	(66,000)	(602,000)
Losses charged off	(673,000)	(153,000)	(734,000)
Recoveries	49,000	69,000	86,000
Balance, end of period	47,820,000	33,192,000	33,222,000
Impact of adoption ASU
Allowance for credit losses:
Initial ACL on PCD loans		1,121,000		$ 434,000
Residential Real Estate
Allowance for credit losses:
Balance, beginning of period			4,875,000
Balance, beginning of period	8,908,000	11,192,000
Allowance for credit losses	6,655,000	(2,238,000)	2,973,000
Losses charged off	(19,000)	(72,000)	(180,000)
Recoveries	1,000	3,000	3,000
Balance, end of period	15,641,000	8,908,000	11,192,000
Residential Real Estate | Impact of adoption ASU
Allowance for credit losses:
Initial ACL on PCD loans		96,000
Construction Real Estate
Allowance for credit losses:
Balance, beginning of period			2,010,000
Balance, beginning of period	2,220,000	2,170,000
Allowance for credit losses	432,000	46,000	281,000
Balance, end of period	2,664,000	2,220,000	2,170,000
Construction Real Estate | Impact of adoption ASU
Allowance for credit losses:
Initial ACL on PCD loans		12,000
Commercial Real Estate
Allowance for credit losses:
Balance, beginning of period			12,132,000
Balance, beginning of period	16,838,000	14,535,000
Allowance for credit losses	5,605,000	2,251,000	(1,364,000)
Losses charged off	(245,000)		(90,000)
Recoveries	12,000		1,000
Balance, end of period	22,838,000	16,838,000	14,535,000
Commercial Real Estate | Impact of adoption ASU
Allowance for credit losses:
Initial ACL on PCD loans		628,000
Consumer loans
Allowance for credit losses:
Balance, beginning of period			1,182,000
Balance, beginning of period	710,000	916,000
Allowance for credit losses	334,000	(205,000)	(1,232,000)
Losses charged off	(327,000)	(65,000)	(146,000)
Recoveries	28,000	64,000	47,000
Balance, end of period	909,000	710,000	916,000
Consumer loans | Impact of adoption ASU
Allowance for credit losses:
Initial ACL on PCD loans		164,000
Commercial loans
Allowance for credit losses:
Balance, beginning of period			4,940,000
Balance, beginning of period	4,516,000	4,409,000
Allowance for credit losses	1,105,000	80,000	(1,260,000)
Losses charged off	(82,000)	(16,000)	(318,000)
Recoveries	8,000	2,000	35,000
Balance, end of period	5,768,000	4,516,000	4,409,000
Commercial loans | Impact of adoption ASU
Allowance for credit losses:
Initial ACL on PCD loans		221,000
Accounting Standards Update 2016-13
Allowance for credit losses:
Balance, beginning of period			25,139,000
Accounting Standards Update 2016-13 | Impact of adoption ASU
Allowance for credit losses:
Balance, beginning of period		9,333,000
Balance, end of period			9,333,000
Accounting Standards Update 2016-13 | Residential Real Estate
Allowance for credit losses:
Balance, beginning of period			4,875,000
Accounting Standards Update 2016-13 | Residential Real Estate | Impact of adoption ASU
Allowance for credit losses:
Balance, beginning of period		3,521,000
Balance, end of period			3,521,000
Accounting Standards Update 2016-13 | Construction Real Estate
Allowance for credit losses:
Balance, beginning of period			2,010,000
Accounting Standards Update 2016-13 | Construction Real Estate | Impact of adoption ASU
Allowance for credit losses:
Balance, beginning of period		(121,000)
Balance, end of period			(121,000)
Accounting Standards Update 2016-13 | Commercial Real Estate
Allowance for credit losses:
Balance, beginning of period			12,132,000
Accounting Standards Update 2016-13 | Commercial Real Estate | Impact of adoption ASU
Allowance for credit losses:
Balance, beginning of period		3,856,000
Balance, end of period			3,856,000
Accounting Standards Update 2016-13 | Consumer loans
Allowance for credit losses:
Balance, beginning of period			1,182,000
Accounting Standards Update 2016-13 | Consumer loans | Impact of adoption ASU
Allowance for credit losses:
Balance, beginning of period		1,065,000
Balance, end of period			1,065,000
Accounting Standards Update 2016-13 | Commercial loans
Allowance for credit losses:
Balance, beginning of period			4,940,000
Accounting Standards Update 2016-13 | Commercial loans | Impact of adoption ASU
Allowance for credit losses:
Balance, beginning of period		1,012,000
Balance, end of period			$ 1,012,000
Accounting Standards Update 2022-02 | Impact of adoption ASU
Allowance for credit losses:
Balance, beginning of period	120,000
Balance, end of period		120,000
Accounting Standards Update 2022-02 | Residential Real Estate | Impact of adoption ASU
Allowance for credit losses:
Balance, beginning of period	23,000
Balance, end of period		23,000
Accounting Standards Update 2022-02 | Construction Real Estate | Impact of adoption ASU
Allowance for credit losses:
Balance, beginning of period	4,000
Balance, end of period		4,000
Accounting Standards Update 2022-02 | Commercial Real Estate | Impact of adoption ASU
Allowance for credit losses:
Balance, beginning of period	52,000
Balance, end of period		52,000
Accounting Standards Update 2022-02 | Commercial loans | Impact of adoption ASU
Allowance for credit losses:
Balance, beginning of period	$ 41,000
Balance, end of period		$ 41,000