Loans and Allowance for Credit Losses - Balance in the ACL and the recorded investment in loans (Details) - USD ($) $ in Thousands	12 Months Ended
	Jun. 30, 2021	Jun. 30, 2020	Jun. 30, 2019
Allowance for credit losses:
Balance, beginning of period	$ 25,139	$ 19,903	$ 18,214
Provision (benefit) charged to expense	(602)	6,002	2,032
Losses charged off	(734)	(853)	(389)
Recoveries	86	87	46
Balance, end of period	33,222	25,139	19,903
Ending Balance: individually evaluated for impairment		0
Ending Balance: collectively evaluated for impairment		25,139
Ending Balance: loans acquired with deteriorated credit quality		0
Loans:
Ending Balance: individually evaluated for impairment		0
Ending Balance: collectively evaluated for impairment		2,149,664
Ending Balance: loans acquired with deteriorated credit quality		21,799
Residential Real Estate
Allowance for credit losses:
Balance, beginning of period	4,875	3,706	3,226
Provision (benefit) charged to expense	2,973	1,529	487
Losses charged off	(180)	(379)	(30)
Recoveries	3	19	23
Balance, end of period	11,192	4,875	3,706
Ending Balance: individually evaluated for impairment		0
Ending Balance: collectively evaluated for impairment		4,875
Ending Balance: loans acquired with deteriorated credit quality		0
Loans:
Ending Balance: individually evaluated for impairment		0
Ending Balance: collectively evaluated for impairment		626,085
Ending Balance: loans acquired with deteriorated credit quality		1,272
Construction Real Estate
Allowance for credit losses:
Balance, beginning of period	2,010	1,365	1,097
Provision (benefit) charged to expense	281	645	268
Balance, end of period	2,170	2,010	1,365
Ending Balance: individually evaluated for impairment		0
Ending Balance: collectively evaluated for impairment		2,010
Ending Balance: loans acquired with deteriorated credit quality		0
Loans:
Ending Balance: individually evaluated for impairment		0
Ending Balance: collectively evaluated for impairment		106,194
Ending Balance: loans acquired with deteriorated credit quality		1,278
Commercial Real Estate
Allowance for credit losses:
Balance, beginning of period	12,132	9,399	8,793
Provision (benefit) charged to expense	(1,364)	2,730	765
Losses charged off	(90)	(12)	(164)
Recoveries	1	15	5
Balance, end of period	14,535	12,132	9,399
Ending Balance: individually evaluated for impairment		0
Ending Balance: collectively evaluated for impairment		12,132
Ending Balance: loans acquired with deteriorated credit quality		0
Loans:
Ending Balance: individually evaluated for impairment		0
Ending Balance: collectively evaluated for impairment		872,716
Ending Balance: loans acquired with deteriorated credit quality		14,703
Consumer loans
Allowance for credit losses:
Balance, beginning of period	1,182	1,046	902
Provision (benefit) charged to expense	(1,232)	300	231
Losses charged off	(146)	(189)	(103)
Recoveries	47	25	16
Balance, end of period	916	1,182	1,046
Ending Balance: individually evaluated for impairment		0
Ending Balance: collectively evaluated for impairment		1,182
Ending Balance: loans acquired with deteriorated credit quality		0
Loans:
Ending Balance: individually evaluated for impairment		0
Ending Balance: collectively evaluated for impairment		80,767
Commercial loans
Allowance for credit losses:
Balance, beginning of period	4,940	4,387	4,196
Provision (benefit) charged to expense	(1,260)	798	281
Losses charged off	(318)	(273)	(92)
Recoveries	35	28	2
Balance, end of period	4,409	4,940	$ 4,387
Ending Balance: individually evaluated for impairment		0
Ending Balance: collectively evaluated for impairment		4,940
Ending Balance: loans acquired with deteriorated credit quality		0
Loans:
Ending Balance: individually evaluated for impairment		0
Ending Balance: collectively evaluated for impairment		463,902
Ending Balance: loans acquired with deteriorated credit quality		4,546
Impact of ASU 2016-13 adoption
Allowance for credit losses:
Balance, beginning of period	9,333
Balance, end of period		9,333
Impact of ASU 2016-13 adoption | Residential Real Estate
Allowance for credit losses:
Balance, beginning of period	3,521
Balance, end of period		3,521
Impact of ASU 2016-13 adoption | Construction Real Estate
Allowance for credit losses:
Balance, beginning of period	(121)
Balance, end of period		(121)
Impact of ASU 2016-13 adoption | Commercial Real Estate
Allowance for credit losses:
Balance, beginning of period	3,856
Balance, end of period		3,856
Impact of ASU 2016-13 adoption | Consumer loans
Allowance for credit losses:
Balance, beginning of period	1,065
Balance, end of period		1,065
Impact of ASU 2016-13 adoption | Commercial loans
Allowance for credit losses:
Balance, beginning of period	$ 1,012
Balance, end of period		$ 1,012