Loans and Allowance for Loan Losses - Financing Receivable Credit Quality Indicators (Details) - USD ($) $ in Thousands	Jun. 30, 2020	Jun. 30, 2019
Consumer loans | Pass
Financing receivable credit quality indicators	$ 80,517	$ 97,012
Consumer loans | Watch
Financing receivable credit quality indicators	45	170
Consumer loans | Special Mention
Financing receivable credit quality indicators	25	26
Consumer loans | Substandard
Financing receivable credit quality indicators	180	291
Consumer loans | Doubtful
Financing receivable credit quality indicators		35
Consumer loans | Total by Credit Quality Indicator
Financing receivable credit quality indicators	80,767	97,534
Commercial loans | Pass
Financing receivable credit quality indicators	457,385	341,069
Commercial loans | Watch
Financing receivable credit quality indicators	4,708	7,802
Commercial loans | Substandard
Financing receivable credit quality indicators	6,355	7,003
Commercial loans | Total by Credit Quality Indicator
Financing receivable credit quality indicators	468,448	355,874
Construction Real Estate | Pass
Financing receivable credit quality indicators	103,105	80,134
Construction Real Estate | Watch
Financing receivable credit quality indicators	4,367
Construction Real Estate | Total by Credit Quality Indicator
Financing receivable credit quality indicators	107,472	80,134
Residential Real Estate | Pass
Financing receivable credit quality indicators	620,004	482,869
Residential Real Estate | Watch
Financing receivable credit quality indicators	1,900	1,236
Residential Real Estate | Special Mention
Financing receivable credit quality indicators		103
Residential Real Estate | Substandard
Financing receivable credit quality indicators	5,453	7,784
Residential Real Estate | Total by Credit Quality Indicator
Financing receivable credit quality indicators	627,357	491,992
Commercial | Pass
Financing receivable credit quality indicators	829,276	802,479
Commercial | Watch
Financing receivable credit quality indicators	45,262	21,693
Commercial | Special Mention
Financing receivable credit quality indicators	403	3,463
Commercial | Substandard
Financing receivable credit quality indicators	11,590	13,142
Commercial | Doubtful
Financing receivable credit quality indicators	888
Commercial | Total by Credit Quality Indicator
Financing receivable credit quality indicators	$ 887,419	$ 840,777