Loans and Allowance for Credit Losses - CARES Act (Details) - USD ($)	Jun. 30, 2021	Jun. 30, 2020
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	$ 2,237,091,000	$ 2,171,463,000
Current
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	2,233,273,000	2,165,070,000
Current | CARES Act
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	23,900,000	380,100,000
30-59 Days Past Due
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	1,238,000	1,686,000
60 to 89 Days Past Due
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	1,369,000	1,977,000
Total Past Due
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	3,818,000	6,393,000
Total Past Due | CARES Act
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	0
Residential Real Estate
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	721,216,000	627,357,000
Residential Real Estate | Current
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	719,927,000	625,553,000
Residential Real Estate | 30-59 Days Past Due
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	312,000	772,000
Residential Real Estate | 30-59 Days Past Due | CARES Act
Financing Receivable, Past Due [Line Items]
Total Loans Receivable		1,000
Residential Real Estate | 60 to 89 Days Past Due
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	364,000	378,000
Residential Real Estate | Total Past Due
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	1,289,000	1,804,000
Consumer loans
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	77,674,000	80,767,000
Consumer loans | Current
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	77,329,000	80,341,000
Consumer loans | 30-59 Days Past Due
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	195,000	180,000
Consumer loans | 30-59 Days Past Due | CARES Act
Financing Receivable, Past Due [Line Items]
Total Loans Receivable		29,000
Consumer loans | 60 to 89 Days Past Due
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	66,000	53,000
Consumer loans | Total Past Due
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	345,000	426,000
Commercial loans
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	414,124,000	468,448,000
Commercial loans | Current
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	412,707,000	466,326,000
Commercial loans | 30-59 Days Past Due
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	368,000	93,000
Commercial loans | 60 to 89 Days Past Due
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	939,000	1,219,000
Commercial loans | 60 to 89 Days Past Due | CARES Act
Financing Receivable, Past Due [Line Items]
Total Loans Receivable		66,000
Commercial loans | Total Past Due
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	$ 1,417,000	$ 2,122,000