Loans and Allowance for Loan Losses - Schedule of Loan Portfolio Aging Analysis (Details) $ in Thousands	12 Months Ended
	Jun. 30, 2019 USD ($)	Jun. 30, 2020 USD ($)
Number of Purchased Credit Impaired Loans	1
30-59 Days Past Due | Loans Receivable
Financing receivable recorded investment	$ 1,075	$ 1,686
30-59 Days Past Due | Consumer loans
Financing receivable recorded investment	128	180
30-59 Days Past Due | Commercial loans
Financing receivable recorded investment	424	93
30-59 Days Past Due | Residential Real Estate
Financing receivable recorded investment	227	772
30-59 Days Past Due | Commercial
Financing receivable recorded investment	296	641
60 to 89 Days Past Due | Loans Receivable
Financing receivable recorded investment	1,126	1,977
60 to 89 Days Past Due | Consumer loans
Financing receivable recorded investment	46	53
60 to 89 Days Past Due | Commercial loans
Financing receivable recorded investment	25	1,219
60 to 89 Days Past Due | Residential Real Estate
Financing receivable recorded investment	1,054	378
60 to 89 Days Past Due | Commercial
Financing receivable recorded investment	1	327
Greater than 90 Days Past Due
Purchased Credit Impaired Loans	3,100	0
Greater than 90 Days Past Due | Loans Receivable
Financing receivable recorded investment	9,409	2,730
Greater than 90 Days Past Due | Consumer loans
Financing receivable recorded investment	176	193
Greater than 90 Days Past Due | Commercial loans
Financing receivable recorded investment	1,902	810
Greater than 90 Days Past Due | Residential Real Estate
Financing receivable recorded investment	1,714	654
Greater than 90 Days Past Due | Commercial
Financing receivable recorded investment	5,617	1,073
Total Past Due | Loans Receivable
Financing receivable recorded investment	11,610	6,393
Total Past Due | Consumer loans
Financing receivable recorded investment	350	426
Total Past Due | Commercial loans
Financing receivable recorded investment	2,351	2,122
Total Past Due | Residential Real Estate
Financing receivable recorded investment	2,995	1,804
Total Past Due | Commercial
Financing receivable recorded investment	5,914	2,041
Current | Loans Receivable
Financing receivable recorded investment	1,854,701	2,165,070
Current | Consumer loans
Financing receivable recorded investment	97,184	80,341
Current | Commercial loans
Financing receivable recorded investment	353,523	466,326
Current | Construction Real Estate
Financing receivable recorded investment	80,134	107,472
Current | Residential Real Estate
Financing receivable recorded investment	488,997	625,553
Current | Commercial
Financing receivable recorded investment	834,863	885,378
Total Loans Receivable | Loans Receivable
Financing receivable recorded investment	1,866,311	2,171,463
Total Loans Receivable | Consumer loans
Financing receivable recorded investment	97,534	80,767
Total Loans Receivable | Commercial loans
Financing receivable recorded investment	355,874	468,448
Total Loans Receivable | Construction Real Estate
Financing receivable recorded investment	80,134	107,472
Total Loans Receivable | Residential Real Estate
Financing receivable recorded investment	491,992	627,357
Total Loans Receivable | Commercial
Financing receivable recorded investment	$ 840,777	$ 887,419