Loans - Schedule of Past Due Financing Receivables (Details) - USD ($) $ in Thousands	Jun. 30, 2020	Jun. 30, 2019
Financing Receivable, Past Due [Line Items]
Past Due	$ 16,133	$ 10,067
Current	2,752,797	2,695,119
Total Loans Receivable	2,768,930	2,705,186
30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	8,245	5,579
90 Days
Financing Receivable, Past Due [Line Items]
Past Due	7,888	4,488
Retail Consumer Loans
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	907,342	1,162,464
Retail Consumer Loans | One-to-four family
Financing Receivable, Past Due [Line Items]
Past Due	4,826	3,004
Current	468,867	657,587
Total Loans Receivable	473,693	660,591
Retail Consumer Loans | One-to-four family | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	1,679	1,615
Retail Consumer Loans | One-to-four family | 90 Days
Financing Receivable, Past Due [Line Items]
Past Due	3,147	1,389
Retail Consumer Loans | HELOCs - originated
Financing Receivable, Past Due [Line Items]
Past Due	752	457
Current	136,695	138,978
Total Loans Receivable	137,447	139,435
Retail Consumer Loans | HELOCs - originated | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	442	226
Retail Consumer Loans | HELOCs - originated | 90 Days
Financing Receivable, Past Due [Line Items]
Past Due	310	231
Retail Consumer Loans | HELOCs - purchased
Financing Receivable, Past Due [Line Items]
Past Due	261	485
Current	71,520	116,487
Total Loans Receivable	71,781	116,972
Retail Consumer Loans | HELOCs - purchased | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	214	0
Retail Consumer Loans | HELOCs - purchased | 90 Days
Financing Receivable, Past Due [Line Items]
Past Due	47	485
Retail Consumer Loans | Construction and land/lots
Financing Receivable, Past Due [Line Items]
Past Due	252	144
Current	81,607	80,458
Total Loans Receivable	81,859	80,602
Retail Consumer Loans | Construction and land/lots | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	0	138
Retail Consumer Loans | Construction and land/lots | 90 Days
Financing Receivable, Past Due [Line Items]
Past Due	252	6
Retail Consumer Loans | Indirect auto finance
Financing Receivable, Past Due [Line Items]
Past Due	1,041	696
Current	131,262	152,752
Total Loans Receivable	132,303	153,448
Retail Consumer Loans | Indirect auto finance | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	756	459
Retail Consumer Loans | Indirect auto finance | 90 Days
Financing Receivable, Past Due [Line Items]
Past Due	285	237
Retail Consumer Loans | Consumer
Financing Receivable, Past Due [Line Items]
Past Due	55	14
Current	10,204	11,402
Total Loans Receivable	10,259	11,416
Retail Consumer Loans | Consumer | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	30	6
Retail Consumer Loans | Consumer | 90 Days
Financing Receivable, Past Due [Line Items]
Past Due	25	8
Commercial Loans
Financing Receivable, Past Due [Line Items]
Total Loans Receivable	1,861,588	1,542,722
Commercial Loans | Commercial real estate
Financing Receivable, Past Due [Line Items]
Past Due	7,420	2,795
Current	1,045,486	924,466
Total Loans Receivable	1,052,906	927,261
Commercial Loans | Commercial real estate | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	4,528	2,279
Commercial Loans | Commercial real estate | 90 Days
Financing Receivable, Past Due [Line Items]
Past Due	2,892	516
Commercial Loans | Construction and development
Financing Receivable, Past Due [Line Items]
Past Due	634	1,133
Current	215,300	209,783
Total Loans Receivable	215,934	210,916
Commercial Loans | Construction and development | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	293	0
Commercial Loans | Construction and development | 90 Days
Financing Receivable, Past Due [Line Items]
Past Due	341	1,133
Commercial Loans | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Past Due	91	306
Current	154,734	160,165
Total Loans Receivable	154,825	160,471
Commercial Loans | Commercial and industrial | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	0	207
Commercial Loans | Commercial and industrial | 90 Days
Financing Receivable, Past Due [Line Items]
Past Due	91	99
Commercial Loans | Equipment finance
Financing Receivable, Past Due [Line Items]
Past Due	801	1,033
Current	228,438	131,025
Total Loans Receivable	229,239	132,058
Commercial Loans | Equipment finance | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	303	649
Commercial Loans | Equipment finance | 90 Days
Financing Receivable, Past Due [Line Items]
Past Due	498	384
Commercial Loans | Municipal leases
Financing Receivable, Past Due [Line Items]
Past Due	0	0
Current	127,987	112,016
Total Loans Receivable	127,987	112,016
Commercial Loans | Municipal leases | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	0	0
Commercial Loans | Municipal leases | 90 Days
Financing Receivable, Past Due [Line Items]
Past Due	0	0
Commercial Loans | paycheck protection program [Domain]
Financing Receivable, Past Due [Line Items]
Past Due	0
Current	80,697
Total Loans Receivable	80,697	$ 0
Commercial Loans | paycheck protection program [Domain] | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	0
Commercial Loans | paycheck protection program [Domain] | 90 Days
Financing Receivable, Past Due [Line Items]
Past Due	$ 0