Loans - Schedule of Past Due Financing Receivables (Details) - USD ($) $ in Thousands	Mar. 31, 2021	Jun. 30, 2020
Financing Receivable, Past Due [Line Items]
Past Due	$ 6,434	$ 16,133
Current	2,683,719	2,752,797
Total Loans	2,690,153	2,768,930
30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	1,574	8,245
90 Days+
Financing Receivable, Past Due [Line Items]
Past Due	4,860	7,888
Commercial Loans
Financing Receivable, Past Due [Line Items]
Total Loans	1,885,758	1,861,588
Commercial Loans | PPP
Financing Receivable, Past Due [Line Items]
Past Due	0	0
Current	73,090	80,697
Total Loans	73,090	80,697
Commercial Loans | 30-89 Days | PPP
Financing Receivable, Past Due [Line Items]
Past Due	0	0
Commercial Loans | 90 Days+ | PPP
Financing Receivable, Past Due [Line Items]
Past Due	0	0
Commercial Loans | Commercial real estate
Financing Receivable, Past Due [Line Items]
Past Due	1,863	7,420
Current	1,086,315	1,045,486
Total Loans	1,088,178	1,052,906
Commercial Loans | Commercial real estate | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	0	4,528
Commercial Loans | Commercial real estate | 90 Days+
Financing Receivable, Past Due [Line Items]
Past Due	1,863	2,892
Commercial Loans | Construction and development
Financing Receivable, Past Due [Line Items]
Past Due	37	634
Current	162,783	215,300
Total Loans	162,820	215,934
Commercial Loans | Construction and development | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	0	293
Commercial Loans | Construction and development | 90 Days+
Financing Receivable, Past Due [Line Items]
Past Due	37	341
Commercial Loans | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Past Due	34	91
Current	140,545	154,734
Total Loans	140,579	154,825
Commercial Loans | Commercial and industrial | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	12	0
Commercial Loans | Commercial and industrial | 90 Days+
Financing Receivable, Past Due [Line Items]
Past Due	22	91
Commercial Loans | Equipment finance
Financing Receivable, Past Due [Line Items]
Past Due	328	801
Current	291,622	228,438
Total Loans	291,950	229,239
Commercial Loans | Equipment finance | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	0	303
Commercial Loans | Equipment finance | 90 Days+
Financing Receivable, Past Due [Line Items]
Past Due	328	498
Commercial Loans | Municipal finance
Financing Receivable, Past Due [Line Items]
Past Due	0	0
Current	129,141	127,987
Total Loans	129,141	127,987
Commercial Loans | Municipal finance | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	0	0
Commercial Loans | Municipal finance | 90 Days+
Financing Receivable, Past Due [Line Items]
Past Due	0	0
Retail Consumer Loans
Financing Receivable, Past Due [Line Items]
Total Loans	804,395	907,342
Retail Consumer Loans | One-to-four family
Financing Receivable, Past Due [Line Items]
Past Due	2,870	4,826
Current	427,131	468,867
Total Loans	430,001	473,693
Retail Consumer Loans | One-to-four family | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	824	1,679
Retail Consumer Loans | One-to-four family | 90 Days+
Financing Receivable, Past Due [Line Items]
Past Due	2,046	3,147
Retail Consumer Loans | HELOCs - originated
Financing Receivable, Past Due [Line Items]
Past Due	110	752
Current	131,757	136,695
Total Loans	131,867	137,447
Retail Consumer Loans | HELOCs - originated | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	32	442
Retail Consumer Loans | HELOCs - originated | 90 Days+
Financing Receivable, Past Due [Line Items]
Past Due	78	310
Retail Consumer Loans | HELOCs - purchased
Financing Receivable, Past Due [Line Items]
Past Due	147	261
Current	45,939	71,520
Total Loans	46,086	71,781
Retail Consumer Loans | HELOCs - purchased | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	50	214
Retail Consumer Loans | HELOCs - purchased | 90 Days+
Financing Receivable, Past Due [Line Items]
Past Due	97	47
Retail Consumer Loans | Construction and land/lots
Financing Receivable, Past Due [Line Items]
Past Due	0	252
Current	68,118	81,607
Total Loans	68,118	81,859
Retail Consumer Loans | Construction and land/lots | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	0	0
Retail Consumer Loans | Construction and land/lots | 90 Days+
Financing Receivable, Past Due [Line Items]
Past Due	0	252
Retail Consumer Loans | Indirect auto finance
Financing Receivable, Past Due [Line Items]
Past Due	747	1,041
Current	118,909	131,262
Total Loans	119,656	132,303
Retail Consumer Loans | Indirect auto finance | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	385	756
Retail Consumer Loans | Indirect auto finance | 90 Days+
Financing Receivable, Past Due [Line Items]
Past Due	362	285
Retail Consumer Loans | Consumer
Financing Receivable, Past Due [Line Items]
Past Due	298	55
Current	8,369	10,204
Total Loans	8,667	10,259
Retail Consumer Loans | Consumer | 30-89 Days
Financing Receivable, Past Due [Line Items]
Past Due	271	30
Retail Consumer Loans | Consumer | 90 Days+
Financing Receivable, Past Due [Line Items]
Past Due	$ 27	$ 25