Loans - Schedule of Past Due Financing Receivables (Details) - USD ($) $ in Thousands	Jun. 30, 2015	Jun. 30, 2014
Total loans
Financing Receivable, Recorded Investment, Past Due [Line Items]
30-89 Days Past Due		$ 7,525
90 Days and Over Past Due		20,379
Total Past Due		27,904
Current		1,470,394
Total loans receivable		1,498,298
Retail consumer loans: | One-to-four family
Financing Receivable, Recorded Investment, Past Due [Line Items]
30-89 Days Past Due	$ 5,548	4,929
90 Days and Over Past Due	8,261	8,208
Total Past Due	13,809	13,137
Current	636,941	647,493
Total loans receivable	650,750	660,630
Retail consumer loans: | HELOCs - originated
Financing Receivable, Recorded Investment, Past Due [Line Items]
30-89 Days Past Due	695	400
90 Days and Over Past Due	808	939
Total Past Due	1,503	1,339
Current	159,701	147,040
Total loans receivable	161,204	148,379
Retail consumer loans: | HELOCs - purchased
Financing Receivable, Recorded Investment, Past Due [Line Items]
30-89 Days Past Due	0	0
90 Days and Over Past Due	0	0
Total Past Due	0	0
Current	72,010	0
Total loans receivable	72,010	0
Retail consumer loans: | Construction and land/lots
Financing Receivable, Recorded Investment, Past Due [Line Items]
30-89 Days Past Due	102	508
90 Days and Over Past Due	307	122
Total Past Due	409	630
Current	45,522	58,619
Total loans receivable	45,931	59,249
Retail consumer loans: | Indirect auto finance
Financing Receivable, Recorded Investment, Past Due [Line Items]
30-89 Days Past Due	0	0
90 Days and Over Past Due	0	0
Total Past Due	0	0
Current	52,494	8,833
Total loans receivable	52,494	8,833
Retail consumer loans: | Consumer
Financing Receivable, Recorded Investment, Past Due [Line Items]
30-89 Days Past Due	23	34
90 Days and Over Past Due	2	16
Total Past Due	25	50
Current	3,683	6,281
Total loans receivable	3,708	6,331
Commercial loans: | Commercial real estate
Financing Receivable, Recorded Investment, Past Due [Line Items]
30-89 Days Past Due	2,758	306
90 Days and Over Past Due	4,636	6,729
Total Past Due	7,394	7,035
Current	434,226	370,734
Total loans receivable	441,620	377,769
Commercial loans: | Construction and development
Financing Receivable, Recorded Investment, Past Due [Line Items]
30-89 Days Past Due	166	1,165
90 Days and Over Past Due	2,992	3,789
Total Past Due	3,158	4,954
Current	61,415	51,503
Total loans receivable	64,573	56,457
Commercial loans: | Commercial and industrial
Financing Receivable, Recorded Investment, Past Due [Line Items]
30-89 Days Past Due	439	183
90 Days and Over Past Due	2,898	576
Total Past Due	3,337	759
Current	81,483	73,676
Total loans receivable	84,820	74,435
Commercial loans: | Municipal leases
Financing Receivable, Recorded Investment, Past Due [Line Items]
30-89 Days Past Due	202	0
90 Days and Over Past Due	0	0
Total Past Due	202	0
Current	108,372	106,215
Total loans receivable	108,574	$ 106,215
Commercial loans: | Total loans
Financing Receivable, Recorded Investment, Past Due [Line Items]
30-89 Days Past Due	9,933
90 Days and Over Past Due	19,904
Total Past Due	29,837
Current	1,655,847
Total loans receivable	$ 1,685,684