Loans - Schedule of PCI Loans Credit Quality Indicators (Details) - USD ($) $ in Thousands	Jun. 30, 2018	Jun. 30, 2017
Financing Receivable, Recorded Investment
Total loans	$ 2,505,712	$ 2,320,874
Pass
Financing Receivable, Recorded Investment
Total loans	2,470,436	2,274,470
Special Mention
Financing Receivable, Recorded Investment
Total loans	11,809	14,995
Substandard
Financing Receivable, Recorded Investment
Total loans	22,484	30,031
Doubtful
Financing Receivable, Recorded Investment
Total loans	952	1,350
Loss
Financing Receivable, Recorded Investment
Total loans	31	28
Retail Consumer Loans | One-to-four family
Financing Receivable, Recorded Investment
Total loans	657,472	676,020
Retail Consumer Loans | HELOCs - originated
Financing Receivable, Recorded Investment
Total loans	137,340	156,780
Retail Consumer Loans | Construction and land/lots
Financing Receivable, Recorded Investment
Total loans	65,157	49,608
Retail Consumer Loans | Consumer
Financing Receivable, Recorded Investment
Total loans	12,379	7,882
Retail Consumer Loans | Pass | One-to-four family
Financing Receivable, Recorded Investment
Total loans	643,077	655,424
Retail Consumer Loans | Pass | HELOCs - originated
Financing Receivable, Recorded Investment
Total loans	135,336	153,676
Retail Consumer Loans | Pass | Construction and land/lots
Financing Receivable, Recorded Investment
Total loans	64,823	48,728
Retail Consumer Loans | Pass | Consumer
Financing Receivable, Recorded Investment
Total loans	11,723	7,828
Retail Consumer Loans | Special Mention | One-to-four family
Financing Receivable, Recorded Investment
Total loans	3,576	4,715
Retail Consumer Loans | Special Mention | HELOCs - originated
Financing Receivable, Recorded Investment
Total loans	113	809
Retail Consumer Loans | Special Mention | Construction and land/lots
Financing Receivable, Recorded Investment
Total loans	23	479
Retail Consumer Loans | Special Mention | Consumer
Financing Receivable, Recorded Investment
Total loans	85	12
Retail Consumer Loans | Substandard | One-to-four family
Financing Receivable, Recorded Investment
Total loans	10,059	14,769
Retail Consumer Loans | Substandard | HELOCs - originated
Financing Receivable, Recorded Investment
Total loans	1,735	2,100
Retail Consumer Loans | Substandard | Construction and land/lots
Financing Receivable, Recorded Investment
Total loans	257	341
Retail Consumer Loans | Substandard | Consumer
Financing Receivable, Recorded Investment
Total loans	558	34
Retail Consumer Loans | Doubtful | One-to-four family
Financing Receivable, Recorded Investment
Total loans	746	1,101
Retail Consumer Loans | Doubtful | HELOCs - originated
Financing Receivable, Recorded Investment
Total loans	150	188
Retail Consumer Loans | Doubtful | Construction and land/lots
Financing Receivable, Recorded Investment
Total loans	54	60
Retail Consumer Loans | Doubtful | Consumer
Financing Receivable, Recorded Investment
Total loans	2	0
Retail Consumer Loans | Loss | One-to-four family
Financing Receivable, Recorded Investment
Total loans	14	11
Retail Consumer Loans | Loss | HELOCs - originated
Financing Receivable, Recorded Investment
Total loans	6	7
Retail Consumer Loans | Loss | Construction and land/lots
Financing Receivable, Recorded Investment
Total loans	0	0
Retail Consumer Loans | Loss | Consumer
Financing Receivable, Recorded Investment
Total loans	11	8
Commercial Loans | Commercial real estate
Financing Receivable, Recorded Investment
Total loans	848,076	713,025
Commercial Loans | Construction and development
Financing Receivable, Recorded Investment
Total loans	189,875	195,337
Commercial Loans | Commercial and industrial
Financing Receivable, Recorded Investment
Total loans	146,870	117,761
Commercial Loans | Pass | Commercial real estate
Financing Receivable, Recorded Investment
Total loans	835,485	700,060
Commercial Loans | Pass | Construction and development
Financing Receivable, Recorded Investment
Total loans	187,187	192,025
Commercial Loans | Pass | Commercial and industrial
Financing Receivable, Recorded Investment
Total loans	145,177	113,923
Commercial Loans | Special Mention | Commercial real estate
Financing Receivable, Recorded Investment
Total loans	5,804	5,847
Commercial Loans | Special Mention | Construction and development
Financing Receivable, Recorded Investment
Total loans	621	992
Commercial Loans | Special Mention | Commercial and industrial
Financing Receivable, Recorded Investment
Total loans	1,279	883
Commercial Loans | Substandard | Commercial real estate
Financing Receivable, Recorded Investment
Total loans	6,787	7,118
Commercial Loans | Substandard | Construction and development
Financing Receivable, Recorded Investment
Total loans	2,067	2,320
Commercial Loans | Substandard | Commercial and industrial
Financing Receivable, Recorded Investment
Total loans	414	2,954
Commercial Loans | Doubtful | Commercial real estate
Financing Receivable, Recorded Investment
Total loans	0	0
Commercial Loans | Doubtful | Construction and development
Financing Receivable, Recorded Investment
Total loans	0	0
Commercial Loans | Doubtful | Commercial and industrial
Financing Receivable, Recorded Investment
Total loans	0	0
Commercial Loans | Loss | Commercial real estate
Financing Receivable, Recorded Investment
Total loans	0	0
Commercial Loans | Loss | Construction and development
Financing Receivable, Recorded Investment
Total loans	0	0
Commercial Loans | Loss | Commercial and industrial
Financing Receivable, Recorded Investment
Total loans	0	1
Purchased Credit Impaired (PCI) Loans
Financing Receivable, Recorded Investment
Total loans	20,904	31,541
Purchased Credit Impaired (PCI) Loans | Pass
Financing Receivable, Recorded Investment
Total loans	12,447	15,571
Purchased Credit Impaired (PCI) Loans | Special Mention
Financing Receivable, Recorded Investment
Total loans	2,550	3,486
Purchased Credit Impaired (PCI) Loans | Substandard
Financing Receivable, Recorded Investment
Total loans	5,907	12,274
Purchased Credit Impaired (PCI) Loans | Doubtful
Financing Receivable, Recorded Investment
Total loans	0	210
Purchased Credit Impaired (PCI) Loans | Loss
Financing Receivable, Recorded Investment
Total loans	0	0
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | One-to-four family
Financing Receivable, Recorded Investment
Total loans	6,817	8,069
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | HELOCs - originated
Financing Receivable, Recorded Investment
Total loans	224	288
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Construction and land/lots
Financing Receivable, Recorded Investment
Total loans	444	528
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Consumer
Financing Receivable, Recorded Investment
Total loans	0	18
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Pass | One-to-four family
Financing Receivable, Recorded Investment
Total loans	4,620	3,115
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Pass | HELOCs - originated
Financing Receivable, Recorded Investment
Total loans	224	258
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Pass | Construction and land/lots
Financing Receivable, Recorded Investment
Total loans	444	487
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Pass | Consumer
Financing Receivable, Recorded Investment
Total loans		4
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Special Mention | One-to-four family
Financing Receivable, Recorded Investment
Total loans	388	1,129
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Special Mention | HELOCs - originated
Financing Receivable, Recorded Investment
Total loans	0	0
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Special Mention | Construction and land/lots
Financing Receivable, Recorded Investment
Total loans	0	0
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Special Mention | Consumer
Financing Receivable, Recorded Investment
Total loans		14
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Substandard | One-to-four family
Financing Receivable, Recorded Investment
Total loans	1,809	3,615
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Substandard | HELOCs - originated
Financing Receivable, Recorded Investment
Total loans	0	30
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Substandard | Construction and land/lots
Financing Receivable, Recorded Investment
Total loans	0	41
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Substandard | Consumer
Financing Receivable, Recorded Investment
Total loans		0
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Doubtful | One-to-four family
Financing Receivable, Recorded Investment
Total loans	0	210
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Doubtful | HELOCs - originated
Financing Receivable, Recorded Investment
Total loans	0	0
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Doubtful | Construction and land/lots
Financing Receivable, Recorded Investment
Total loans	0	0
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Doubtful | Consumer
Financing Receivable, Recorded Investment
Total loans		0
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Loss | One-to-four family
Financing Receivable, Recorded Investment
Total loans	0	0
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Loss | HELOCs - originated
Financing Receivable, Recorded Investment
Total loans	0	0
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Loss | Construction and land/lots
Financing Receivable, Recorded Investment
Total loans	0	0
Purchased Credit Impaired (PCI) Loans | Retail Consumer Loans | Loss | Consumer
Financing Receivable, Recorded Investment
Total loans		0
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Commercial real estate
Financing Receivable, Recorded Investment
Total loans	9,239	17,383
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Construction and development
Financing Receivable, Recorded Investment
Total loans	2,227	2,629
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Commercial and industrial
Financing Receivable, Recorded Investment
Total loans	1,953	2,626
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Pass | Commercial real estate
Financing Receivable, Recorded Investment
Total loans	4,718	8,909
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Pass | Construction and development
Financing Receivable, Recorded Investment
Total loans	547	338
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Pass | Commercial and industrial
Financing Receivable, Recorded Investment
Total loans	1,894	2,460
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Special Mention | Commercial real estate
Financing Receivable, Recorded Investment
Total loans	2,162	2,299
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Special Mention | Construction and development
Financing Receivable, Recorded Investment
Total loans	0	0
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Special Mention | Commercial and industrial
Financing Receivable, Recorded Investment
Total loans	0	44
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Substandard | Commercial real estate
Financing Receivable, Recorded Investment
Total loans	2,359	6,175
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Substandard | Construction and development
Financing Receivable, Recorded Investment
Total loans	1,680	2,291
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Substandard | Commercial and industrial
Financing Receivable, Recorded Investment
Total loans	59	122
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Doubtful | Commercial real estate
Financing Receivable, Recorded Investment
Total loans	0	0
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Doubtful | Construction and development
Financing Receivable, Recorded Investment
Total loans	0	0
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Doubtful | Commercial and industrial
Financing Receivable, Recorded Investment
Total loans	0	0
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Loss | Commercial real estate
Financing Receivable, Recorded Investment
Total loans	0	0
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Loss | Construction and development
Financing Receivable, Recorded Investment
Total loans	0	0
Purchased Credit Impaired (PCI) Loans | Commercial Loans | Loss | Commercial and industrial
Financing Receivable, Recorded Investment
Total loans	$ 0	$ 0