Unaudited Interim Financial Information - Schedule of Quarterly Financial Information (Details) - USD ($) $ / shares in Units, $ in Thousands	3 Months Ended								12 Months Ended
	Jun. 30, 2020	Mar. 31, 2020	Dec. 31, 2019	Sep. 30, 2019	Jun. 30, 2019	Mar. 31, 2019	Dec. 31, 2018	Sep. 30, 2018	Jun. 30, 2020	Jun. 30, 2019	Jun. 30, 2018
Quarterly Financial Information Disclosure [Abstract]
Interest and dividend income	$ 31,074	$ 33,037	$ 35,896	$ 36,247	$ 35,820	$ 34,714	$ 34,400	$ 32,280	$ 136,254	$ 137,214	$ 117,402
Interest expense	6,386	7,728	8,862	9,174	8,931	8,145	7,299	6,008	32,150	30,383	16,072
Net Interest Income	24,688	25,309	27,034	27,073	26,889	26,569	27,101	26,272	104,104	106,831	101,330
Provision for loan losses	2,700	5,400	400	0	200	5,500	0	0
Net interest income after provision for loan losses	21,988	19,909	26,634	27,073	26,689	21,069	27,101	26,272
Noninterest income	7,223	6,375	9,074	7,660	6,846	5,396	5,085	5,613	30,332	22,940	18,972
Noninterest expense	24,652	24,903	24,041	23,533	23,415	22,978	21,858	21,883	97,129	90,134	85,331
Net income before income taxes	4,559	1,381	11,667	11,200	10,120	3,487	10,328	10,002
Income tax expense	964	188	2,476	2,396	2,107	185	2,287	2,212	6,024	6,791	26,736
Net Income	$ 3,595	$ 1,193	$ 9,191	$ 8,804	$ 8,013	$ 3,302	$ 8,041	$ 7,790	$ 22,783	$ 27,146	$ 8,235
Basic net income per common share (in dollars per share)	$ 0.22	$ 0.07	$ 0.54	$ 0.51	$ 0.45	$ 0.19	$ 0.45	$ 0.43
Diluted net income per common share (in dollars per share)	$ 0.22	$ 0.07	$ 0.52	$ 0.49	$ 0.44	$ 0.18	$ 0.43	$ 0.41