Loans and Allowance for Credit Losses on Loans - Schedule of Ending Balances of Loans and the Related Allowance by Segment and Class (Details) - USD ($) $ in Thousands	Mar. 31, 2022	Dec. 31, 2021	Jun. 30, 2021	Mar. 31, 2021	Dec. 31, 2020	Jun. 30, 2020
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Loans individually evaluated for impairment	$ 1,311		$ 467
Allowance for credit losses, Loans collectively evaluated	29,723		35,001
Allowance for credit losses, Total	31,034	$ 30,933	35,468	$ 36,059	$ 39,844	$ 28,072
Total loans receivable, Loans individually evaluated for impairment	4,927		8,821
Total loans receivable, Loans collectively evaluated	2,694,611		2,724,446
Total loans receivable	2,699,538		2,733,267
Commercial Loans
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Total	22,322	22,968	24,746	23,398	24,899	21,116
Total loans receivable	2,032,839		1,968,035
Commercial Loans | PPP
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Loans individually evaluated for impairment	0		0
Allowance for credit losses, Loans collectively evaluated	0		0
Allowance for credit losses, Total	0		0
Total loans receivable, Loans individually evaluated for impairment	0		0
Total loans receivable, Loans collectively evaluated	2,756		46,650
Total loans receivable	2,756		46,650
Commercial Loans | Commercial real estate
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Loans individually evaluated for impairment	0		456
Allowance for credit losses, Loans collectively evaluated	9,157		12,826
Allowance for credit losses, Total	9,157		13,282
Total loans receivable, Loans individually evaluated for impairment	284		5,729
Total loans receivable, Loans collectively evaluated	1,101,900		1,136,547
Total loans receivable	1,102,184		1,142,276
Commercial Loans | Construction and development
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Loans individually evaluated for impairment	0		0
Allowance for credit losses, Loans collectively evaluated	3,680		1,801
Allowance for credit losses, Total	3,680		1,801
Total loans receivable, Loans individually evaluated for impairment	0		80
Total loans receivable, Loans collectively evaluated	251,668		179,347
Total loans receivable	251,668		179,427
Commercial Loans | Commercial and industrial
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Loans individually evaluated for impairment	1,311		9
Allowance for credit losses, Loans collectively evaluated	1,593		2,583
Allowance for credit losses, Total	2,904		2,592
Total loans receivable, Loans individually evaluated for impairment	2,150		760
Total loans receivable, Loans collectively evaluated	165,192		140,581
Total loans receivable	167,342		141,341
Commercial Loans | Equipment finance
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Loans individually evaluated for impairment	0		0
Allowance for credit losses, Loans collectively evaluated	6,284		6,537
Allowance for credit losses, Total	6,284		6,537
Total loans receivable, Loans individually evaluated for impairment	0		275
Total loans receivable, Loans collectively evaluated	378,629		317,645
Total loans receivable	378,629		317,920
Commercial Loans | Municipal finance
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Loans individually evaluated for impairment	0		0
Allowance for credit losses, Loans collectively evaluated	297		534
Allowance for credit losses, Total	297		534
Total loans receivable, Loans individually evaluated for impairment	0		0
Total loans receivable, Loans collectively evaluated	130,260		140,421
Total loans receivable	130,260		140,421
Retail Consumer Loans
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Total	8,712	$ 7,965	10,722	$ 12,661	$ 14,945	$ 6,956
Total loans receivable	666,699		765,232
Retail Consumer Loans | One-to-four family
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Loans individually evaluated for impairment	0		2
Allowance for credit losses, Loans collectively evaluated	4,400		5,407
Allowance for credit losses, Total	4,400		5,409
Total loans receivable, Loans individually evaluated for impairment	2,493		1,977
Total loans receivable, Loans collectively evaluated	345,452		404,572
Total loans receivable	347,945		406,549
Retail Consumer Loans | HELOCs - originated
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Loans individually evaluated for impairment	0		0
Allowance for credit losses, Loans collectively evaluated	1,409		1,512
Allowance for credit losses, Total	1,409		1,512
Total loans receivable, Loans individually evaluated for impairment	0		0
Total loans receivable, Loans collectively evaluated	128,445		130,225
Total loans receivable	128,445		130,225
Retail Consumer Loans | HELOCs - purchased
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Loans individually evaluated for impairment	0		0
Allowance for credit losses, Loans collectively evaluated	295		452
Allowance for credit losses, Total	295		452
Total loans receivable, Loans individually evaluated for impairment	0		0
Total loans receivable, Loans collectively evaluated	26,911		38,976
Total loans receivable	26,911		38,976
Retail Consumer Loans | Construction and land/lots
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Loans individually evaluated for impairment	0		0
Allowance for credit losses, Loans collectively evaluated	903		812
Allowance for credit losses, Total	903		812
Total loans receivable, Loans individually evaluated for impairment	0		0
Total loans receivable, Loans collectively evaluated	72,735		66,027
Total loans receivable	72,735		66,027
Retail Consumer Loans | Indirect auto finance
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Loans individually evaluated for impairment	0		0
Allowance for credit losses, Loans collectively evaluated	1,586		2,367
Allowance for credit losses, Total	1,586		2,367
Total loans receivable, Loans individually evaluated for impairment	0		0
Total loans receivable, Loans collectively evaluated	83,903		115,093
Total loans receivable	83,903		115,093
Retail Consumer Loans | Consumer
Financing Receivable, Allowance for Credit Loss [Line Items]
Allowance for credit losses, Loans individually evaluated for impairment	0		0
Allowance for credit losses, Loans collectively evaluated	119		170
Allowance for credit losses, Total	119		170
Total loans receivable, Loans individually evaluated for impairment	0		0
Total loans receivable, Loans collectively evaluated	6,760		8,362
Total loans receivable	$ 6,760		$ 8,362