Loans and Allowance for Credit Losses on Loans - Past Due Financing Receivables (Details) - USD ($) $ in Thousands	Jun. 30, 2022	Jun. 30, 2021
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	$ 2,769,295	$ 2,733,267
30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	2,079	3,313
90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	1,823	3,802
Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	3,902	7,115
Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	2,765,393	2,726,152
Commercial real estate
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	1,370,105	1,321,703
Commercial real estate | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	396
Commercial real estate | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	52	1,717
Commercial real estate | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	52	2,113
Commercial real estate | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	1,370,053	1,319,590
Commercial real estate | Construction and land development
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	291,202	179,427
Commercial real estate | Construction and land development | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	0
Commercial real estate | Construction and land development | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	37
Commercial real estate | Construction and land development | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	37
Commercial real estate | Construction and land development | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	291,202	179,390
Commercial real estate | Commercial Real Estate Loan, Owner Occupied
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	335,658	324,350
Commercial real estate | Commercial Real Estate Loan, Owner Occupied | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	396
Commercial real estate | Commercial Real Estate Loan, Owner Occupied | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	52	1,680
Commercial real estate | Commercial Real Estate Loan, Owner Occupied | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	52	2,076
Commercial real estate | Commercial Real Estate Loan, Owner Occupied | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	335,606	322,274
Commercial real estate | Commercial Real Estate Loan, Non-Owner Occupied
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	662,159	727,361
Commercial real estate | Commercial Real Estate Loan, Non-Owner Occupied | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	0
Commercial real estate | Commercial Real Estate Loan, Non-Owner Occupied | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	0
Commercial real estate | Commercial Real Estate Loan, Non-Owner Occupied | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	0
Commercial real estate | Commercial Real Estate Loan, Non-Owner Occupied | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	662,159	727,361
Commercial real estate | Multifamily
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	81,086	90,565
Commercial real estate | Multifamily | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	0
Commercial real estate | Multifamily | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	0
Commercial real estate | Multifamily | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	0
Commercial real estate | Multifamily | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	81,086	90,565
Commercial
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	717,620	646,332
Commercial | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	441	634
Commercial | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	56	366
Commercial | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	497	1,000
Commercial | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	717,123	645,332
Commercial | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	192,652	141,341
Commercial | Commercial and industrial | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	255	634
Commercial | Commercial and industrial | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	19
Commercial | Commercial and industrial | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	255	653
Commercial | Commercial and industrial | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	192,397	140,688
Commercial | Equipment finance
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	394,541	317,920
Commercial | Equipment finance | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	186	0
Commercial | Equipment finance | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	56	347
Commercial | Equipment finance | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	242	347
Commercial | Equipment finance | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	394,299	317,573
Commercial | Municipal Loans
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	129,766	140,421
Commercial | Municipal Loans | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	0
Commercial | Municipal Loans | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	0
Commercial | Municipal Loans | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	0
Commercial | Municipal Loans | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	129,766	140,421
Commercial | PPP Loan
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	661	46,650
Commercial | PPP Loan | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	0
Commercial | PPP Loan | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	0
Commercial | PPP Loan | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	0	0
Commercial | PPP Loan | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	661	46,650
Residential Portfolio Segment
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	596,187	641,777
Residential Portfolio Segment | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	1,308	1,606
Residential Portfolio Segment | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	1,538	1,424
Residential Portfolio Segment | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	2,846	3,030
Residential Portfolio Segment | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	593,341	638,747
Residential Portfolio Segment | Construction and land development
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	81,847	66,027
Residential Portfolio Segment | Construction and land development | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	115	6
Residential Portfolio Segment | Construction and land development | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	22	35
Residential Portfolio Segment | Construction and land development | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	137	41
Residential Portfolio Segment | Construction and land development | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	81,710	65,986
Residential Portfolio Segment | One-to-four family
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	354,203	406,549
Residential Portfolio Segment | One-to-four family | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	910	1,112
Residential Portfolio Segment | One-to-four family | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	1,394	1,124
Residential Portfolio Segment | One-to-four family | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	2,304	2,236
Residential Portfolio Segment | One-to-four family | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	351,899	404,313
Residential Portfolio Segment | HELOCs
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	160,137	169,201
Residential Portfolio Segment | HELOCs | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	283	488
Residential Portfolio Segment | HELOCs | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	122	265
Residential Portfolio Segment | HELOCs | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	405	753
Residential Portfolio Segment | HELOCs | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	159,732	168,448
Consumer
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	85,383	123,455
Consumer | 30-89 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	330	677
Consumer | 90 Days
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	177	295
Consumer | Financial Asset, Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	507	972
Consumer | Financial Asset, Not Past Due
Financing Receivable, Past Due [Line Items]
Total loans, net of deferred loan fees and costs	$ 84,876	$ 122,483