Loans and Allowance for Credit Losses on Loans - Schedule of Allowance for Credit Losses on Financing Receivables After Adoption of ASU 2016-13 (Details) - USD ($) $ in Thousands	12 Months Ended
	Jun. 30, 2022	Jun. 30, 2021
Financing Receivable, Excluding Accrued Interest, Allowance for Credit Loss [Roll Forward]
Balance at beginning of period	$ 35,468	$ 28,072
Provision (benefit) for credit losses	(1,472)	(7,270)
Charge-offs	(2,512)	(3,533)
Recoveries	3,206	3,390
Net recoveries (charge-offs)	694	(143)
Balance at end of period	34,690	35,468
Cumulative-effect adjustment on the change in accounting for share-based payments
Financing Receivable, Excluding Accrued Interest, Allowance for Credit Loss [Roll Forward]
Provision (benefit) for credit losses		14,809
Commercial real estate
Financing Receivable, Excluding Accrued Interest, Allowance for Credit Loss [Roll Forward]
Balance at beginning of period	15,084	15,413
Provision (benefit) for credit losses	(2,273)	(311)
Charge-offs	(485)	(1,000)
Recoveries	1,088	149
Net recoveries (charge-offs)	603	(851)
Balance at end of period	13,414	15,084
Commercial real estate | Cumulative-effect adjustment on the change in accounting for share-based payments
Financing Receivable, Excluding Accrued Interest, Allowance for Credit Loss [Roll Forward]
Provision (benefit) for credit losses		833
Commercial
Financing Receivable, Excluding Accrued Interest, Allowance for Credit Loss [Roll Forward]
Balance at beginning of period	9,663	5,703
Provision (benefit) for credit losses	3,110	(446)
Charge-offs	(1,728)	(977)
Recoveries	991	2,143
Net recoveries (charge-offs)	(737)	1,166
Balance at end of period	12,036	9,663
Commercial | Cumulative-effect adjustment on the change in accounting for share-based payments
Financing Receivable, Excluding Accrued Interest, Allowance for Credit Loss [Roll Forward]
Provision (benefit) for credit losses		3,240
Residential Portfolio Segment
Financing Receivable, Excluding Accrued Interest, Allowance for Credit Loss [Roll Forward]
Balance at beginning of period	8,185	5,685
Provision (benefit) for credit losses	(1,423)	(6,308)
Charge-offs	(116)	(611)
Recoveries	965	732
Net recoveries (charge-offs)	849	121
Balance at end of period	7,611	8,185
Residential Portfolio Segment | Cumulative-effect adjustment on the change in accounting for share-based payments
Financing Receivable, Excluding Accrued Interest, Allowance for Credit Loss [Roll Forward]
Provision (benefit) for credit losses		8,687
Consumer
Financing Receivable, Excluding Accrued Interest, Allowance for Credit Loss [Roll Forward]
Balance at beginning of period	2,536	1,271
Provision (benefit) for credit losses	(886)	(205)
Charge-offs	(183)	(945)
Recoveries	162	366
Net recoveries (charge-offs)	(21)	(579)
Balance at end of period	$ 1,629	2,536
Consumer | Cumulative-effect adjustment on the change in accounting for share-based payments
Financing Receivable, Excluding Accrued Interest, Allowance for Credit Loss [Roll Forward]
Provision (benefit) for credit losses		$ 2,049