Loans and Allowance for Loan Losses - Risk category of our loan portfolio (Details) $ in Thousands	Jun. 30, 2020 USD ($)
2020	$ 494,949
2019	918,756
2018	980,586
2017	759,352
2016	680,196
Prior	1,923,659
Revolving Loans Amortized Cost Basis	202,096
Lines of credit converted to term loans	23,681
One-To-Four Family - Residential Portfolio Segment [Member]
2020	11,091
2019	25,155
2018	28,224
2017	15,649
2016	12,273
Prior	70,438
Revolving Loans Amortized Cost Basis	8,281
Lines of credit converted to term loans	23,681
One-To-Four Family - Residential Portfolio Segment [Member] | Pass [Member]
2020	11,091
2019	25,155
2018	26,395
2017	14,736
2016	11,318
Prior	62,336
Revolving Loans Amortized Cost Basis	7,836
Lines of credit converted to term loans	18,842
One-To-Four Family - Residential Portfolio Segment [Member] | Watch [Member]
2017	913
Prior	2,174
Revolving Loans Amortized Cost Basis	299
Lines of credit converted to term loans	2,276
One-To-Four Family - Residential Portfolio Segment [Member] | Special Mention [Member]
2018	1,829
Prior	2,265
Lines of credit converted to term loans	1,197
One-To-Four Family - Residential Portfolio Segment [Member] | Substandard [Member]
2016	955
Prior	3,663
Revolving Loans Amortized Cost Basis	146
Lines of credit converted to term loans	1,366
One-To-Four Family - Mixed-Use Property Portfolio Segment [Member]
2020	23,451
2019	69,499
2018	78,478
2017	62,917
2016	54,821
Prior	306,538
One-To-Four Family - Mixed-Use Property Portfolio Segment [Member] | Pass [Member]
2020	23,451
2019	66,989
2018	76,271
2017	62,016
2016	53,660
Prior	293,719
One-To-Four Family - Mixed-Use Property Portfolio Segment [Member] | Watch [Member]
2019	2,510
2018	1,022
2017	901
2016	1,161
Prior	9,406
One-To-Four Family - Mixed-Use Property Portfolio Segment [Member] | Special Mention [Member]
2018	391
Prior	1,658
One-To-Four Family - Mixed-Use Property Portfolio Segment [Member] | Substandard [Member]
2018	794
Prior	1,755
Commercial Real Estate Portfolio Segment [Member]
2020	90,991
2019	261,521
2018	282,767
2017	198,071
2016	255,184
Prior	560,716
Commercial Real Estate Portfolio Segment [Member] | Pass [Member]
2020	90,991
2019	250,747
2018	282,767
2017	195,472
2016	231,133
Prior	491,110
Commercial Real Estate Portfolio Segment [Member] | Watch [Member]
2019	9,072
2017	2,599
2016	22,694
Prior	66,239
Commercial Real Estate Portfolio Segment [Member] | Special Mention [Member]
2016	1,357
Prior	2,370
Commercial Real Estate Portfolio Segment [Member] | Substandard [Member]
2019	1,702
Prior	997
Construction Portfolio Segment [Member]
2020	7,333
2019	15,635
2018	36,890
2016	9,399
Construction Portfolio Segment [Member] | Pass [Member]
2020	7,333
2019	15,635
2018	36,212
Construction Portfolio Segment [Member] | Watch [Member]
2018	678
2016	9,399
Multi-family Residential Portfolio Segment [Member]
2020	132,962
2019	312,625
2018	369,758
2017	375,665
2016	277,767
Prior	819,380
Revolving Loans Amortized Cost Basis	4,205
Multi-family Residential Portfolio Segment [Member] | Pass [Member]
2020	131,983
2019	311,043
2018	367,770
2017	372,340
2016	274,937
Prior	800,168
Revolving Loans Amortized Cost Basis	3,884
Multi-family Residential Portfolio Segment [Member] | Watch [Member]
2020	979
2019	1,582
2017	2,586
2016	2,121
Prior	17,199
Revolving Loans Amortized Cost Basis	321
Multi-family Residential Portfolio Segment [Member] | Special Mention [Member]
2017	739
Prior	982
Multi-family Residential Portfolio Segment [Member] | Substandard [Member]
2018	1,988
2016	709
Prior	1,031
Commercial Business Secured By Portfolio Segment [member]
2020	61,354
2019	91,638
2018	64,001
2017	23,543
2016	47,503
Prior	87,248
Commercial Business Secured By Portfolio Segment [member] | Pass [Member]
2020	61,354
2019	91,638
2018	56,916
2017	22,223
2016	44,822
Prior	83,498
Commercial Business Secured By Portfolio Segment [member] | Watch [Member]
2018	7,085
2017	1,320
2016	2,681
Prior	3,750
Commercial Business Portfolio Segment [Member]
2020	76,160
2019	141,720
2018	116,855
2017	79,007
2016	20,452
Prior	74,033
Revolving Loans Amortized Cost Basis	189,486
Commercial Business Portfolio Segment [Member] | Pass [Member]
2020	75,715
2019	140,668
2018	110,312
2017	72,507
2016	17,769
Prior	71,794
Revolving Loans Amortized Cost Basis	174,234
Commercial Business Portfolio Segment [Member] | Watch [Member]
2020	445
2019	1,001
2018	4,110
2017	3,145
Prior	112
Revolving Loans Amortized Cost Basis	10,361
Commercial Business Portfolio Segment [Member] | Special Mention [Member]
2018	2,433
2016	2,683
Prior	447
Revolving Loans Amortized Cost Basis	4,543
Commercial Business Portfolio Segment [Member] | Substandard [Member]
2019	51
2017	3,355
Prior	1,680
Revolving Loans Amortized Cost Basis	348
Small Business Administration Portfolio Segment [Member]
2020	91,607
2019	963
2018	3,613
2017	4,500
2016	2,797
Prior	1,988
Small Business Administration Portfolio Segment [Member] | Pass [Member]
2020	91,607
2019	963
2018	3,613
2017	1,058
2016	2,620
Prior	1,669
Small Business Administration Portfolio Segment [Member] | Watch [Member]
2017	2,272
Prior	266
Small Business Administration Portfolio Segment [Member] | Special Mention [Member]
Prior	53
Small Business Administration Portfolio Segment [Member] | Substandard [Member]
2017	1,170
2016	177
Taxi Medallion Portfolio Segment [Member]
Prior	3,270
Taxi Medallion Portfolio Segment [Member] | Substandard [Member]
Prior	3,270
Other Portfolio Segment [Member]
Prior	48
Revolving Loans Amortized Cost Basis	124
Other Portfolio Segment [Member] | Pass [Member]
Prior	48
Revolving Loans Amortized Cost Basis	$ 124