Loans and Allowance for Loan Losses - Activity in the Allowance for Loan Losses (Details) - USD ($) $ in Thousands	3 Months Ended								12 Months Ended
	Dec. 31, 2020	Sep. 30, 2020	Jun. 30, 2020	Mar. 31, 2020	Dec. 31, 2019	Sep. 30, 2019	Jun. 30, 2019	Mar. 31, 2019	Dec. 31, 2020	Dec. 31, 2019	Dec. 31, 2018
Beginning balance				$ 21,751				$ 20,945	$ 21,751	$ 20,945	$ 20,351
Impact of CECL Adoption									379
Impact of Day 1 PCD - Empire Acquisition									4,099
Charge-off's									(4,005)	(2,778)	(932)
Recoveries									366	773	951
Provision (benefit)	$ 3,862	$ 2,470	$ 9,619	7,178	$ (318)	$ 683	$ 1,474	972	22,600	2,811	575
Provision (benefit)									22,563
Ending balance	45,153				21,751				45,153	21,751	20,945
Multi-family Residential Portfolio Segment [Member]
Beginning balance				5,391				5,676	5,391	5,676	5,823
Impact of CECL Adoption									(650)
Impact of Day 1 PCD - Empire Acquisition									444
Charge-off's										(190)	(99)
Recoveries									38	44	6
Provision (benefit)										(139)	(54)
Provision (benefit)									1,334
Ending balance	6,557				5,391				6,557	5,391	5,676
Commercial Real Estate
Beginning balance				4,429				4,315	4,429	4,315	4,643
Impact of CECL Adoption									1,170
Impact of Day 1 PCD - Empire Acquisition									587
Recoveries										37
Provision (benefit)										77	(328)
Provision (benefit)									2,141
Ending balance	8,327				4,429				8,327	4,429	4,315
One-To-Four Family - Mixed-Use Property Portfolio Segment [Member]
Beginning balance				1,817				1,867	1,817	1,867	2,545
Impact of CECL Adoption									(55)
Impact of Day 1 PCD - Empire Acquisition									183
Charge-off's									(3)	(89)	(3)
Recoveries									138	197	136
Provision (benefit)										(158)	(811)
Provision (benefit)									(94)
Ending balance	1,986				1,817				1,986	1,817	1,867
One-To-Four Family - Residential Portfolio Segment [Member]
Beginning balance				756				749	756	749	1,082
Impact of CECL Adoption									(160)
Impact of Day 1 PCD - Empire Acquisition									158
Charge-off's										(113)	(1)
Recoveries									12	13	569
Provision (benefit)										107	(901)
Provision (benefit)									103
Ending balance	869				756				869	756	749
Co-Operative Apartments Portfolio Segment [Member]
Beginning balance
Impact of CECL Adoption
Impact of Day 1 PCD - Empire Acquisition
Charge-off's
Recoveries
Provision (benefit)
Provision (benefit)
Ending balance
Construction Portfolio Segment [Member]
Beginning balance				441				329	441	329	68
Impact of CECL Adoption									(279)
Impact of Day 1 PCD - Empire Acquisition									20
Provision (benefit)										112	261
Provision (benefit)									315
Ending balance	497				441				497	441	329
Small Business Administration Portfolio Segment [Member]
Beginning balance				363				418	363	418	669
Impact of CECL Adoption									1,180
Impact of Day 1 PCD - Empire Acquisition									278
Charge-off's									(178)		(392)
Recoveries									70	60	51
Provision (benefit)										(115)	90
Provision (benefit)									538
Ending balance	2,251				363				2,251	363	418
Taxi Medallion Portfolio Segment [Member]
Impact of Day 1 PCD - Empire Acquisition									124
Charge-off's									(1,075)		(393)
Recoveries										134	143
Provision (benefit)										(134)	250
Provision (benefit)									951
Commercial Business and Other Portfolio Segment [Member]
Beginning balance				$ 8,554				$ 7,591	8,554	7,591	5,521
Impact of CECL Adoption									(827)
Impact of Day 1 PCD - Empire Acquisition									2,305
Charge-off's									(2,749)	(2,386)	(44)
Recoveries									108	288	46
Provision (benefit)										3,061	2,068
Provision (benefit)									17,275
Ending balance	$ 24,666				$ 8,554				$ 24,666	$ 8,554	$ 7,591