Loans and Allowance for Loan Losses (Details) - USD ($) $ in Thousands		Jun. 30, 2015	Mar. 31, 2015	Dec. 31, 2014	Jun. 30, 2014	Mar. 31, 2014	Dec. 31, 2013
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		$ 3,242,948		$ 3,158,332
Less: Allowance for loan losses		45,571	$ 45,944	46,327	$ 46,935	$ 47,035	$ 47,714
Net loans		3,197,377		3,112,005
Real Estate Construction Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		26,800		38,500
Residential Borrowers [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		12,400		17,600
Commercial Borrowers Residential Purpose [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		14,400		20,900
Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		209,399		223,382
Less: Allowance for loan losses		4,022	4,024	4,071	4,073	3,840	4,019
Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		185,879		194,124
Commercial [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		23,520		29,258
Real Estate Mortgage - 1 to 4 Family [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		3,024,875		2,927,356
Less: Allowance for loan losses		41,087	41,529	42,088	42,752	43,091	43,597
Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		2,610,658		2,517,567
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		59,271		57,655
Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		354,946		352,134
Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		8,674		7,594
Less: Allowance for loan losses		462	$ 391	168	$ 110	$ 104	$ 98
New York and Other States [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	2,635,858		2,612,379
New York and Other States [Member] | Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		193,371		203,988
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	169,957		174,788
New York and Other States [Member] | Commercial [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	23,414		29,200
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	2,075,448		2,041,140
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	51,857		51,713
New York and Other States [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	307,519		308,764
New York and Other States [Member] | Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net	[1]	7,663		6,774
Florida [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		607,090		545,953
Florida [Member] | Commercial [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		16,028		19,394
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		15,922		19,336
Florida [Member] | Commercial [Member] | Other [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		106		58
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | First Mortgages [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		535,210		476,427
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Loans [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		7,414		5,942
Florida [Member] | Real Estate Mortgage - 1 to 4 Family [Member] | Home Equity Lines of Credit [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		47,427		43,370
Florida [Member] | Installment [Member]
Loans and Leases Receivable, Net Amount [Abstract]
Total loans, net		$ 1,011		$ 820

[1] Includes New York, New Jersey, Vermont and Massachusetts.