Loans and Allowance for Loan Losses, Risk Category of Loans (Details) - USD ($) $ in Thousands		Jun. 30, 2015	Dec. 31, 2014
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		$ 3,242,948	$ 3,158,332
Impaired loans included in classified loans		3,600	4,100
Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		209,399	223,382
Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		197,346	210,660
Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		12,053	12,722
Commercial [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		185,879	194,124
Commercial [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		174,683	181,925
Commercial [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		11,196	12,199
Commercial [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		23,520	29,258
Commercial [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		22,663	28,735
Commercial [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		857	523
New York and Other States [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	2,635,858	2,612,379
New York and Other States [Member] | Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		193,371	203,988
New York and Other States [Member] | Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		181,318	191,266
New York and Other States [Member] | Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		12,053	12,722
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	169,957	174,788
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		158,761	162,589
New York and Other States [Member] | Commercial [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		11,196	12,199
New York and Other States [Member] | Commercial [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net	[1]	23,414	29,200
New York and Other States [Member] | Commercial [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		22,557	28,677
New York and Other States [Member] | Commercial [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		857	523
Florida [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		607,090	545,953
Florida [Member] | Commercial [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		16,028	19,394
Florida [Member] | Commercial [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		16,028	19,394
Florida [Member] | Commercial [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		0	0
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		15,922	19,336
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		15,922	19,336
Florida [Member] | Commercial [Member] | Commercial Real Estate [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		0	0
Florida [Member] | Commercial [Member] | Other [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		106	58
Florida [Member] | Commercial [Member] | Other [Member] | Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		106	58
Florida [Member] | Commercial [Member] | Other [Member] | Classified [Member]
Financing Receivable, Recorded Investment [Line Items]
Total loans, net		$ 0	$ 0

[1] Includes New York, New Jersey, Vermont and Massachusetts.