Loans and the Allowance for Loan Losses (Details) - Credit quality indicators - USD ($) $ in Thousands	Sep. 30, 2018		Dec. 31, 2017
Financing Receivable, Recorded Investment [Line Items]
Gross loans	$ 4,465,774		$ 4,174,810
Commercial Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	933,169	[1]	824,082
Commercial Real Estate Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	2,739,943		2,592,909
Commercial Construction Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	494,206		483,216
Residential Real Estate Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	295,948		271,795
Consumer Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	2,508		2,808
Pass [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	4,361,295		4,045,989
Pass [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	887,024	[1]	767,020
Pass [Member] | Commercial Real Estate Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	2,699,338		2,534,973
Pass [Member] | Commercial Construction Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	481,226		475,066
Pass [Member] | Residential Real Estate Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	291,223		266,163
Pass [Member] | Consumer Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	2,484		2,767
Special Mention [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	28,748		43,620
Special Mention [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	12,639	[1]	3,764
Special Mention [Member] | Commercial Real Estate Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	10,651		34,335
Special Mention [Member] | Commercial Construction Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	5,458		5,521
Special Mention [Member] | Residential Real Estate Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans
Special Mention [Member] | Consumer Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans
Substandard [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	75,731		85,201
Substandard [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	33,506	[1]	53,298
Substandard [Member] | Commercial Real Estate Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	29,954		23,601
Substandard [Member] | Commercial Construction Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	7,522		2,629
Substandard [Member] | Residential Real Estate Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	4,725		5,632
Substandard [Member] | Consumer Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans	24		41
Doubtful [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans
Doubtful [Member] | Commercial Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans		[1]
Doubtful [Member] | Commercial Real Estate Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans
Doubtful [Member] | Commercial Construction Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans
Doubtful [Member] | Residential Real Estate Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans
Doubtful [Member] | Consumer Portfolio Segment [Member]
Financing Receivable, Recorded Investment [Line Items]
Gross loans

[1] Reflects a $17.0 million charge-off related to the taxi medallion loans classified as "substandard".