Loans and the Allowance for Loan Losses (Details) - Schedule of analysis of impaired loans, by class - USD ($) $ in Thousands		3 Months Ended		9 Months Ended
		Sep. 30, 2018	Sep. 30, 2017	Sep. 30, 2018	Sep. 30, 2017	Dec. 31, 2017
Financing Receivable, Impaired [Line Items]
No related allowance recorded, Recorded Investment		$ 65,619		$ 65,619		$ 83,531
No related allowance recorded, Unpaid Principal Balance		127,050		127,050		135,146
Impaired loans with No Related Allowance Average Recorded Investment		65,872	$ 29,264	74,868	$ 28,840
Impaired loans with No Related Allowance Interest Income Recognized		233	321	964	762
With an allowance recorded, Recorded Investment		8,792		8,792
With an allowance recorded, Unpaid Principal Balance		8,795		8,795
With an allowance recorded, Related Allowance		741		741
Impaired loans With An Allowance Recorded Average Recorded Investment		8,798	1,645	8,811	1,654
Impaired loans With An Allowance Recorded Interest Income Recognized		11	2	34	39
Total, Recorded Investment		74,411		74,411		84,664
Total, Unpaid Principal Balance		135,845		135,845		136,279
Total, Related Allowance		741		741		39
Total Impaired Loans Average Recorded Investment		74,670	30,909	83,679	30,494
Total Impaired Loans Interest Income Recognized		244	323	998	801
Commercial Portfolio Segment [Member]
Financing Receivable, Impaired [Line Items]
No related allowance recorded, Recorded Investment		31,471		31,471		49,761
No related allowance recorded, Unpaid Principal Balance		92,452		92,452		101,066
Impaired loans with No Related Allowance Average Recorded Investment	[1]	31,769	3,100	39,132	3,149
Impaired loans with No Related Allowance Interest Income Recognized	[1]	36	34	102	115
Total, Recorded Investment		31,471		31,471		49,761
Total, Unpaid Principal Balance		92,452		92,452		101,066
Total, Related Allowance
Total Impaired Loans Average Recorded Investment	[1]	31,769	3,100	39,132	3,149
Total Impaired Loans Interest Income Recognized	[1]	36	34	102	115
Commercial Real Estate Portfolio Segment [Member]
Financing Receivable, Impaired [Line Items]
No related allowance recorded, Recorded Investment		21,483		21,483		23,905
No related allowance recorded, Unpaid Principal Balance		21,636		21,636		23,976
Impaired loans with No Related Allowance Average Recorded Investment		21,557	19,302	21,714	18,813
Impaired loans with No Related Allowance Interest Income Recognized		105	221	475	424
With an allowance recorded, Recorded Investment		8,529		8,529		1,133
With an allowance recorded, Unpaid Principal Balance		8,529		8,529		1,133
With an allowance recorded, Related Allowance		709		709		39
Impaired loans With An Allowance Recorded Average Recorded Investment		8,534	1,645	8,544	1,654
Impaired loans With An Allowance Recorded Interest Income Recognized		11	2	34	39
Total, Recorded Investment		30,012		30,012		25,038
Total, Unpaid Principal Balance		30,165		30,165		25,109
Total, Related Allowance		709		709		39
Total Impaired Loans Average Recorded Investment		30,091	20,947	30,258	20,467
Total Impaired Loans Interest Income Recognized		116	223	509	463
Commercial Construction Portfolio Segment [Member]
Financing Receivable, Impaired [Line Items]
No related allowance recorded, Recorded Investment		10,428		10,428		6,662
No related allowance recorded, Unpaid Principal Balance		10,429		10,429		6,662
Impaired loans with No Related Allowance Average Recorded Investment		10,297	4,285	11,718	4,273
Impaired loans with No Related Allowance Interest Income Recognized		92	63	387	215
Total, Recorded Investment		10,428		10,428		6,662
Total, Unpaid Principal Balance		10,429		10,429		6,662
Total, Related Allowance
Total Impaired Loans Average Recorded Investment		10,297	4,285	11,718	4,273
Total Impaired Loans Interest Income Recognized		92	63	387	215
Residential Real Estate Portfolio Segment [Member]
Financing Receivable, Impaired [Line Items]
No related allowance recorded, Recorded Investment		2,237		2,237		3,203
No related allowance recorded, Unpaid Principal Balance		2,533		2,533		3,442
Impaired loans with No Related Allowance Average Recorded Investment		2,249	2,529	2,304	2,551
Impaired loans with No Related Allowance Interest Income Recognized			2		6
With an allowance recorded, Recorded Investment		263		263
With an allowance recorded, Unpaid Principal Balance		266		266
With an allowance recorded, Related Allowance		32		32
Impaired loans With An Allowance Recorded Average Recorded Investment		264		267
Impaired loans With An Allowance Recorded Interest Income Recognized
Total, Recorded Investment		2,500		2,500		3,203
Total, Unpaid Principal Balance		2,799		2,799		3,442
Total, Related Allowance		32		32
Total Impaired Loans Average Recorded Investment		2,513	2,259	2,571	2,551
Total Impaired Loans Interest Income Recognized			2		6
Consumer Portfolio Segment [Member]
Financing Receivable, Impaired [Line Items]
No related allowance recorded, Recorded Investment
No related allowance recorded, Unpaid Principal Balance
Impaired loans with No Related Allowance Average Recorded Investment			48		54
Impaired loans with No Related Allowance Interest Income Recognized			1		2
Total, Recorded Investment
Total, Unpaid Principal Balance
Total, Related Allowance
Total Impaired Loans Average Recorded Investment			48		54
Total Impaired Loans Interest Income Recognized			$ 1		$ 2

[1] Reflects the entire taxi medallion portfolio moving back to the loans held-for-investment category from the loans held-for-sale category in November 2017.