REPORTABLE SEGMENTS - Narrative (Details) - USD ($)	3 Months Ended								12 Months Ended
	Dec. 31, 2020	Sep. 30, 2020	Jun. 30, 2020	Mar. 31, 2020	Dec. 31, 2019	Sep. 30, 2019	Jun. 30, 2019	Mar. 31, 2019	Dec. 31, 2020	Dec. 31, 2019	Dec. 31, 2018
Segment Reporting Information [Line Items]
Net interest income	$ 59,407,000	$ 57,824,000	$ 58,938,000	$ 57,257,000	$ 56,125,000	$ 56,153,000	$ 57,460,000	$ 58,338,000	$ 233,426,000	$ 228,076,000	$ 234,566,000
Fees and other income	26,656,000	23,045,000	22,662,000	21,521,000	26,793,000	25,126,000	24,380,000	25,248,000	93,884,000	101,547,000	149,997,000
Total revenue	86,063,000	80,869,000	81,600,000	78,778,000	82,918,000	81,279,000	81,840,000	83,586,000	327,310,000	329,623,000	384,563,000
Provision/(credit) for loan losses	(2,999,000)	(4,569,000)	22,604,000	16,962,000	(3,668,000)	167,000	1,363,000	(1,426,000)	31,998,000	(3,564,000)	(2,198,000)
Total operating expense	57,967,000	60,937,000	61,453,000	60,908,000	58,457,000	55,537,000	55,659,000	60,553,000	241,265,000	230,206,000	267,355,000
Income/(loss) before income taxes	31,095,000	24,501,000	(2,457,000)	908,000	28,129,000	25,575,000	24,818,000	24,459,000	54,047,000	102,981,000	119,406,000
Income tax expense/(benefit)	6,124,000	1,821,000	841,000	102,000	6,788,000	5,517,000	5,369,000	4,917,000	8,888,000	22,591,000	37,537,000
Net income from continuing operations									45,159,000	80,390,000	81,869,000
Less: Net income attributable to noncontrolling interests	0	0	0	6,000	97,000	96,000	69,000	100,000	6,000	362,000	3,487,000
Net income from discontinued operations									0	0	2,002,000
Net income/(loss) attributable to the Company	24,971,000	$ 22,680,000	$ (3,298,000)	$ 800,000	21,244,000	$ 19,962,000	$ 19,380,000	$ 19,442,000	45,153,000	80,028,000	80,384,000
Assets	10,048,733,000				8,830,501,000				10,048,733,000	8,830,501,000	8,494,625,000
Amortization of intangibles									2,697,000	2,691,000	2,929,000
Depreciation									12,237,000	11,228,000	10,694,000
Restructuring									0	1,646,000	7,828,000
Private Banking
Segment Reporting Information [Line Items]
Restructuring										1,300,000	6,600,000
Wealth Management and Trust
Segment Reporting Information [Line Items]
Restructuring										400,000	1,200,000
Operating Segments | Private Banking
Segment Reporting Information [Line Items]
Net interest income									235,984,000	231,796,000	238,036,000
Fees and other income									13,625,000	13,869,000	9,366,000
Total revenue									249,609,000	245,665,000	247,402,000
Provision/(credit) for loan losses									31,998,000	(3,564,000)	(2,198,000)
Total operating expense									167,980,000	157,891,000	165,263,000
Income/(loss) before income taxes									49,631,000	91,338,000	84,337,000
Income tax expense/(benefit)									6,564,000	19,110,000	16,313,000
Net income from continuing operations									43,067,000	72,228,000	68,024,000
Net income/(loss) attributable to the Company									43,067,000	72,228,000	68,024,000
Assets	9,964,691,000				8,746,289,000				9,964,691,000	8,746,289,000	8,424,967,000
Amortization of intangibles									304,000	37,000	0
Depreciation									10,881,000	9,737,000	8,646,000
Operating Segments | Wealth Management and Trust
Segment Reporting Information [Line Items]
Net interest income									81,000	407,000	338,000
Fees and other income									73,100,000	75,949,000	79,192,000
Total revenue									73,181,000	76,356,000	79,530,000
Total operating expense									61,948,000	58,375,000	66,492,000
Income/(loss) before income taxes									11,233,000	17,981,000	13,038,000
Income tax expense/(benefit)									3,506,000	5,768,000	4,145,000
Net income from continuing operations									7,727,000	12,213,000	8,893,000
Net income/(loss) attributable to the Company									7,727,000	12,213,000	8,893,000
Assets	152,412,000				148,803,000				152,412,000	148,803,000	130,346,000
Amortization of intangibles									2,393,000	2,654,000	2,775,000
Depreciation									1,205,000	1,297,000	1,660,000
Holding Company and Eliminations
Segment Reporting Information [Line Items]
Net interest income									(2,639,000)	(4,127,000)	(3,808,000)
Fees and other income									7,159,000	11,729,000	61,439,000
Total revenue									4,520,000	7,602,000	57,631,000
Total operating expense									11,337,000	13,940,000	35,600,000
Income/(loss) before income taxes									(6,817,000)	(6,338,000)	22,031,000
Income tax expense/(benefit)									(1,182,000)	(2,287,000)	17,079,000
Net income from continuing operations									(5,635,000)	(4,051,000)	4,952,000
Less: Net income attributable to noncontrolling interests									6,000	362,000	3,487,000
Net income from discontinued operations									0	0	2,002,000
Net income/(loss) attributable to the Company									(5,641,000)	(4,413,000)	3,467,000
Assets	$ (68,370,000)				$ (64,592,000)				(68,370,000)	(64,592,000)	(60,688,000)
Amortization of intangibles									0	0	154,000
Depreciation									$ 151,000	$ 194,000	$ 388,000