Fair Value - Fair Value of Financial Instruments (Details) - USD ($) $ in Thousands	Dec. 31, 2018	Dec. 31, 2017
Financial assets:
Cash and money market instruments	$ 167,214	$ 169,112
Mortgage loans held for sale	4,158	4,148
Impaired loans carried at fair value	6,399	3,574
Derivative Asset	87	94
Other loans, net (3)	5,570,136	5,247,021
Loans receivable, net	5,580,780	5,254,837
Financial liabilities:
Time Deposits, Fair Value	1,044,620	1,035,093
Other	1,267	1,269
Total deposits	1,045,887	1,036,362
Short-term borrowings	221,966	391,289
Long-term debt	400,203	504,503
Subordinated debentures/notes	12,959	13,370
Derivative financial instruments:
Investment Owned, at Fair Value	1,354,843	1,455,660
Level 1
Financial assets:
Cash and money market instruments	167,214	169,112
Derivative Asset	0	0
Loans receivable, net	0	0
Financial liabilities:
Other	1,267	1,269
Total deposits	1,267	1,269
Level 2
Financial assets:
Mortgage loans held for sale	4,158	4,148
Derivative Asset	87	94
Loans receivable, net	4,245	4,242
Financial liabilities:
Time Deposits, Fair Value	1,044,620	1,035,093
Total deposits	1,044,620	1,035,093
Short-term borrowings	221,966	391,289
Long-term debt	400,203	504,503
Subordinated debentures/notes	12,959	13,370
Derivative financial instruments:
Investment Owned, at Fair Value	1,354,843	1,455,660
Level 3
Financial assets:
Impaired loans carried at fair value	6,399	3,574
Derivative Asset	0	0
Other loans, net (3)	5,570,136	5,247,021
Loans receivable, net	5,576,535	5,250,595
Financial liabilities:
Total deposits	0	0
Equity securities
Derivative financial instruments:
Investment Owned, at Fair Value	1,649	1,935
Equity securities | Level 1
Derivative financial instruments:
Investment Owned, at Fair Value	1,225	1,518
Equity securities | Level 3
Derivative financial instruments:
Investment Owned, at Fair Value	424	417
Fair value swap
Derivative financial instruments:
Swaps, fair value	226	226
Fair value swap | Level 3
Derivative financial instruments:
Swaps, fair value	226	226
Reported Value Measurement [Member]
Financial assets:
Cash and money market instruments	167,214	169,112
Mortgage loans held for sale	4,158	4,148
Impaired loans carried at fair value	6,399	3,574
Derivative Asset	87	94
Other loans, net (3)	5,629,976	5,314,679
Loans receivable, net	5,640,620	5,322,495
Financial liabilities:
Time Deposits, Fair Value	1,043,177	1,033,476
Other	1,267	1,269
Total deposits	1,044,444	1,034,745
Short-term borrowings	221,966	391,289
Long-term debt	400,000	500,000
Subordinated debentures/notes	15,000	15,000
Derivative financial instruments:
Investment Owned, at Fair Value	1,355,229	1,449,078
Reported Value Measurement [Member] | Equity securities
Derivative financial instruments:
Investment Owned, at Fair Value	1,649	1,935
Reported Value Measurement [Member] | Fair value swap
Derivative financial instruments:
Swaps, fair value	$ 226	$ 226