Parent Company Only Financial Information (Statement of Cash Flow) (Details) (USD $)	12 Months Ended
	Dec. 31, 2013	Dec. 31, 2012	Dec. 31, 2011	Dec. 31, 2013 Holding Company	Dec. 31, 2012 Holding Company	Dec. 31, 2011 Holding Company
Operating activities:
Net income	$ 17,574,000	$ 20,385,000	$ 12,555,000	$ 17,574,000	$ 20,385,000	$ 12,555,000
Adjustments to reconcile net income to net cash provided by operating activities:
(Equity in) excess dividends from undistributed earnings of subsidiaries				(5,600,000)	(11,276,000)	10,584,000
Gain on investment securities	(489,000)	(3,548,000)	(473,000)	0	(273,000)	0
Loss on debt extinguishment	0	4,144,000	0	0	1,033,000	0
Other, net				1,803,000	(663,000)	2,534,000
Net cash provided by operating activities	40,464,000	41,254,000	43,384,000	13,777,000	9,206,000	25,673,000
Investing activities:
Net proceeds from sales and maturities of investment securities				0	273,000	25,000
Investment in subsidiaries				0	(9,815,000)	0
Change in receivable from subsidiary				(619,000)	3,814,000	(3,451,000)
Net cash used in investing activities	(80,013,000)	(69,567,000)	(8,879,000)	(619,000)	(5,728,000)	(3,426,000)
Financing activities:
Proceeds from long-term borrowings	0	24,000,000	0	0	24,000,000	0
Payments on long-term borrowings	(7,025,000)	(40,517,000)	(15,391,000)	(4,800,000)	0	0
Repurchase of preferred shares			39,000,000	0	0	(39,000,000)
Redemption of junior subordinated debentures				0	(23,668,000)	0
Preferred stock dividends	0	0	(1,232,000)	0	0	(1,232,000)
Purchase of treasury stock	(228,000)	(156,000)	(187,000)	(228,000)	(1,357,000)	(187,000)
Proceeds from issuance of common shares	8,000	6,000	10,000	8,000	6,000	10,000
Cash dividends paid	(5,419,000)	(4,457,000)	(3,922,000)	(5,419,000)	(4,457,000)	(3,922,000)
Excess tax benefit from share-based payments	79,000	16,000	0	79,000	709,000	0
Net cash provided by (used in) financing activities	30,827,000	51,905,000	(70,199,000)	(10,360,000)	(4,767,000)	(44,331,000)
Net (decrease) increase in cash and cash equivalents	(8,722,000)	23,592,000	(35,694,000)	2,798,000	(1,289,000)	(22,084,000)
Cash and cash equivalents at the beginning of year					2,793,000	4,082,000
Cash and cash equivalents at beginning of period	62,542,000	38,950,000	74,644,000	2,793,000	4,082,000
Cash and cash equivalents at end of period	53,820,000	62,542,000	38,950,000	5,591,000	2,793,000	4,082,000
Supplemental cash flow information:
Interest paid	$ 11,839,000	$ 15,570,000	$ 21,386,000	$ 915,000	$ 2,246,000	$ 1,981,000