Regulatory Matters and Significant Risks or Uncertainties (Details) (USD $)	3 Months Ended		12 Months Ended	27 Months Ended	60 Months Ended		3 Months Ended			3 Months Ended			3 Months Ended
	Mar. 31, 2013	Mar. 31, 2012	Dec. 31, 2012	Mar. 31, 2013	Dec. 31, 2012	Dec. 31, 2010	Mar. 31, 2013 Series A Preferred Stock [Member]	Mar. 31, 2013 Commercial real estate and construction and land development loans [Member]	Dec. 31, 2012 Commercial real estate and construction and land development loans [Member]	Mar. 31, 2013 Nonperforming Financing Receivable [Member]	Mar. 31, 2012 Nonperforming Financing Receivable [Member]	Dec. 31, 2012 Nonperforming Financing Receivable [Member]	Mar. 31, 2013 Royal Bank [Member]	Dec. 31, 2011 Royal Bank [Member]	Jul. 15, 2009 Royal Bank [Member]	Mar. 31, 2013 Royal Tax Lien Services, LLC [Member]	Mar. 31, 2013 Crusader Servicing Corporation [Member]
Federal Deposit Insurance Corporation and Department of Banking Orders [Abstract]
Minimum Tier one leverage ratio required (in hundredths)	8.00%			8.00%										8.00%	8.00%
Minimum total capital to risk weighted assets ratio required (in hundredths)	12.00%			12.00%										12.00%	12.00%
Tier I capital (to average assets, leverage) (in hundredths)	8.96%			8.96%									8.96%
Total capital (to risk-weighted assets) (in hundredths)	15.92%			15.92%									15.92%
Continued Losses [Abstract]
Continued significant losses over five calendar years					$ 119,600,000
Net income (loss)	118,000	(869,000)											203,000
Gain (loss) on sale of premises	676,000	0
Decline in provision for loan and lease losses	335,000
Department of Justice fine	0	1,600,000	2,000,000
Percentage of entity's share in loss contingency (in hundredths)			60.00%
Department of Justice fine in period attributable to parent		960,000
Deferred tax valuation allowance	39,600,000			39,600,000
Financing Receivable, Recorded Investment [Line Items]
Non accrual financing receivables	17,556,000		23,004,000	17,556,000	23,004,000					17,600,000		23,000,000
Non performing loans, charge offs										2,000,000	153,000
Other real estate owned (OREO)	13,264,000		13,435,000	13,264,000	13,435,000					13,264,000		13,435,000
Net classified loans and forecosed property	45,600,000		51,800,000	45,600,000	51,800,000
Delinquent loans held for investment										4,700,000		4,600,000
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total gross loans	358,055,000	400,675,000	344,682,000	358,055,000	344,682,000			221,800,000	216,100,000			21,432,000
Percentage of loans held for investment (in hundredths)								62.00%	63.00%
Liquidity and Funds Management [Abstract]
Collateralized delivery requirement with Federal Home Loan Banks (in hundredths)	105.00%			105.00%
Available borrowing capacity at the federal home loan bank	13,300,000			13,300,000
Unpledged agency securities were available to be pledged as collateral	189,400,000			189,400,000
Cash on hand	20,000,000			20,000,000
Limited availability to borrow from the federal reserve discount window	7,300,000			7,300,000
Liquidity to deposits ratio (in hundredths)	44.20%			44.20%
Percentage of policy target (in hundredths)	12.00%			12.00%
Liquidity to total liabilities ratio (in hundredths)	34.20%			34.20%
Percentage of total liability ratio, policy target (in hundredths)	10.00%			10.00%
Borrowings	108,200,000		108,300,000	108,200,000	108,300,000
Unfunded pension plan obligations	16,800,000		16,900,000	16,800,000	16,900,000
Class of Stock [Line Items]
Series A Preferred stock on which quarterly dividend suspended	29,531,000		29,396,000	29,531,000	29,396,000		30,400,000
Trust preferred securities with suspend interest payments	25,774,000		25,774,000	25,774,000	25,774,000
Preferred stock dividend in arrears							6,300,000
Trust preferred securities interest expense in arrears	2,600,000			2,600,000
Capital Adequacy [Abstract]
Minimum total capital to risk weighted assets ratio required (in hundredths)	12.00%			12.00%										12.00%	12.00%
Minimum Tier one leverage ratio required (in hundredths)	8.00%			8.00%										8.00%	8.00%
Total capital (to risk-weighted assets) (in hundredths)	15.92%			15.92%									15.92%
Tier I capital (to average assets, leverage) (in hundredths)	8.96%			8.96%									8.96%
Noncontrolling Interest [Line Items]
Percentage of ownership interest in subsidiary (in hundredths)																60.00%	60.00%
Company Plans and Strategy [Abstract]
Brokered deposits	0			0		89,100,000
Reduction in borrowings				46,700,000
Increase loans held for investment	13,900,000
An annualized reduction of discretionary expenses (in hundredths)	0.1
Reduction in workforce (in hundredths)	0.09
Restructuring charge directly related to one-time employee termination benefits	$ 87,000