Regulatory Matters and Significant Risks or Uncertainties (Details) (USD $)	3 Months Ended			6 Months Ended		12 Months Ended	36 Months Ended
	Jun. 30, 2012	Mar. 31, 2012	Jun. 30, 2011	Jun. 30, 2012	Jun. 30, 2011	Dec. 31, 2011	Jun. 30, 2012	Dec. 31, 2010	Jun. 30, 2009
Federal Deposit Insurance Corporation and Department of Banking Orders [Abstract]
Minimum Tier one leverage ratio required (in hundredths)	8.00%			8.00%			8.00%
Minimum total capital to risk weighted assets ratio required (in hundredths)	12.00%			12.00%			12.00%
Continued Losses [Abstract]
Continued significant losses over four calendar years	$ 104,000,000			$ 104,000,000			$ 104,000,000
Net loss	1,950,000		4,225,000	2,819,000	5,736,000
Improvement in net loss				2,900,000
Percentage of improvement in net loss (in hundredths)				50.90%
Reduction in provision for loan losses				3,500,000
Loss contingency accrual	400,000	1,600,000	0	2,000,000	0
Percentage of entity's share in loss contingency (in hundredths)				60.00%
Amount of loss contingency	1,200,000			1,200,000			1,200,000
Deferred tax valuation allowance	36,400,000			36,400,000			36,400,000
Financing Receivable, Recorded Investment [Line Items]
Charge-offs to the allowance for loan and lease losses	628,000		6,897,000	781,000	7,074,000	13,211,000
Other real estate owned (OREO)	23,727,000	24,304,000	25,448,000	23,727,000	25,448,000	21,016,000	23,727,000
Net Classified Loans and Forecosed Property	60,500,000			60,500,000			60,500,000		149,600,000
Delinquent loans held for investment	4,100,000			4,100,000			4,100,000		36,300,000
Other than temporary impairment losses	859,000		381,000	859,000	381,000
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total gross loans	372,836,000		438,561,000	372,836,000	438,561,000	414,866,000	372,836,000
Liquidity and Funds Management [Abstract]
Percentage of collateralized delivery requirement with federal home loan banks (in hundredths)	105.00%			105.00%			105.00%
Available borrowing capacity at the federal home loan bank	30,100,000			30,100,000			30,100,000
Unpledged agency securities were available to be pledged as collateral	183,600,000			183,600,000			183,600,000
Cash on hand	32,418,000		54,806,000	32,418,000	54,806,000	24,506,000	32,418,000	51,733,000
Amount of limited availability to borrow from the federal reserve discount window	5,600,000			5,600,000			5,600,000
Liquidity to deposits ratio (in hundredths)	45.90%			45.90%			45.90%
Percentage of policy target (in hundredths)	12.00%			12.00%			12.00%
Liquidity to total liabilities ratio (in hundredths)	35.50%			35.50%			35.50%
Percentage of aggregate policy target (in hundredths)	10.00%			10.00%			10.00%
Brokered certificates of deposit	0			0		5,823,000	0		226,900,000
Borrowings declined							160,300,000
Borrowings	123,600,000			123,600,000			123,600,000		283,900,000
Unfunded pension plan obligations	15,000,000			15,000,000			15,000,000
Class of Stock [Line Items]
Suspend interest payments of trust preferred securities	25,774,000			25,774,000			25,774,000
Interest expense, trust preferred securities				2,100,000
Actual Ratio Under RAP [Abstract]
Total capital (to risk-weighted assets) (in hundredths)	16.52%			16.52%			16.52%
Tier I capital (to average assets, leverage) (in hundredths)	9.37%			9.37%			9.37%
Noncontrolling Interest [Line Items]
Change in loss contingency loss in period	400,000
Management Plans [Abstract]
Assets	830,626,000		909,684,000	830,626,000	909,684,000	848,448,000	830,626,000		1,300,000,000
Reduction in assets							490,000,000
Crusader Servicing Corporation [Member]
Noncontrolling Interest [Line Items]
Ownership interest percentage in subsidiary (in hundredths)	60.00%			60.00%			60.00%
Royal Tax Lien Services, LLC [Member]
Noncontrolling Interest [Line Items]
Ownership interest percentage in subsidiary (in hundredths)	60.00%			60.00%			60.00%
Royal Bank America Leasing, LP [Member]
Noncontrolling Interest [Line Items]
Ownership interest percentage in subsidiary (in hundredths)	60.00%			60.00%			60.00%
Series A Preferred Stock [Member]
Class of Stock [Line Items]
Series A Preferred stock on which quarterly dividend suspended	30,400,000			30,400,000			30,400,000
Preferred stock dividend in arrears				4,900,000
Commercial real estate and construction and land development loans [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total gross loans	212,900,000			212,900,000		236,700,000	212,900,000
Percentage of loans held for investment (in hundredths)	57.00%			57.00%		57.00%	57.00%
CRE Loan Concentration	157,800,000			157,800,000			157,800,000		289,100,000
Commercial real estate [Member]
Financing Receivable, Recorded Investment [Line Items]
Charge-offs to the allowance for loan and lease losses	0		1,685,000	0	1,685,000	1,685,000
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total gross loans	166,315,000		180,103,000	166,315,000	180,103,000	182,579,000	166,315,000
CRE Concentration As A Percentage Of Total Capital RAP	190.40%			190.40%
CRE Concentration Percentage Of Tier I Capital RAP	206.40%			206.40%
CRE Concentration Percentage Of Tier I Capital US GAAP	184.20%			184.20%
CRE Concentration As A Percentage Of Total Capital US GAAP	171.10%			171.10%
Construction and land development [Member]
Financing Receivable, Recorded Investment [Line Items]
Charge-offs to the allowance for loan and lease losses	0		3,649,000	0	3,649,000	5,755,000
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total gross loans	46,548,000		66,313,000	46,548,000	66,313,000	54,120,000	46,548,000
CRE Concentration As A Percentage Of Total Capital RAP	56.70%			56.70%
CRE Concentration Percentage Of Tier I Capital RAP	61.40%			61.40%
CRE Concentration Percentage Of Tier I Capital US GAAP	54.80%			54.80%
CRE Concentration As A Percentage Of Total Capital US GAAP	50.90%			50.90%
Loans Receivable	47,000,000			47,000,000			47,000,000
Nonperforming loans [Member]
Financing Receivable, Recorded Investment [Line Items]
Non performing loans	32,145,000		62,729,000	32,145,000	62,729,000		32,145,000
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total gross loans	28,571,000			28,571,000		38,755,000	28,571,000
Nonperforming loans [Member] | Commercial real estate [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total gross loans	12,047,000			12,047,000		16,671,000	12,047,000
Nonperforming loans [Member] | Construction and land development [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total gross loans	$ 6,818,000			$ 6,818,000		$ 13,014,000	$ 6,818,000