FDIC Loss Sharing Asset FDIC Loss Sharing Asset (Details) (USD $) In Thousands, unless otherwise specified	3 Months Ended
	Mar. 31, 2015
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC loss sharing asset	$ 16,532
FDIC Indemnification True Up	7,290
NonSingle Family Covered Assets	213,486	[1]
Single Family Covered Assets	95,480
Affinity Bank [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC loss sharing asset	140
NonSingle Family Covered Assets	105,614	[1]
Single Family Covered Assets	10,932
Los Padres Bank [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC loss sharing asset	8,161
NonSingle Family Covered Assets	73,072	[1]
Single Family Covered Assets	55,447
Western Commercial Bank [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC loss sharing asset	710
FDIC Indemnification True Up	1,773
NonSingle Family Covered Assets	8,799	[1]
San Luis Trust Bank [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC loss sharing asset	7,521
FDIC Indemnification True Up	5,517
NonSingle Family Covered Assets	26,001	[1]
Single Family Covered Assets	$ 29,101

[1] (1) Excludes securities