FDIC Loss Sharing Asset (Details) $ in Thousands	9 Months Ended
	Sep. 30, 2015 USD ($)
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC Indemnification Asset	$ 7,605
FDIC Indemnification True Up	7,502
NonSingle Family Covered Assets	167,083	[1]
Single Family Covered Assets	85,122
Affinity Bank [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC Indemnification Asset	(647)
NonSingle Family Covered Assets	87,093	[1]
Single Family Covered Assets	10,828
Los Padres Bank [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC Indemnification Asset	2,885
NonSingle Family Covered Assets	52,317	[1]
Single Family Covered Assets	50,237
Western Commercial Bank [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC Indemnification Asset	271
FDIC Indemnification True Up	1,810
NonSingle Family Covered Assets	7,848	[1]
San Luis Trust Bank [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC Indemnification Asset	5,096
FDIC Indemnification True Up	5,692
NonSingle Family Covered Assets	19,825	[1]
Single Family Covered Assets	$ 24,057
Residential Mortgage [Member] | Affinity Bank [Member] | FDIC indemnification provision under loss share arrangement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision	Sep. 30, 2019
Residential Mortgage [Member] | Affinity Bank [Member] | Recovery provision for assuming bank under FDIC loss-share agreement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision	Sep. 30, 2019
Residential Mortgage [Member] | Los Padres Bank [Member] | FDIC indemnification provision under loss share arrangement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision	Sep. 30, 2020
Residential Mortgage [Member] | Los Padres Bank [Member] | Recovery provision for assuming bank under FDIC loss-share agreement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision	Sep. 30, 2020
Residential Mortgage [Member] | San Luis Trust Bank [Member] | FDIC indemnification provision under loss share arrangement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision	Mar. 31, 2021
Residential Mortgage [Member] | San Luis Trust Bank [Member] | Recovery provision for assuming bank under FDIC loss-share agreement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision	Mar. 31, 2021
Commercial Real Estate [Member] | Affinity Bank [Member] | FDIC indemnification provision under loss share arrangement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision	Sep. 30, 2015
Commercial Real Estate [Member] | Affinity Bank [Member] | Recovery provision for assuming bank under FDIC loss-share agreement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision	Sep. 30, 2017
Commercial Real Estate [Member] | Los Padres Bank [Member] | FDIC indemnification provision under loss share arrangement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision	Sep. 30, 2015
Commercial Real Estate [Member] | Los Padres Bank [Member] | Recovery provision for assuming bank under FDIC loss-share agreement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision	Sep. 30, 2018
Commercial Real Estate [Member] | Western Commercial Bank [Member] | FDIC indemnification provision under loss share arrangement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision	Dec. 31, 2015
Commercial Real Estate [Member] | Western Commercial Bank [Member] | Recovery provision for assuming bank under FDIC loss-share agreement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision	Dec. 31, 2018
Commercial Real Estate [Member] | San Luis Trust Bank [Member] | FDIC indemnification provision under loss share arrangement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision	Mar. 31, 2016
Commercial Real Estate [Member] | San Luis Trust Bank [Member] | Recovery provision for assuming bank under FDIC loss-share agreement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision	Mar. 31, 2019

[1] Excludes securities