FDIC Loss Sharing Asset FDIC Loss Sharing Asset (Details) - Jun. 30, 2015 - USD ($) $ in Thousands		Total
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC Indemnification Asset		$ 11,890
FDIC Indemnification True Up		7,393
NonSingle Family Covered Assets	[1]	197,604
Single Family Covered Assets		88,893
Affinity Bank [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC Indemnification Asset		(201)
NonSingle Family Covered Assets	[1]	100,255
Single Family Covered Assets		10,923
Los Padres Bank [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC Indemnification Asset		5,523
NonSingle Family Covered Assets	[1]	66,867
Single Family Covered Assets		52,416
Western Commercial Bank [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC Indemnification Asset		406
FDIC Indemnification True Up		1,790
NonSingle Family Covered Assets	[1]	8,018
San Luis Trust Bank [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
FDIC Indemnification Asset		6,162
FDIC Indemnification True Up		5,603
NonSingle Family Covered Assets	[1]	22,464
Single Family Covered Assets		$ 25,554
Residential Mortgage [Member] | Affinity Bank [Member] | FDIC indemnification provision under loss share arrangement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision		Sep. 30, 2019
Residential Mortgage [Member] | Affinity Bank [Member] | Recovery provision for assuming bank under FDIC loss-share agreement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision		Sep. 30, 2019
Residential Mortgage [Member] | Los Padres Bank [Member] | FDIC indemnification provision under loss share arrangement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision		Sep. 30, 2020
Residential Mortgage [Member] | Los Padres Bank [Member] | Recovery provision for assuming bank under FDIC loss-share agreement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision		Sep. 30, 2020
Residential Mortgage [Member] | San Luis Trust Bank [Member] | FDIC indemnification provision under loss share arrangement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision		Mar. 31, 2021
Residential Mortgage [Member] | San Luis Trust Bank [Member] | Recovery provision for assuming bank under FDIC loss-share agreement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision		Mar. 31, 2021
Commercial Real Estate [Member] | Affinity Bank [Member] | FDIC indemnification provision under loss share arrangement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision		Sep. 30, 2015
Commercial Real Estate [Member] | Affinity Bank [Member] | Recovery provision for assuming bank under FDIC loss-share agreement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision		Sep. 30, 2017
Commercial Real Estate [Member] | Los Padres Bank [Member] | FDIC indemnification provision under loss share arrangement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision		Sep. 30, 2015
Commercial Real Estate [Member] | Los Padres Bank [Member] | Recovery provision for assuming bank under FDIC loss-share agreement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision		Sep. 30, 2018
Commercial Real Estate [Member] | Western Commercial Bank [Member] | FDIC indemnification provision under loss share arrangement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision		Dec. 31, 2015
Commercial Real Estate [Member] | Western Commercial Bank [Member] | Recovery provision for assuming bank under FDIC loss-share agreement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision		Dec. 31, 2018
Commercial Real Estate [Member] | San Luis Trust Bank [Member] | FDIC indemnification provision under loss share arrangement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision		Mar. 31, 2016
Commercial Real Estate [Member] | San Luis Trust Bank [Member] | Recovery provision for assuming bank under FDIC loss-share agreement [Member]
Schedule of FDIC Loss Sharing Asset [Line Items]
Expiration date of provision		Mar. 31, 2019

[1] (1) Excludes securities