Loans Held for Investment (Details) (USD $)	3 Months Ended
	Mar. 31, 2015	Dec. 31, 2014	Mar. 31, 2014	Dec. 31, 2013
Loans Held for Investment
Total gross loans	2,130,853,000	$ 1,628,445,000	$ 1,325,469,000
Total gross loans held for investment	2,130,853,000	1,628,445,000	1,325,469,000
Deferred loan origination costs/(fees) and premiums/(discounts), net	534,000	177,000	(97,000)
Allowance for loan losses	(13,646,000)	(12,200,000)	(8,685,000)	(8,200,000)
Loans held for investment, net	2,117,741,000	1,616,422,000	1,316,687,000
Secured loans limit to one borrower	83,900,000
Unsecured loans limit to one borrower	50,400,000
Aggregate outstanding balance of loans to one borrower of secured credit	44,900,000
Purchased Credit Impaired
Total purchase credit impaired	5,978,000
Purchased credit impaired loans, nonaccrual status	1,700,000
Maximum
Loans Held for Investment
Secured loans limit to one borrower (as a percent)	25.00%
Unsecured loans limit to one borrower (as a percent)	15.00%
Business loans: Commercial and industrial
Loans Held for Investment
Total gross loans	420,218,000	428,207,000	271,877,000
Allowance for loan losses	(4,932,000)	(4,200,000)	(2,365,000)	(1,968,000)
Purchased Credit Impaired
Total purchase credit impaired	696,000
Business loans: Commercial owner occupied
Loans Held for Investment
Total gross loans	352,351,000	210,995,000	223,848,000
Allowance for loan losses	(1,853,000)	(1,757,000)	(1,754,000)	(1,818,000)
Purchased Credit Impaired
Total purchase credit impaired	2,937,000
Business loans: SBA
Loans Held for Investment
Total gross loans	49,855,000	28,404,000	11,045,000
Allowance for loan losses	(547,000)	(568,000)	(179,000)	(151,000)
Business loans: Warehouse facilities
Loans Held for Investment
Total gross loans	216,554,000	113,798,000	81,033,000
Allowance for loan losses	(889,000)	(546,000)	(376,000)	(392,000)
Real estate loans: Commercial non-owner occupied
Loans Held for Investment
Total gross loans	452,422,000	359,213,000	333,490,000
Allowance for loan losses	(2,131,000)	(2,007,000)	(1,857,000)	(1,658,000)
Purchased Credit Impaired
Total purchase credit impaired	2,344,000
Real estate loans: Multi-family
Loans Held for Investment
Total gross loans	397,130,000	262,965,000	223,200,000
Allowance for loan losses	(1,303,000)	(1,060,000)	(805,000)	(817,000)
Real estate loans: One-to-four family
Loans Held for Investment
Total gross loans	116,735,000	122,795,000	141,469,000
Allowance for loan losses	(674,000)	(842,000)	(919,000)	(1,099,000)
Purchased Credit Impaired
Total purchase credit impaired	1,000
Real estate loans: Construction
Loans Held for Investment
Total gross loans	111,704,000	89,682,000	29,857,000
Allowance for loan losses	(1,210,000)	(1,088,000)	(323,000)	(136,000)
Real estate loans: Land
Loans Held for Investment
Total gross loans	7,243,000	9,088,000	6,170,000
Allowance for loan losses	(78,000)	(108,000)	(81,000)	(127,000)
Other loans
Loans Held for Investment
Total gross loans	6,641,000	3,298,000	3,480,000
Allowance for loan losses	(29,000)	(24,000)	(26,000)	(34,000)
Canyon National
Loans Held for Investment
Unaccreted mark-to-market discount	1,200,000
Purchased Credit Impaired
Total purchase credit impaired	1,609,000
Canyon National | Business loans: Commercial and industrial
Purchased Credit Impaired
Total purchase credit impaired	95,000
Canyon National | Business loans: Commercial owner occupied
Purchased Credit Impaired
Total purchase credit impaired	549,000
Canyon National | Real estate loans: Commercial non-owner occupied
Purchased Credit Impaired
Total purchase credit impaired	965,000
Palm Desert National
Loans Held for Investment
Unaccreted mark-to-market discount	1,300,000
Purchased Credit Impaired
Total purchase credit impaired	1,000
Palm Desert National | Real estate loans: One-to-four family
Purchased Credit Impaired
Total purchase credit impaired	1,000
San Diego Trust Bank
Loans Held for Investment
Unaccreted mark-to-market discount	151,000
Independence Bank
Loans Held for Investment
Unaccreted mark-to-market discount	6,900,000
Purchased Credit Impaired
Total purchase credit impaired	4,368,000
Independence Bank | Business loans: Commercial and industrial
Purchased Credit Impaired
Total purchase credit impaired	601,000
Independence Bank | Business loans: Commercial owner occupied
Purchased Credit Impaired
Total purchase credit impaired	2,388,000
Independence Bank | Real estate loans: Commercial non-owner occupied
Purchased Credit Impaired
Total purchase credit impaired	1,379,000
First Association Bank
Loans Held for Investment
Mark-to-market premium	28,000