Loans Held for Investment (Details 4) (USD $) In Thousands, unless otherwise specified	3 Months Ended
	Mar. 31, 2014 item	Dec. 31, 2013	Mar. 31, 2013
Loans Held for Investment
Number of areas where the entity's credit quality is maintained and credit risk managed	2
Number of Pass scale grades	6
Credit Risk Grades
Total Gross Loans	$ 1,325,469	$ 1,243,252	$ 945,991
Business loans: Commercial and industrial
Credit Risk Grades
Total Gross Loans	271,877	187,035	140,592
Business loans: Commercial owner occupied
Credit Risk Grades
Total Gross Loans	223,848	221,089	166,571
Business loans: SBA
Credit Risk Grades
Total Gross Loans	11,045	10,659	5,116
Business loans: Warehouse facilities
Credit Risk Grades
Total Gross Loans	81,033	87,517	138,935
Real estate loans: Commercial non-owner occupied
Credit Risk Grades
Total Gross Loans	333,490	333,544	256,015
Real estate loans: Multi-family
Credit Risk Grades
Total Gross Loans	223,200	233,689	139,100
Real estate loans: One-to-four family
Credit Risk Grades
Total Gross Loans	141,469	145,235	87,109
Real estate loans: Construction
Credit Risk Grades
Total Gross Loans	29,857	13,040
Real estate loans: Land
Credit Risk Grades
Total Gross Loans	6,170	7,605	7,863
Other loans
Credit Risk Grades
Total Gross Loans	3,480	3,839	4,690
Pass
Credit Risk Grades
Total Gross Loans	1,305,532	1,221,125	913,266
Pass | Business loans: Commercial and industrial
Credit Risk Grades
Total Gross Loans	270,024	184,247	136,947
Pass | Business loans: Commercial owner occupied
Credit Risk Grades
Total Gross Loans	212,663	207,872	149,787
Pass | Business loans: SBA
Credit Risk Grades
Total Gross Loans	11,031	10,659	5,063
Pass | Business loans: Warehouse facilities
Credit Risk Grades
Total Gross Loans	81,033	87,517	138,935
Pass | Real estate loans: Commercial non-owner occupied
Credit Risk Grades
Total Gross Loans	328,645	329,538	247,140
Pass | Real estate loans: Multi-family
Credit Risk Grades
Total Gross Loans	222,178	232,661	137,014
Pass | Real estate loans: One-to-four family
Credit Risk Grades
Total Gross Loans	140,453	144,152	85,849
Pass | Real estate loans: Construction
Credit Risk Grades
Total Gross Loans	29,857	13,040
Pass | Real estate loans: Land
Credit Risk Grades
Total Gross Loans	6,170	7,605	7,853
Pass | Other loans
Credit Risk Grades
Total Gross Loans	3,478	3,834	4,678
Special Mention
Credit Risk Grades
Total Gross Loans	780	2,092	3,765
Special Mention | Business loans: Commercial and industrial
Credit Risk Grades
Total Gross Loans		12	96
Special Mention | Business loans: Commercial owner occupied
Credit Risk Grades
Total Gross Loans	272	1,217	2,792
Special Mention | Real estate loans: Commercial non-owner occupied
Credit Risk Grades
Total Gross Loans		352	360
Special Mention | Real estate loans: Multi-family
Credit Risk Grades
Total Gross Loans	508	511	517
Substandard
Credit Risk Grades
Total Gross Loans	19,157	20,035	28,960
Substandard | Business loans: Commercial and industrial
Credit Risk Grades
Total Gross Loans	1,853	2,776	3,549
Substandard | Business loans: Commercial owner occupied
Credit Risk Grades
Total Gross Loans	10,913	12,000	13,992
Substandard | Business loans: SBA
Credit Risk Grades
Total Gross Loans	14		53
Substandard | Real estate loans: Commercial non-owner occupied
Credit Risk Grades
Total Gross Loans	4,845	3,654	8,515
Substandard | Real estate loans: Multi-family
Credit Risk Grades
Total Gross Loans	514	517	1,569
Substandard | Real estate loans: One-to-four family
Credit Risk Grades
Total Gross Loans	1,016	1,083	1,260
Substandard | Real estate loans: Land
Credit Risk Grades
Total Gross Loans			10
Substandard | Other loans
Credit Risk Grades
Total Gross Loans	$ 2	$ 5	$ 12