Loans Held for Investment - Internal Risk Grading System (Details) - USD ($) $ in Thousands	Mar. 31, 2017	Dec. 31, 2016	Mar. 31, 2016
Credit Risk Grades
Totals	$ 3,382,447	$ 3,245,694
Gross loans	3,393,537	3,245,694	$ 2,850,494
Commercial and industrial
Credit Risk Grades
Totals	593,457	563,169
Gross loans	593,457	563,169	491,112
Franchise
Credit Risk Grades
Totals	493,158	459,421
Gross loans	493,158	459,421	371,875
Commercial owner occupied
Credit Risk Grades
Totals	482,295	454,918
Gross loans	482,295	454,918	424,289
SBA
Credit Risk Grades
Totals	96,486	96,705
Gross loans	107,233	96,705	71,069
Commercial non-owner occupied
Credit Risk Grades
Totals	612,444	586,975
Gross loans	612,787	586,975	522,080
Multi-family
Credit Risk Grades
Totals	682,237	690,955
Gross loans	682,237	690,955	619,485
One-to-four family
Credit Risk Grades
Totals	100,423	100,451
Gross loans	100,423	100,451	106,854
Construction
Credit Risk Grades
Totals	298,279	269,159
Gross loans	298,279	269,159	218,069
Land
Credit Risk Grades
Totals	19,738	19,829
Gross loans	19,738	19,829	18,222
Other loans
Credit Risk Grades
Totals	3,930	4,112
Gross loans	3,930	4,112	$ 6,045
Pass
Credit Risk Grades
Totals	3,371,129	3,216,683
Pass | Commercial and industrial
Credit Risk Grades
Totals	579,569	550,919
Pass | Franchise
Credit Risk Grades
Totals	493,158	459,421
Pass | Commercial owner occupied
Credit Risk Grades
Totals	477,109	450,416
Pass | SBA
Credit Risk Grades
Totals	106,644	96,190
Pass | Commercial non-owner occupied
Credit Risk Grades
Totals	611,357	585,093
Pass | Multi-family
Credit Risk Grades
Totals	681,690	681,942
Pass | One-to-four family
Credit Risk Grades
Totals	99,992	100,010
Pass | Construction
Credit Risk Grades
Totals	298,279	269,159
Pass | Land
Credit Risk Grades
Totals	19,724	19,814
Pass | Other loans
Credit Risk Grades
Totals	3,607	3,719
Special Mention
Credit Risk Grades
Totals	11,186	15,970
Special Mention | Commercial and industrial
Credit Risk Grades
Totals	9,365	8,216
Special Mention | Franchise
Credit Risk Grades
Totals	0	0
Special Mention | Commercial owner occupied
Credit Risk Grades
Totals	997	281
Special Mention | SBA
Credit Risk Grades
Totals	18	53
Special Mention | Commercial non-owner occupied
Credit Risk Grades
Totals	806	810
Special Mention | Multi-family
Credit Risk Grades
Totals	0	6,610
Special Mention | One-to-four family
Credit Risk Grades
Totals	0	0
Special Mention | Construction
Credit Risk Grades
Totals	0	0
Special Mention | Land
Credit Risk Grades
Totals	0	0
Special Mention | Other loans
Credit Risk Grades
Totals	0	0
Substandard
Credit Risk Grades
Totals	11,222	12,791
Substandard | Commercial and industrial
Credit Risk Grades
Totals	4,523	3,784
Substandard | Franchise
Credit Risk Grades
Totals	0	0
Substandard | Commercial owner occupied
Credit Risk Grades
Totals	4,189	4,221
Substandard | SBA
Credit Risk Grades
Totals	571	462
Substandard | Commercial non-owner occupied
Credit Risk Grades
Totals	624	1,072
Substandard | Multi-family
Credit Risk Grades
Totals	547	2,403
Substandard | One-to-four family
Credit Risk Grades
Totals	431	441
Substandard | Construction
Credit Risk Grades
Totals	0	0
Substandard | Land
Credit Risk Grades
Totals	14	15
Substandard | Other loans
Credit Risk Grades
Totals	323	393
Doubtful
Credit Risk Grades
Totals	0	250
Doubtful | Commercial and industrial
Credit Risk Grades
Totals	0	250
Doubtful | Franchise
Credit Risk Grades
Totals	0	0
Doubtful | Commercial owner occupied
Credit Risk Grades
Totals	0	0
Doubtful | SBA
Credit Risk Grades
Totals	0	0
Doubtful | Commercial non-owner occupied
Credit Risk Grades
Totals	0	0
Doubtful | Multi-family
Credit Risk Grades
Totals	0	0
Doubtful | One-to-four family
Credit Risk Grades
Totals	0	0
Doubtful | Construction
Credit Risk Grades
Totals	0	0
Doubtful | Land
Credit Risk Grades
Totals	0	0
Doubtful | Other loans
Credit Risk Grades
Totals	$ 0	$ 0