Loans Held for Investment - Internal Risk Grading System (Details) - USD ($) $ in Thousands	Jun. 30, 2016	Dec. 31, 2015	Jun. 30, 2015
Credit Risk Grades
Totals	$ 2,927,554	$ 2,262,683	$ 2,118,323
Commercial and industrial
Credit Risk Grades
Totals	508,141	309,741	284,873
Franchise
Credit Risk Grades
Totals	403,855	328,925	257,582
Commercial owner occupied
Credit Risk Grades
Totals	443,060	294,726	294,545
SBA
Credit Risk Grades
Totals	86,076	62,256	50,306
Warehouse facilities
Credit Risk Grades
Totals	0	143,200	198,113
Commercial non-owner occupied
Credit Risk Grades
Totals	526,362	421,583	402,786
Multi-family
Credit Risk Grades
Totals	613,573	429,003	400,237
One-to-four family
Credit Risk Grades
Totals	106,538	80,050	84,283
Construction
Credit Risk Grades
Totals	215,786	169,748	124,448
Land
Credit Risk Grades
Totals	18,341	18,340	16,339
Other loans
Credit Risk Grades
Totals	5,822	5,111	$ 4,811
Pass
Credit Risk Grades
Totals	2,897,622	2,242,242
Pass | Commercial and industrial
Credit Risk Grades
Totals	498,556	306,513
Pass | Franchise
Credit Risk Grades
Totals	402,394	327,295
Pass | Commercial owner occupied
Credit Risk Grades
Totals	432,542	286,270
Pass | SBA
Credit Risk Grades
Totals	85,747	62,256
Pass | Warehouse facilities
Credit Risk Grades
Totals		143,200
Pass | Commercial non-owner occupied
Credit Risk Grades
Totals	522,108	418,917
Pass | Multi-family
Credit Risk Grades
Totals	611,143	425,616
Pass | One-to-four family
Credit Risk Grades
Totals	105,619	78,997
Pass | Construction
Credit Risk Grades
Totals	215,786	169,748
Pass | Land
Credit Risk Grades
Totals	18,323	18,319
Pass | Other loans
Credit Risk Grades
Totals	5,404	5,111
Special Mention
Credit Risk Grades
Totals	5,107	700
Special Mention | Commercial and industrial
Credit Risk Grades
Totals	3,694	73
Special Mention | Franchise
Credit Risk Grades
Totals	0	0
Special Mention | Commercial owner occupied
Credit Risk Grades
Totals	1,413	627
Special Mention | SBA
Credit Risk Grades
Totals	0	0
Special Mention | Warehouse facilities
Credit Risk Grades
Totals		0
Special Mention | Commercial non-owner occupied
Credit Risk Grades
Totals	0	0
Special Mention | Multi-family
Credit Risk Grades
Totals	0	0
Special Mention | One-to-four family
Credit Risk Grades
Totals	0	0
Special Mention | Construction
Credit Risk Grades
Totals	0	0
Special Mention | Land
Credit Risk Grades
Totals	0	0
Special Mention | Other loans
Credit Risk Grades
Totals	0	0
Substandard
Credit Risk Grades
Totals	23,364	18,280
Substandard | Commercial and industrial
Credit Risk Grades
Totals	5,891	3,155
Substandard | Franchise
Credit Risk Grades
Totals	0	169
Substandard | Commercial owner occupied
Credit Risk Grades
Totals	9,105	7,829
Substandard | SBA
Credit Risk Grades
Totals	329	0
Substandard | Warehouse facilities
Credit Risk Grades
Totals		0
Substandard | Commercial non-owner occupied
Credit Risk Grades
Totals	4,254	2,666
Substandard | Multi-family
Credit Risk Grades
Totals	2,430	3,387
Substandard | One-to-four family
Credit Risk Grades
Totals	919	1,053
Substandard | Construction
Credit Risk Grades
Totals	0	0
Substandard | Land
Credit Risk Grades
Totals	18	21
Substandard | Other loans
Credit Risk Grades
Totals	418	0
Doubtful
Credit Risk Grades
Totals	1,461	1,461
Doubtful | Commercial and industrial
Credit Risk Grades
Totals	0	0
Doubtful | Franchise
Credit Risk Grades
Totals	1,461	1,461
Doubtful | Commercial owner occupied
Credit Risk Grades
Totals	0	0
Doubtful | SBA
Credit Risk Grades
Totals	0	0
Doubtful | Warehouse facilities
Credit Risk Grades
Totals		0
Doubtful | Commercial non-owner occupied
Credit Risk Grades
Totals	0	0
Doubtful | Multi-family
Credit Risk Grades
Totals	0	0
Doubtful | One-to-four family
Credit Risk Grades
Totals	0	0
Doubtful | Construction
Credit Risk Grades
Totals	0	0
Doubtful | Land
Credit Risk Grades
Totals	0	0
Doubtful | Other loans
Credit Risk Grades
Totals	$ 0	$ 0