Allowance for Loan Losses (Details) (USD $) In Thousands, unless otherwise specified	3 Months Ended
	Mar. 31, 2014	Mar. 31, 2013	Dec. 31, 2013
Allocation of allowance as well as the activity in allowance
Balance, at the beginning of the period	$ 8,200	$ 7,994
Charge-offs	(501)	(480)
Recoveries	37	184
Provisions for (reduction in) loan losses	949	296
Balance, at the end of the period	8,685	7,994
Other disclosures
Amount of allowance attributed to: Specifically evaluated impaired loans	104	616	105
Amount of allowance attributed to: General portfolio allocation	8,581	7,378
Loans individually evaluated for impairment	2,683	3,752
Specific reserves to total loans individually evaluated for impairment (as a percent)	3.88%	16.24%
Loans collectively evaluated for impairment	1,322,786	942,239
General reserves to total loans collectively evaluated for impairment (as a percent)	0.65%	0.78%
Total gross loans	1,325,469	945,991	1,243,252
Total allowance to gross loans (as a percent)	0.66%	0.85%
Owner Occupied Commercial Real Estate Loans, Commercial and Industrial Loans and SBA Loans
Allowance for Loan Losses
Actual trailing period considered for determination of allowance for loan losses factor	36 months
Trailing period one considered for determination of allowance for loan losses factor	24 months
Trailing period two considered for determination of allowance for loan losses factor	12 months
Annualized trailing period considered for determination of allowance for loan losses factor	6 months
Period considered for comparison of allowance for loan losses factor	10 years
Period considered for comparison of entity's allowance for loan losses factor	15 years
Trailing period considered for comparison of allowance for loan losses factor	12 months
Multi-Family and Non-Owner Occupied Commercial Real Estate Loans
Allowance for Loan Losses
Actual trailing period considered for determination of allowance for loan losses factor	36 months
Trailing period one considered for determination of allowance for loan losses factor	24 months
Trailing period two considered for determination of allowance for loan losses factor	12 months
Annualized trailing period considered for determination of allowance for loan losses factor	6 months
Period considered for comparison of allowance for loan losses factor	10 years
Period considered for comparison of entity's allowance for loan losses factor	15 years
Trailing period considered for comparison of allowance for loan losses factor	12 months
One-to-Four Family and Consumer Loans
Allowance for Loan Losses
Actual trailing period considered for determination of allowance for loan losses factor	36 months
Trailing period one considered for determination of allowance for loan losses factor	24 months
Trailing period two considered for determination of allowance for loan losses factor	12 months
Annualized trailing period considered for determination of allowance for loan losses factor	6 months
Period considered for comparison of allowance for loan losses factor	10 years
Period considered for comparison of entity's allowance for loan losses factor	15 years
Trailing period considered for comparison of allowance for loan losses factor	12 months
Commercial and industrial
Allocation of allowance as well as the activity in allowance
Balance, at the beginning of the period	1,968	1,310
Charge-offs	(124)	(58)
Recoveries	5	7
Provisions for (reduction in) loan losses	516	1,037
Balance, at the end of the period	2,365	2,296
Other disclosures
Amount of allowance attributed to: Specifically evaluated impaired loans		256
Amount of allowance attributed to: General portfolio allocation	2,365	2,040
Loans individually evaluated for impairment	31	580
Specific reserves to total loans individually evaluated for impairment (as a percent)	0.00%	44.14%
Loans collectively evaluated for impairment	271,846	140,012
General reserves to total loans collectively evaluated for impairment (as a percent)	0.87%	1.46%
Total gross loans	271,877	140,592	187,035
Total allowance to gross loans (as a percent)	0.87%	1.63%
Commercial owner occupied
Allocation of allowance as well as the activity in allowance
Balance, at the beginning of the period	1,818	1,512
Provisions for (reduction in) loan losses	(64)	153
Balance, at the end of the period	1,754	1,665
Other disclosures
Amount of allowance attributed to: General portfolio allocation	1,754	1,665
Loans individually evaluated for impairment	718	245
Specific reserves to total loans individually evaluated for impairment (as a percent)	0.00%	0.00%
Loans collectively evaluated for impairment	223,130	166,326
General reserves to total loans collectively evaluated for impairment (as a percent)	0.79%	1.00%
Total gross loans	223,848	166,571	221,089
Total allowance to gross loans (as a percent)	0.78%	1.00%
SBA
Allocation of allowance as well as the activity in allowance
Balance, at the beginning of the period	151	79
Charge-offs		(5)
Recoveries	2	19
Provisions for (reduction in) loan losses	26	(43)
Balance, at the end of the period	179	50
Other disclosures
Amount of allowance attributed to: General portfolio allocation	179	50
Loans individually evaluated for impairment	14	129
Specific reserves to total loans individually evaluated for impairment (as a percent)	0.00%	0.00%
Loans collectively evaluated for impairment	11,031	4,987
General reserves to total loans collectively evaluated for impairment (as a percent)	1.62%	1.00%
Total gross loans	11,045	5,116	10,659
Total allowance to gross loans (as a percent)	1.62%	0.98%
Warehouse
Allowance for Loan Losses
Period considered for comparison of allowance for loan losses factor	10 years
Period considered for comparison of entity's allowance for loan losses factor	15 years
Trailing period considered for comparison of allowance for loan losses factor	12 months
Allocation of allowance as well as the activity in allowance
Balance, at the beginning of the period	392	1,544
Provisions for (reduction in) loan losses	(16)	(814)
Balance, at the end of the period	376	730
Other disclosures
Amount of allowance attributed to: General portfolio allocation	376	730
Specific reserves to total loans individually evaluated for impairment (as a percent)	0.00%	0.00%
Loans collectively evaluated for impairment	81,033	138,935
General reserves to total loans collectively evaluated for impairment (as a percent)	0.46%	0.53%
Total gross loans	81,033	138,935	87,517
Total allowance to gross loans (as a percent)	0.46%	0.53%
Commercial non-owner occupied
Allocation of allowance as well as the activity in allowance
Balance, at the beginning of the period	1,658	1,459
Charge-offs	(365)	(401)
Provisions for (reduction in) loan losses	564	345
Balance, at the end of the period	1,857	1,403
Other disclosures
Amount of allowance attributed to: Specifically evaluated impaired loans			1
Amount of allowance attributed to: General portfolio allocation	1,857	1,403
Loans individually evaluated for impairment	1,327	1,974
Specific reserves to total loans individually evaluated for impairment (as a percent)	0.00%	0.00%
Loans collectively evaluated for impairment	332,163	254,041
General reserves to total loans collectively evaluated for impairment (as a percent)	0.56%	0.55%
Total gross loans	333,490	256,015	333,544
Total allowance to gross loans (as a percent)	0.56%	0.55%
Multi-family
Allocation of allowance as well as the activity in allowance
Balance, at the beginning of the period	817	1,145
Provisions for (reduction in) loan losses	(12)	(639)
Balance, at the end of the period	805	506
Other disclosures
Amount of allowance attributed to: General portfolio allocation	805	506
Specific reserves to total loans individually evaluated for impairment (as a percent)	0.00%	0.00%
Loans collectively evaluated for impairment	223,200	139,100
General reserves to total loans collectively evaluated for impairment (as a percent)	0.36%	0.36%
Total gross loans	223,200	139,100	233,689
Total allowance to gross loans (as a percent)	0.36%	0.36%
One-to-four family
Allocation of allowance as well as the activity in allowance
Balance, at the beginning of the period	1,099	862
Charge-offs	(12)	(10)
Recoveries	30	43
Provisions for (reduction in) loan losses	(198)	279
Balance, at the end of the period	919	1,174
Other disclosures
Amount of allowance attributed to: Specifically evaluated impaired loans	104	360	104
Amount of allowance attributed to: General portfolio allocation	815	814
Loans individually evaluated for impairment	593	824
Specific reserves to total loans individually evaluated for impairment (as a percent)	17.54%	43.69%
Loans collectively evaluated for impairment	140,876	86,285
General reserves to total loans collectively evaluated for impairment (as a percent)	0.58%	0.94%
Total gross loans	141,469	87,109	145,235
Total allowance to gross loans (as a percent)	0.65%	1.35%
Construction
Allocation of allowance as well as the activity in allowance
Balance, at the beginning of the period	136
Provisions for (reduction in) loan losses	187
Balance, at the end of the period	323
Other disclosures
Amount of allowance attributed to: General portfolio allocation	323
Specific reserves to total loans individually evaluated for impairment (as a percent)	0.00%	0.00%
Loans collectively evaluated for impairment	29,857
General reserves to total loans collectively evaluated for impairment (as a percent)	1.08%	0.00%
Total gross loans	29,857		13,040
Total allowance to gross loans (as a percent)	1.08%	0.00%
Land
Allocation of allowance as well as the activity in allowance
Balance, at the beginning of the period	127	31
Provisions for (reduction in) loan losses	(46)	90
Balance, at the end of the period	81	121
Other disclosures
Amount of allowance attributed to: General portfolio allocation	81	121
Specific reserves to total loans individually evaluated for impairment (as a percent)	0.00%	0.00%
Loans collectively evaluated for impairment	6,170	7,863
General reserves to total loans collectively evaluated for impairment (as a percent)	1.31%	1.54%
Total gross loans	6,170	7,863	7,605
Total allowance to gross loans (as a percent)	1.31%	1.54%
Other loans
Allocation of allowance as well as the activity in allowance
Balance, at the beginning of the period	34	52
Charge-offs		(6)
Recoveries		115
Provisions for (reduction in) loan losses	(8)	(112)
Balance, at the end of the period	26	49
Other disclosures
Amount of allowance attributed to: General portfolio allocation	26	49
Specific reserves to total loans individually evaluated for impairment (as a percent)	0.00%	0.00%
Loans collectively evaluated for impairment	3,480	4,690
General reserves to total loans collectively evaluated for impairment (as a percent)	0.75%	1.04%
Total gross loans	$ 3,480	$ 4,690	$ 3,839
Total allowance to gross loans (as a percent)	0.75%	1.04%