Consolidated Statements of Financial Condition (Parenthetical) - USD ($) $ in Thousands	Sep. 30, 2018	Dec. 31, 2017
Statement of Financial Position [Abstract]
Interest-earning instruments	$ 35,763	$ 30,268
Investment securities held at fair value	831,102	846,268
Fair value of loans held for sale	350,948	577,313
Allowance for losses on loans held for investment	40,438	37,847
Fair value of loans held for investment	4,089	5,477
Fair value of single family MSR	$ 263,622	$ 258,560
Preferred stock, par value (in dollars per share)	$ 0	$ 0
Preferred stock, shares authorized	10,000	10,000
Preferred stock, shares issued	0	0
Preferred stock, shares outstanding	0	0
Common stock, par value (in dollars per share)	$ 0	$ 0
Common stock, shares authorized	160,000,000	160,000,000
Common stock, shares issued	26,989,742	26,888,288
Common stock, shares outstanding	26,989,742	26,888,288