Mortgage Banking Operations - Single Family MSR Roll Forward (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2019	Mar. 31, 2018
Servicing Asset at Fair Value, Amount [Roll Forward]
Beginning balance	$ 75,047
Additions and amortization:
Changes due to modeled amortization	8,983	$ 8,870
Changes in fair value of MSR due to changes in market inputs and/or model updates	(5,278)	30,019
Ending balance	68,250
Single family [Member]
Servicing Asset at Fair Value, Amount [Roll Forward]
Beginning balance	252,168	258,560
Additions and amortization:
Originations	7,287	14,353
Sale of single family MSRs	(176,944)	0
Changes due to modeled amortization	(8,983)	(8,870)
Net additions and amortization	(178,640)	5,483
Changes in fair value of MSR due to changes in market inputs and/or model updates	(5,278)	30,019
Ending balance	$ 68,250	$ 294,062