Commitments and Contingencies - Legal (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2019	Sep. 30, 2018	Sep. 30, 2019	Sep. 30, 2018	Sep. 30, 2019	Dec. 31, 2018
Representation and Warranty Claims
Roll-forward of claims activity for loans put-back to the mortgage origination segment
Balance, beginning of period	$ 33,074	$ 30,545	$ 33,784	$ 33,702
Claims made	6,423	5,477	16,110	17,827
Claims resolved with no payment	(7,022)	(578)	(14,289)	(12,331)
Repurchases	(1,506)	(3,513)	(4,150)	(6,847)
Indemnification payments	(243)		(729)	(420)
Balance, end of period	30,726	31,931	30,726	31,931
Reserve for Indemnification Liability:
Total	33,074	30,545	33,784	33,702	$ 30,726	$ 33,784
Indemnification Agreement
Commitments and Contingencies
Provision for indemnification losses	1,000	900	2,200	2,600
Roll-forward of claims activity for loans put-back to the mortgage origination segment
Balance, beginning of period	10,833	23,910	10,701	23,472
Additions for new sales	954	857	2,236	2,599
Repurchases	(117)	(228)	(325)	(474)
Early payment defaults	(51)	(72)	(290)	(259)
Indemnification payments	(87)	(4)	(182)	(124)
Change in reserves for loans sold in prior years	(81)	(292)	(689)	(1,043)
Balance, end of period	11,451	24,171	11,451	24,171
Reserve for Indemnification Liability:
Specific claims					958	676
Incurred but not reported claims					10,493	10,025
Total	$ 10,833	$ 23,910	$ 11,451	23,472	$ 11,451	$ 10,701
DOJ and HUD agreements increase in specific claims that were included in incurred but not reported claims in prior periods				$ 13,500