Other Assets (Real Estate Owned - Activity) (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2018	Sep. 30, 2017	Sep. 30, 2018	Sep. 30, 2017
Real Estate Owned [Roll Forward]
Balance at beginning of period	$ 192,162	$ 104,443	$ 152,356	$ 80,503
Adjustments to record at lower of cost or fair value	(4,056)	(3,129)	(11,592)	(7,306)
Transfer from residential whole loans	58,051	38,944	161,572	97,388
Purchases and capital improvements	4,897	15,342	10,179	17,224
Disposals	(27,949)	(17,621)	(89,410)	(49,830)
Balance at end of period	223,105	137,979	223,105	137,979
Gain recorded on transfer from residential whole loans to real estate owned	$ 5,800	$ 2,800	$ 17,000	$ 5,300