Other Assets (Real Estate Owned - Activity) (Details) - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2018	Jun. 30, 2017	Jun. 30, 2018	Jun. 30, 2017
Real Estate Owned [Roll Forward]
Balance at beginning of period	$ 182,940	$ 98,708	$ 152,356	$ 80,503
Adjustments to record at lower of cost or fair value	(4,121)	(2,354)	(7,536)	(4,177)
Transfer from residential whole loans	48,699	27,345	103,521	58,444
Purchases and capital improvements	2,604	1,109	5,282	1,882
Disposals	(37,960)	(20,365)	(61,461)	(32,209)
Balance at end of period	192,162	104,443	192,162	104,443
Gain recorded on transfer from residential whole loans to real estate owned	$ 5,300	$ 1,100	$ 11,700	$ 2,500