Other Assets (Real Estate Owned - Activity) (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2018	Mar. 31, 2017
Real Estate Owned [Roll Forward]
Balance at beginning of period	$ 152,356	$ 80,503
Adjustments to record at lower of cost or fair value	(3,415)	(1,823)
Transfer from residential whole loans	54,822	31,098
Purchases and capital improvements	2,678	774
Disposals	(23,501)	(11,844)
Balance at end of period	182,940	98,708
Gain recorded on transfer from residential whole loans to real estate owned	$ 6,400	$ 1,300