Other Assets (Real Estate Owned - Activity) (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2018 USD ($) property	Dec. 31, 2017 USD ($) property	Dec. 31, 2016 USD ($)
Real Estate Owned [Roll Forward]
Balance at beginning of period	$ 152,356	$ 80,503
Adjustments to record at lower of cost or fair value	(15,929)	(11,018)
Transfer from residential whole loans	215,038	136,734	$ 91,896
Purchases and capital improvements	13,367	19,801
Disposals	(115,419)	(73,664)
Balance at end of period	$ 249,413	$ 152,356	$ 80,503
Number of properties | property	1,093	709
Gain recorded on transfer from residential whole loans to real estate owned	$ 19,600	$ 10,200