Summary of Significant Accounting Policies Investment in WMS Series LLC (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2015	Dec. 31, 2014	Dec. 31, 2013
Schedule of Equity Method Investments [Line Items]
Purchase of single family mortgage loans from WMS	$ 616,900	$ 491,300	$ 695,700
Commitments and Contingencies
Windermere Mortgage Services Series LLC [Member]
Schedule of Equity Method Investments [Line Items]
Income from equity method investments	$ 2,500	$ 1,300	1,700
Equity method investment in WMS	2,900	3,100
Contracted service income (loss)	(960)	(1,205)	$ (951)
Commitments and Contingencies	25,000
Outstanding balance of secured line of credit	8,600	7,100
Highest balance of loans to borrower	$ 13,400	$ 12,400