Real Estate Owned (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2015 USD ($) property		Dec. 31, 2014 USD ($) property		Dec. 31, 2013 USD ($)
Real Estate Properties [Line Items]
Number of real estate properties owned | property	182		46
Other Real Estate	$ 28,000
Number of real estate properties acquired during period | property	13		24
Number of real estate properties transferred during period | property	186		22
Real estate acquired through foreclosure	$ 26,100
Number of Real Estate Properties Sold During Period | property	63
Proceeds from Sale of Real Estate	$ 6,500
Gains (losses) on sales of other real estate	76
Real Estate Owned [Roll Forward]
Balance at beginning of period	5,492		$ 0
Adjustments to record at lower of cost or fair value	(3,475)		0
Transfer from residential whole loans (1)	30,104	[1]	2,904	[1]	$ 0
Purchases and capital improvements	2,461		2,588
Disposals	(6,556)		0
Balance at end of period	28,026		5,492		$ 0
Gain recorded on transfer from residential whole loans to real estate owned	1,700		331
Fair value
Real Estate Properties [Line Items]
Foreclosure in process	394,900
Real Estate Owned [Roll Forward]
Transfer from residential whole loans (1)	30,100		2,900
Carrying value
Real Estate Properties [Line Items]
Foreclosure in process	17,300
Residential Real Estate
Real Estate Owned [Roll Forward]
Purchases and capital improvements	$ 1,700		$ 2,600

[1] Includes net gain recorded on transfer of approximately $1.7 million and $331,000, respectively, for the years ended December 31, 2015 and 2014.