Residential Whole Loans-Carrying Value (Details) $ in Thousands	3 Months Ended	9 Months Ended
	Sep. 30, 2015 USD ($)	Sep. 30, 2015 USD ($)
Accounts, Notes, Loans and Financing Receivable [Line Items]
Contractual cash flows not expected to be collected (non-accretable yield)	$ (1,313)	$ (18,465)
Interest component of expected cash flows (accretable yield)	(3,620)	(34,901)
QTD
Accounts, Notes, Loans and Financing Receivable [Line Items]
Contractually required principal and interest	10,646	10,646
Expected cash flows to be collected	9,333	9,333
Fair value at the date of acquisition	5,713	5,713
YTD
Accounts, Notes, Loans and Financing Receivable [Line Items]
Contractually required principal and interest	103,950	103,950
Expected cash flows to be collected	85,485	85,485
Fair value at the date of acquisition	$ 50,584	$ 50,584