Financing Arrangements, Portfolio Investments - Narrative (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2018 USD ($) counterparty	Dec. 31, 2017 USD ($) counterparty	Dec. 31, 2016 USD ($)
Accounts, Notes, Loans and Financing Receivable [Line Items]
Repurchase agreement, average days to maturity	62 days	44 days
Accrued interest payable	$ 3,900	$ 2,500
Advance rate (as a percent)	87.70%
Average haircut (as a percent)	12.30%
Repurchase agreement, number of counterparties | counterparty	11	10
Cash and cash equivalents	$ 103,724	$ 95,191	$ 83,554
Unencumbered investment securities	$ 262,500
Liquidation proceeds (as a percent)	23.70%
Stockholders' Equity, Total | Lender Concentration Risk
Accounts, Notes, Loans and Financing Receivable [Line Items]
Concentration risk, percentage	5.04%	5.04%
CMBS
Accounts, Notes, Loans and Financing Receivable [Line Items]
Average haircut (as a percent)	23.00%
Unencumbered investment securities	$ 107,000
Agency RMBS
Accounts, Notes, Loans and Financing Receivable [Line Items]
Average haircut (as a percent)	5.00%
Unencumbered investment securities	$ 59,400
Non-Agency RMBS
Accounts, Notes, Loans and Financing Receivable [Line Items]
Average haircut (as a percent)	25.00%
Unencumbered investment securities	$ 96,100
Loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Repurchase agreement, outstanding balance	$ 1,500,000	$ 1,300,000
Weighted average interest rate (as a percent)	3.41%	2.18%