Borrowings - Schedule of Borrowings (Detail) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2016	Dec. 31, 2015
Repurchase Agreements
Amount Outstanding	$ 11,160,669	$ 12,126,048
Weighted Average Interest Rate (Percent)	1.23%	0.96%
Weighted Average Remaining Maturity (Days)	30 days	25 days
Secured Loans:
Secured loans, amount outstanding	$ 1,650,000	$ 1,650,000
Secured loans, weighted average remaining maturity (days)	2682 days	2937 days
Total Borrowings
Exchangeable senior notes, carrying value	$ 397,041	$ 394,573
Agency RMBS
Repurchase Agreements
Amount Outstanding	$ 8,148,220	$ 8,389,643
Weighted Average Interest Rate (Percent)	0.93%	0.65%
Weighted Average Remaining Maturity (Days)	32 days	24 days
Non-Agency RMBS
Repurchase Agreements
Amount Outstanding	$ 1,519,859	$ 2,077,240
Weighted Average Interest Rate (Percent)	2.06%	1.68%
Weighted Average Remaining Maturity (Days)	28 days	32 days
GSE CRT
Repurchase Agreements
Amount Outstanding	$ 547,872	$ 488,275
Weighted Average Interest Rate (Percent)	2.25%	1.91%
Weighted Average Remaining Maturity (Days)	16 days	19 days
CMBS
Repurchase Agreements
Amount Outstanding	$ 944,718	$ 1,170,890
Weighted Average Interest Rate (Percent)	1.86%	1.49%
Weighted Average Remaining Maturity (Days)	16 days	23 days
Secured Loans
Secured Loans:
Secured loans, weighted average interest rate	0.74%	0.55%
Exchangeable Senior Notes
Exchangeable Senior Notes
Exchangeable senior notes, amount outstanding	$ 400,000	$ 400,000
Exchangeable senior notes, weighted average interest rate	5.00%	5.00%
Exchangeable senior notes, weighted average remaining maturity (days)	439 days	805 days
Total Borrowings
Debt issuance costs	$ 3,000	$ 5,400
Secured Debt, Excluding Asset-backed Securities
Total Borrowings
Total borrowings, amount outstanding	$ 13,210,669	$ 14,176,048
Total borrowings, weighted average interest rate (Percent)	1.28%	1.02%
Total borrowings, weighted average remaining maturity (Days)	373 days	386 days