Interest Receivable (Details) - USD ($) $ in Thousands	Jun. 30, 2016	Dec. 31, 2015
Interest receivable by investment category
Total interest receivable	$ 28,034	$ 29,002
Total MBS interest receivable
Interest receivable by investment category
Total interest receivable	24,721	26,629
Fannie Mae
Interest receivable by investment category
Total interest receivable	8,266	8,999
Freddie Mac
Interest receivable by investment category
Total interest receivable	2,007	2,177
Ginnie Mae
Interest receivable by investment category
Total interest receivable	15	15
Non-Agency MBS
Interest receivable by investment category
Total interest receivable	14,433	15,438
Residential whole loans
Interest receivable by investment category
Total interest receivable	3,131	2,259
CRT securities
Interest receivable by investment category
Total interest receivable	140	92
Money market and other investments
Interest receivable by investment category
Total interest receivable	$ 42	$ 22