Residential Mortgage Securities and MSR Related Assets (Credit Loss Component of OTTI) (Details) - USD ($) $ in Thousands	12 Months Ended
Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Other than Temporary Impairment, Credit Losses Recognized in Earnings [Roll Forward]
Credit loss component of OTTI at beginning of period	$ 39,596	$ 38,337	$ 37,305
Additions for credit related OTTI not previously recognized	180	1,259	63
Subsequent additional credit related OTTI recorded	0	0	969
Credit loss component of OTTI at end of period	$ 39,776	$ 39,596	$ 38,337

This item represents an increase to the cumulative amount of credit losses recognized in earnings for an other than temporary impairment (OTTI) of a debt security held for which a previous OTTI was recognized and the investor does not intend to sell the debt security and it is not more likely than not that the investor will be required to sell the debt security before recovery of its amortized cost basis.

Name:	us-gaap_OtherThanTemporaryImpairmentCreditLossesRecognizedInEarningsAdditionsAdditionalCreditLosses
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	duration

- Definition

This item represents an increase to the cumulative amount of credit losses recognized in earnings for an other than temporary impairment (OTTI) of a debt security held for which no such other than temporary impairment (OTTI) was previously recognized.

+ References

+ Details

Name:	us-gaap_OtherThanTemporaryImpairmentCreditLossesRecognizedInEarningsAdditionsNoPreviousImpairment
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	duration

- Definition

The cumulative amount of credit losses recognized in earnings related to debt securities held for which a portion of an other than temporary impairment (OTTI) was recognized in other comprehensive income (a component of shareholders' equity).

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 8B
-Subparagraph (a)
-URI http://asc.fasb.org/extlink&oid=118939692&loc=SL6284393-111563

Reference 2: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 50
-Paragraph 8B
-Subparagraph (g)
-URI http://asc.fasb.org/extlink&oid=118939692&loc=SL6284393-111563

+ Details

Name:	us-gaap_OtherThanTemporaryImpairmentCreditLossesRecognizedInEarningsCreditLossesOnDebtSecuritiesHeld
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

A roll forward is a reconciliation of a concept from the beginning of a period to the end of a period.

+ References

No definition available.

+ Details

Name:	us-gaap_OtherThanTemporaryImpairmentCreditLossesRecognizedInEarningsRollForward
Namespace Prefix:	us-gaap_
Data Type:	xbrli:stringItemType
Balance Type:	na
Period Type:	duration