Investment Securities - Summary of Amortized Cost and Fair Value by Contractual Maturity (Details) - USD ($) $ in Thousands	Mar. 31, 2016	Dec. 31, 2015
Amortized Cost
Available-for-sale: Due in one year or less	$ 2,249
Available-for-sale: Due after one year through five years	17,887
Available-for-sale: Due after five years through ten years	2,871
Available-for-sale: Due after ten years	617
Mortgage-backed securities and collateralized mortgage obligations: Residential	268,889
Mortgage-backed securities and collateralized mortgage obligations: Commercial	3,951
Mutual funds	2,100
Available-for-sale, Amortized Cost	298,564	$ 317,375
Fair Value
Available-for-sale: Due in one year or less	2,276
Available-for-sale: Due after one year through five years	18,379
Available-for-sale: Due after five years through ten years	2,973
Available-for-sale: Due after ten years	611
Mortgage-backed securities and collateralized mortgage obligations: Residential	271,882
Mortgage-backed securities and collateralized mortgage obligations: Commercial	3,908
Mutual funds	2,122
Available-for-sale, Fair Value	302,151	318,159
Amortized Cost
Held-to-maturity: Due in one year or less	479
Held-to-maturity: Due after one year through five years	3,454
Held-to-maturity: Due after five years through ten years	11,390
Held-to-maturity: Due after ten years	28,107
Mortgage-backed securities and collateralized mortgage obligations: Residential	63,852
Mortgage-backed securities and collateralized mortgage obligations: Commercial	6,341
Held-to-maturity, Amortized Cost	113,623	116,792
Fair Value
Held-to-maturity: Due in one year or less	479
Held-to-maturity: Due after one year through five years	3,501
Held-to-maturity: Due after five years through ten years	11,691
Held-to-maturity: Due after ten years	28,690
Mortgage-backed securities and collateralized mortgage obligations: Residential	64,903
Mortgage-backed securities and collateralized mortgage obligations: Commercial	6,367
Held-to-maturity, Fair Value	$ 115,631	$ 117,698