Investment Securities - Summary of Amortized Cost and Fair Value by Contractual Maturity (Details) - USD ($) $ in Thousands	Jun. 30, 2016	Dec. 31, 2015
Amortized Cost
Available-for-sale: Due in one year or less	$ 3,333
Available-for-sale: Due after one year through five years	14,755
Available-for-sale: Due after five years through ten years	2,865
Available-for-sale: Due after ten years	3,273
Mortgage-backed securities and collateralized mortgage obligations: Residential	275,930
Mortgage-backed securities and collateralized mortgage obligations: Commercial	3,858
Mutual funds	2,100
Available-for-sale, Amortized Cost	312,614	$ 317,375
Fair Value
Available-for-sale: Due in one year or less	3,384
Available-for-sale: Due after one year through five years	15,115
Available-for-sale: Due after five years through ten years	3,016
Available-for-sale: Due after ten years	3,287
Mortgage-backed securities and collateralized mortgage obligations: Residential	280,973
Mortgage-backed securities and collateralized mortgage obligations: Commercial	3,829
Mutual funds	2,135
Available-for-sale, Fair Value	318,239	318,159
Available for Sale Securities, Debt Maturities, Corporate Securities Amortized Cost	6,500
Available for Sale Securities, Debt Maturities, Corporate securities Fair Value	6,500
Amortized Cost
Held-to-maturity: Due in one year or less	2,475
Held-to-maturity: Due after one year through five years	5,374
Held-to-maturity: Due after five years through ten years	9,419
Held-to-maturity: Due after ten years	25,964
Mortgage-backed securities and collateralized mortgage obligations: Residential	60,243
Mortgage-backed securities and collateralized mortgage obligations: Commercial	5,945
Held-to-maturity, Amortized Cost	109,420	116,792
Fair Value
Held-to-maturity: Due in one year or less	2,477
Held-to-maturity: Due after one year through five years	5,507
Held-to-maturity: Due after five years through ten years	9,770
Held-to-maturity: Due after ten years	26,901
Mortgage-backed securities and collateralized mortgage obligations: Residential	61,622
Mortgage-backed securities and collateralized mortgage obligations: Commercial	5,996
Held-to-maturity, Fair Value	$ 112,273	$ 117,698