Fair Value of Measurement - Estimated Fair Values Of The Financial Instruments (Details) - USD ($) $ in Thousands	Dec. 31, 2017	Dec. 31, 2016
Financial assets:
Trading securities	$ 9,597	$ 7,857
Securities available-for-sale	515,121	498,897
Securities held-to-maturity	9,892	10,118
Carrying Value
Financial assets:
Cash and cash equivalents	57,839	96,085
Trading securities	9,597	7,857
Securities available-for-sale	515,121	498,897
Securities held-to-maturity	9,931	10,148
FHLB of New York stock, at cost	25,046	25,123
Net loans held-for-investment	3,114,659	2,943,489
Financial liabilities:
Deposits	2,836,979	2,713,587
Repurchase agreements and other borrowings	471,549	473,206
Advance payments by borrowers	14,798	12,331
Estimated Fair Value
Financial assets:
Cash and cash equivalents	57,839	96,085
Trading securities	9,597	7,857
Securities available-for-sale	515,121	498,897
Securities held-to-maturity	9,892	10,118
FHLB of New York stock, at cost	25,046	25,123
Net loans held-for-investment	3,157,829	2,970,438
Financial liabilities:
Deposits	2,839,666	2,720,176
Repurchase agreements and other borrowings	466,625	472,387
Advance payments by borrowers	14,798	12,331
Estimated Fair Value | Level 1
Financial assets:
Cash and cash equivalents	57,839	96,085
Trading securities	9,597	7,857
Securities available-for-sale	323	271
Securities held-to-maturity	0	0
FHLB of New York stock, at cost	0	0
Net loans held-for-investment	0	0
Financial liabilities:
Deposits	0	0
Repurchase agreements and other borrowings	0	0
Advance payments by borrowers	0	0
Estimated Fair Value | Level 2
Financial assets:
Cash and cash equivalents	0	0
Trading securities	0	0
Securities available-for-sale	514,798	498,626
Securities held-to-maturity	9,892	10,118
FHLB of New York stock, at cost	25,046	25,123
Net loans held-for-investment	0	0
Financial liabilities:
Deposits	2,839,666	2,720,176
Repurchase agreements and other borrowings	466,625	472,387
Advance payments by borrowers	14,798	12,331
Estimated Fair Value | Level 3
Financial assets:
Cash and cash equivalents	0	0
Trading securities	0	0
Securities available-for-sale	0	0
Securities held-to-maturity	0	0
FHLB of New York stock, at cost	0	0
Net loans held-for-investment	3,157,829	2,970,438
Financial liabilities:
Deposits	0	0
Repurchase agreements and other borrowings	0	0
Advance payments by borrowers	$ 0	$ 0