Exhibit 23.0

Consent of Independent Registered Public Accounting Firm

We hereby consent to the incorporation by reference in the Registration Statements on Form S-8 (Nos.333-175544 and 333-138651) and Registration Statement on Form S-3 (No.333-156727) of our reports dated March 9, 2012, relating to Intervest Bancshares Corporation and Subsidiaries’ consolidated financial statements and its internal control over financial reporting, included in this Annual Report on Form 10-K for the year ended December 31, 2011.

/s/ Hacker, Johnson & Smith, P.A., P.C.

Hacker, Johnson & Smith, P.A., P.C.

Tampa, Florida

March 9, 2012