Exhibit 99.4

GREATER BAY BANCORP

Offer to Exchange Its

5.25% Senior Notes, Series B, Due March 31, 2008,

Which Have Been Registered Under the Securities Act (the “Exchange Notes”)

For

5.25% Senior Notes, Series A, Due March 31, 2008 (the “Original Notes”)

To Our Clients:

Enclosed for your consideration is a prospectus dated             , 2003 (as the same may be amended or supplemented from time to time, the “Prospectus”), and the related letter of transmittal (the “Letter of Transmittal”), relating to the offer (the “Exchange Offer”) of Greater Bay Bancorp (“Greater Bay”) to exchange 5.25% Exchange Notes for any and all outstanding 5.25% Original Notes, upon the terms and subject to the conditions described in the Prospectus. The Exchange Offer is being made in order to satisfy certain obligations of Greater Bay contained in the Registration Rights Agreement dated March 24, 2003, among Greater Bay and Keefe, Bruyette & Woods, Inc., Sandler O’Neill & Partners, L.P. and FTN Financial Securities Corp.

This material is being forwarded to you as the beneficial owner of the Original Notes carried by us in your account but not registered in your name. A TENDER OF SUCH ORIGINAL NOTES MAY ONLY BE MADE BY US AS THE HOLDER OF RECORD AND PURSUANT TO YOUR INSTRUCTIONS.

Accordingly, we request instructions as to whether you wish us to tender on your behalf the Original Notes held by us for your account, pursuant to the terms and conditions set forth in the enclosed Prospectus and Letter of Transmittal.

Your instructions should be forwarded to us as promptly as possible in order to permit us to tender the Original Notes on your behalf in accordance with the provisions of the Exchange Offer. The Exchange Offer shall expire at 5:00 p.m., Eastern time, on             , 2003, or on such later date or time to which Greater Bay may extend the Exchange Offer. Any Original Notes tendered pursuant to the Exchange Offer may be withdrawn at any time before the Expiration Date.

Your attention is directed to the following:

1. The Exchange Offer is for any and all Original Notes.

2. The Exchange Offer is subject to certain conditions set forth in the Prospectus in the section captioned “The Exchange Offer—Conditions to the Exchange Offer.”

3. Any transfer taxes incident to the transfer of Original Note from the holder to Greater Bay will be paid by Greater Bay, except as otherwise provided in Instruction 13 in the Letter of Transmittal.

4. The Exchange Offer expires at 5:00 p.m., Eastern time, on             , 2003, or on such later date or time to which Greater Bay may extend the Exchange Offer.

If you wish to have us tender your Original Notes, please so instruct us by completing, executing and returning to us the instruction form on the back of this letter. THE LETTER OF TRANSMITTAL IS FURNISHED TO YOU FOR INFORMATION ONLY AND MAY NOT BE USED DIRECTLY BY YOU TO TENDER ORIGINAL NOTES.

INSTRUCTIONS WITH RESPECT TO THE EXCHANGE OFFER

The undersigned acknowledge(s) receipt of your letter and the enclosed material referred to therein relating to the Exchange Offer made by Greater Bay with respect to its Original Notes.

This will instruct you to tender the Original Notes held by you for the account of the undersigned, upon and subject to the terms and conditions set forth in the Prospectus and the related Letter of Transmittal.

Please tender the Original Notes held by you for my account as indicated below:

Total Principal Amount at Maturity of Original Notes Tendered:*                                                                                                                                       

¨ Please do not tender any Original Notes held by you for my account.

Dated:             , 2003

Authorized Signature(s):                                                                                                                                                                                                                 

Please print name(s) here:                                                                                                                                                                                                               

Address(es):                                                                                                                                                                                                                                        

Area Code and Telephone Number:                                                                                                                                                                                           

Taxpayer Identification or Social Security Number(s):                                                                                                                                                        

None of the Original Notes held by us for your account will be tendered unless we receive written instructions from you to do so. Unless a specific contrary instruction is given in the space provided, your signature(s) hereon shall constitute an instruction to us to tender all the Original Notes held by us for your account.

* Must be in denominations of $1,000. However, if the Original Notes are represented by physical certificates, each physical certificate must be in denominations of $100,000 in principal amount or any integral of $1,000 in excess of $100,000, provided that if any Original Notes represented by physical certificates are tendered for exchange in part, the untendered principal amount thereof must be $100,000 or any integral multiple of $1,000 in excess thereof.

2