EXHIBIT 10.4


                                AMENDMENT TO THE
                      AMERICAN MEDICAL SECURITY GROUP, INC.
                    CHANGE OF CONTROL SEVERANCE BENEFIT PLAN
                   (AS AMENDED AND RESTATED NOVEMBER 29, 2001)

         This Amendment to the American Medical Security Group, Inc. Change of
Control Severance Benefit Plan (As Amended and Restated November 29, 2001) (the
"Plan") is made and effective the 15th day of September, 2004, subject to, with
respect to each participant, the participant's consent to the amendment.

                                   WITNESSETH:

         WHEREAS, American Medical Security Group, Inc. ("AMSG") desires to
amend the Plan as set forth below and the Board of Directors of AMSG has
authorized amending the Plan as set forth below.

         WHEREAS, the Company and the Executives mutually agree to amend the
Plan as set forth below.

         NOW THEREFORE, in consideration for the following promises and for
other good and valuable consideration, the receipt and sufficiency of which is
hereby acknowledged, the parties agree to amend the Plan as follows:

         1. Article VII. "Amendment and Termination," is amended to change the
reference to the one year period therein to a two year period and is hereby
amended and restated in its entirety to read as follows:

                         VII. AMENDMENT AND TERMINATION

         The Plan may be modified, amended, changed, or terminated only upon the
approval of the Company's Board of Directors during a regular, annual or special
meeting; provided, however, that any modification, amendment, change or
termination of the Plan that has an adverse effect on any Executive who is a
participant in the Plan on the date such modification, amendment, change or
termination is adopted shall not be effective until two (2) years after the date
of adoption. Any modification, amendment, change, or termination of the Plan
shall be in writing and signed by at least two officers of the Company at the
level of executive vice president and above.

2. Section 3.4 is amended to provide that the benefits under the Plan will apply
only to the first Change of Control following the adoption of this amendment and
is hereby amended and restated in its entirety to read as follows:

3.4 CONTROL CHANGE DATE. The "Control Change Date" means the first date on which
an event described in Section 3.3 occurs following the effective date of this
Plan (as provided in Article II herein). If a Change of Control occurs on
account of a series of



transactions, the Control Change Date is the date of occurrence of the last of
such transactions.

3. Section 4.4 is amended to provide that the Company shall satisfy obligations
under Section 4.4 by providing individual insurance policies and is hereby
amended and restated in its entirety to read as follows:

4.4 INSURANCE COVERAGE. In addition to the Plan Benefits, the Company shall
provide an Executive with individual health, dental, long-term disability, and
life insurance policies (one or more of which may be furnished directly by the
Company) which shall provide the Executive and his dependents with substantially
equivalent insurance coverage to each coverage that was in place at the time of
the Qualifying Separation for a period of three (3) years immediately following
the Qualifying Separation with respect to a Senior Executive and for a period of
one and one-half (1.5) years immediately following the Qualifying Separation
with respect to an Other Executive. Thereafter, the Company will provide an
Executive with an explanation of any right to continue such coverage under COBRA
and any other applicable state or federal law.

4. Section 4.6 is hereby amended and restated in its entirety to read as
follows:

4.6 EXCISE TAXES. (a) If the excise tax imposed under the Internal Revenue Code
of 1986, as amended, (the "Code") Section 4999 on "excess parachute payments",
as defined in the Code Section 280G, is incurred on account of (i) any amount
paid or payable to or for the benefit of an Executive under this Section as
legal fees and expenses, or (ii) any payments or benefits which an Executive
receives or has the right to receive under this Plan or any other plan or
compensation arrangement of the Company, any entity whose actions result in a
Change of Control or an affiliate of the Company or any such entity (the "Change
of Control Benefits"), the Company shall indemnify the Executive and hold him or
her harmless against all claims, losses, damages, penalties, expenses, and
excise taxes. To effect this indemnification, the Company shall pay the
Executive the Additional Amount on the date the Executive becomes entitled to
receive any payment or benefit pursuant to clause (i) or (ii) above that is
subject to the excise tax imposed under Section 4999 of the Code (or such
successor provision thereto) (the "Excise Tax"). For purposes of the Plan, the
"Additional Amount" shall mean the amount necessary to indemnify and hold the
Executive harmless from (A) the Excise Tax with respect to the Change of Control
Benefits and (B) the amount required to satisfy (y) the additional Excise Tax,
and (z) the federal, state and local income taxes for which an Executive is
liable with respect to the Additional Amount (the sum of items (A) and (B) of
this Section 4.6 being hereunder referred to as the "Additional Tax Liability").

(b) For purposes of determining the amount and timing of the payments of the
Additional Amount, (i) all Change of Control Benefits shall be treated as
"parachute payments" (within the meaning of Section 280G(b)(2) of the Code)
unless, in the opinion of the Company's independent auditors or other qualified
professional engaged by the Company (the "Auditor"), such payments or benefits
(in whole or in part) do not constitute parachute payments including by reason
of Section 280G(b)(4)(A) of the Code,

                                       2



(ii) all "excess parachute payments" within the meaning of Section 280G(b)(1) of
the Code shall be treated as subject to the Excise Tax unless, in the opinion of
the Auditor, such excess parachute payments (in whole or in part) represent
reasonable compensation for services actually rendered (within the meaning of
Section 280G(b)(4)(B) of the Code) in excess of the base amount (within the
meaning of Section 280G(b)(3) of the Code) allocable to such reasonable
compensation, or are otherwise not subject to the Excise Tax, and (iii) the
value of any noncash benefits or any deferred payment or benefit shall be
determined by the Auditor in accordance with the principles of Sections
280G(d)(3) and (4) of the Code. For purposes of determining the amount of the
Additional Amount, the Executive shall be deemed to pay federal income tax at
the highest marginal rate of federal income taxation and state and local income
taxes at the highest marginal rate of taxation in the state and locality of the
Executive's residence, net of the maximum reduction in federal income taxes
which could be obtained from deduction of such state and local taxes. The
Auditor shall address any opinion required hereunder to the Executive and such
opinion shall state that the Executive is entitled to rely on such opinion
without risk of any penalty.

(c) If the Executive's Additional Tax Liability is subsequently determined to be
less than the amount of the Additional Amount paid to the Executive, the
Executive shall repay to the Company that portion of the Additional Amount
payment attributable to such reduction (plus interest on the amount of such
repayment at the rate provided in Section 1274(b)(2)(B) of the Code). If the
Executive's Additional Tax Liability is subsequently determined to be more than
the amount of the Additional Amount paid to the Executive, the Company shall
make an additional payment in respect of such excess, as well as the amount of
any penalty or interest assessed with respect thereto at the time that the
amount of such excess, penalty or interest is finally determined.

(d) Notwithstanding any other provision of this Section 4.6 to the contrary, if
the aggregate "After-Tax Amount" (as defined below) of the Change of Control
Benefits that would be payable to Employee does not equal or exceed 110% of the
"After-Tax Floor Amount" (as defined below), then no Additional Amount shall be
payable to Employee and the aggregate amount of Change of Control Benefits
payable to Employee shall be reduced (but not below the "Floor Amount" as
defined below) to the largest amount that would both (i) not cause any
Additional Tax Liability to be payable by Employee and (ii) not cause any Change
of Control Benefits to become nondeductible by the Company by reason of Section
280G of the Code (or any successor provision). For purposes of the preceding
sentence, Employee shall be deemed to be subject to the highest marginal rate of
federal, state, and local taxes, excluding Social Security, Medicare, and
alternative minimum taxes or similar tax consequences.

         "After-Tax Amount" means the portion of a specified amount that would
remain after payment of all federal, state, and local taxes (excluding Social
Security, Medicare, and alternative minimum taxes or similar tax consequences),
and Additional Tax Liability paid or payable by Employee in respect of such
specified amount.

         "After-Tax Floor Amount" means the After-Tax Amount of the Floor
Amount.

                                       3



         "Floor Amount" means the greatest pre-tax amount of Change of Control
Benefits that could be paid to Employee without causing Employee to become
liable for any Additional Tax Liability in connection therewith.

5. A new Section 4.9 is hereby added to the Plan to provide for non-duplication
of benefits under the Plan and shall read as follows:

4.9 NON-DUPLICATION OF BENEFITS. No Executive is eligible to receive benefits
under this Plan more than one time.


                                     * * * *

This Amendment may be executed in one or more counterparts, each of which
together shall be deemed an original, but all of which together shall constitute
one and the same instrument.

Approved by the Board of Directors this 15th day of September, 2004.

Required Officer Signatures

/s/ Timothy J. Moore                            /s/ John R. Lombardi
Timothy J. Moore

/s/ Samuel V. Miller
Samuel V. Miller


Required Executive Signature

/s/ Clifford A. Bowers
Executive       Clifford A. Bowers

/s/ John R. Lombardi
Executive       John R. Lombardi

/s/ James C. Modaff
Executive       James C. Modaff

/s/ Timothy J. Moore
Executive       Timothy J. Moore

/s/ John R. Wirch
Executive       John R. Wirch

/s/ Thomas G. Zielinski
Executive       Thomas G. Zielinski

                                       4