EXHIBIT 10.7

INTELLECTUAL PROPERTY SECURITY AGREEMENT

THIS INTELLECTUAL PROPERTY SECURITY AGREEMENT (this “Security Agreement”) dated as of November 3, 2010, is made by PHOENIX FOOTWEAR GROUP, INC., a Delaware corporation (the “Borrower”), in favor of GIBRALTAR BUSINESS CAPITAL, LLC, a Delaware limited liability company (together with its successors and assigns, “Agent”) for the benefit of the Lenders.

RECITALS

WHEREAS, Borrower, the Agent and the lenders parties thereto (the “Lenders”) are parties to that certain Loan and Security Agreement dated as of even date herewith (as the same may be amended, supplemented or modified from time to time, the “Loan Agreement”), which provides (i) for Lenders to make certain loans to Borrower, and (ii) for the grant by Borrower to Agent for the benefit of the Lenders of a security interest in the Borrower’s assets, including, without limitation, patents, its trademarks, trademark applications and registrations, trade names, copyrights and copyright registrations, service marks, service mark applications, all goodwill relating thereto and licenses, and all proceeds thereof.

NOW, THEREFORE, in consideration of the premises set forth herein and for other good and valuable consideration, the receipt, adequacy and sufficiency of which are hereby acknowledged, the Borrower (intending to be legally bound) hereby agrees as follows:

1. Incorporation of Loan Agreement. The Loan Agreement and the terms and provisions thereof are hereby incorporated herein in their entirety by this reference thereto. All terms capitalized but not otherwise defined herein shall have the same meanings herein as in the Loan Agreement.

2. Security Interest in Intellectual Property. To secure the complete satisfaction and payment and performance when due (or declared due in accordance with the terms of the Loan Agreement) of the “Liabilities”, as defined in the Loan Agreement (collectively, the “Obligations”), the Borrower hereby grants to Agent for the benefit of the Lenders a continuing security interest in and to any and all of the Borrower’s right, title and interest in and to all of the following now owned and existing and hereafter arising, created or acquired property (collectively, the “Intellectual Property”):

(i) trademarks, trademark registrations, trademark applications, trade names and tradestyles, brand names, service marks, service mark registrations and service mark applications, including, without limitation, the trademarks, trade names, brand names, service marks and applications and registrations thereof listed on Exhibit A attached hereto and hereby made a part hereof, and (a) all renewals or extensions thereof, (b) all income, royalties, proceeds, damages and payments now and hereafter due or payable with respect thereto, including, without limitation, damages and payments for past or future infringements thereof, (c) the right to sue for past, present and future infringements thereof, and (d) all rights corresponding thereto throughout the world, if any (all of the foregoing trademarks, trade names and tradestyles, brand names, service marks and applications and registrations thereof, together with the items described in clauses (a)-(d) of this subsection 2(i), are sometimes hereinafter referred to individually as a “Trademark” and, collectively, as the “Trademarks”);

(ii) license agreements (to the extent such license agreements may be assigned without violating the terms of any such license agreement) with respect to any of the Intellectual Property or any other patent, trademark, service mark or any application or registration thereof or any other trade name or tradestyle between Borrower and any other party, whether Borrower is a licensor or licensee under any such license agreement (all of the foregoing license agreements and Borrower’s rights thereunder are referred to collectively as the “Licenses”);

(iii) the goodwill of Borrower’s business connected with and symbolized by the Trademarks;

(iv) copyrights, copyright registrations and copyright applications, used in the United States, including, without limitation, namely the copyright registrations and copyright applications listed on Exhibit B attached hereto and made a part hereof, and (a) renewals or extensions thereof, (b) all income, royalties, proceeds, damages and payments now and hereafter due and/or payable with respect thereto, including, without limitation, damages and payments for past or future infringements thereof, (c) the right to sue for past, present and future infringements thereof, and (d) all rights corresponding thereto throughout the world (all of the foregoing copyrights, copyright registrations and copyright applications, together with the items described in clauses (a)-(d), are sometimes hereinafter individually and/or collectively referred to as the “Copyrights”);

(v) all trade secrets, formulas, processes, devices, know-how, or compilations of information (including technical information and non-technical information such as customer lists and marketing plans), collectively referred to as trade secrets, which are not available to others and which are maintained as confidential by Borrower, and the right to prevent misappropriation and unauthorized disclosures thereof and all rights corresponding thereto throughout the world (all of the foregoing trade secrets and associated rights are sometimes hereinafter individually and/or collectively referred to as the “Trade Secrets”); and

(vi) patents and patent applications, including, without limitation, the inventions and improvements described and claimed therein, and those patents listed on Exhibit C attached hereto and hereby made a part hereof, and (a) all reissues, divisions, continuations, renewals, extensions and continuations-in-part thereof, (b) all income, royalties, damages, proceeds and payments now and hereafter due or payable under or with respect thereto, including, without limitation, damages and payments for past or future infringements thereof, (c) the right to sue for past, present and future infringements thereof, and (d) all rights corresponding thereto throughout the world (all of the foregoing patents and applications, together with the items described in clauses (a)-(d) of this subsection 2(i), are sometimes hereinafter referred to individually as a “Patent” and, collectively, as the “Patents”).

2

3. Representations and Warranties. The Borrower hereby represents and warrants to Agent, which representations and warranties shall survive the execution and delivery of this Security Agreement, that as of the date hereof:

(i) None of the Intellectual Property owned by Borrower that is United States Intellectual Property and material to Borrower’s business has been adjudged invalid or unenforceable nor has any such Intellectual Property been cancelled, in whole or in part, and each such material Intellectual Property is presently subsisting;

(ii) To Borrower’s knowledge, each of the Intellectual Property owned by Borrower and material to Borrower’s business is valid and enforceable;

(iii) Borrower is the sole and exclusive owner of the entire and unencumbered right, title and interest in and to the Intellectual Property that is scheduled in Exhibits A and B hereto, free and clear of any liens, security interests, mortgages, charges and encumbrances, (including, without limitation, licenses other than non-exclusive licenses which may be granted in the ordinary course of business, consent-to-use agreements, shop rights and covenants by Borrower not to sue third persons) other than a security interest granted in favor of Agent;

(iv) Borrower has adopted, used and is currently using all of the Trademarks and Patents owned by Borrower that are material to Borrower’s business;

(v) Borrower has no knowledge of any suits or actions commenced or threatened within the last three years with reference to or in connection with any of the Intellectual Property;

(vi) No trademark opposition or cancellation proceedings have been filed with the United States Patent and Trademark Office against any of the Trademarks owned by Borrower;

(vii) The Licenses are valid and binding agreements, enforceable in accordance with their terms. Each of the Licenses is in full force and effect and to Borrower’s knowelde there is no material default under any of the Licenses;

(viii) To Borrower’s knowledge, none of the Intellectual Property owned by Borrower infringes upon the rights or property of any other person or entity or is currently being challenged in any way, and there are no pending or threatened claims, litigation, proceedings or other investigations regarding any such Intellectual Property; and

(ix) This Security Agreement has been executed and delivered on behalf of the Borrower by a duly authorized officer of such Borrower.

4. Restrictions on Future Agreements. The Borrower agrees that until all Obligations shall have been satisfied and indefeasibly paid in full and the Loan Agreement shall have been terminated in accordance with its terms, Borrower shall not, without the prior written consent of Agent, sell, transfer, mortgage, convey, dispose, encumber or assign any or all of, or grant an irrevocable license or sublicense in perpetuity under, the Intellectual Property, or enter into any other agreement with respect to the Intellectual Property (other than licenses in the ordinary course of business) that materially adversely affects the rights of Agent in and to the Intellectual Property, or take any action or permit any action to be taken by others subject to its control, including, without limitation, licensees or sublicensees, or fail to take any action, in each of the foregoing cases which would materially adversely affect the validity or enforcement of the rights provided or transferred to Agent under this Security Agreement.

3

5. New Intellectual Property. The Borrower hereby represents and warrants to Agent that the Intellectual Property listed on Exhibits A, B and C respectively, constitute all of the registered United States Trademarks and Copyrights, as applicable, now owned by Borrower and material to Borrower’s business. If, before all Obligations shall have been satisfied in full or before the Loan Agreement has been terminated in accordance with its terms, Borrower shall obtain rights to any new Intellectual Property, the provisions of this Security Agreement above shall automatically apply thereto and, with respect to any US Trademark and Copyright registrations and applications Borrower shall give to Agent prompt written notice thereof. Borrower hereby authorizes Agent to modify this Security Agreement by amending Exhibits A, B and C, as applicable, to include any such Intellectual Property, and to file or refile this Security Agreement with the U.S. Patent and Trademark Office and U.S. Copyright Office or Library of Congress (at Borrower’s sole cost and expense).

6. Royalties; Terms; Rights Upon Default. The term of this Security Agreement shall extend until the earlier of (i) the expiration of all of the respective Intellectual Property subject to the grant of security interest hereunder, and (ii) the indefeasible payment in full of all Obligations and the termination of the Loan Agreement. The Borrower agrees that upon the occurrence and during the continuance of an Event of Default, the use by Agent shall have a non-exclusive license and right to use all Intellectual Property to the same extent as the Borrower uses such Intellectual Property, and without any liability for royalties or other related charges from Agent to any Borrower. Upon the occurrence and during the continuance of any Event of Default, Borrower hereby authorizes: (a) the Register of Copyrights, United States Copyright Office, to issue any and all certificates of registration or renewal for all of the Copyrights to Agent as assignee of Borrower’s entire interest therein; and (b) the Commissioner of Patents and Trademarks, United States Patent and Trademark Office to issue any and all certificates of registration or renewal for all of the Trademarks to Agent as assignee of Borrower’s entire interest therein and in the goodwill of Borrower’s business connected therewith and symbolized thereby.

7. Effect on Loan Agreement. The Borrower acknowledges and agrees that this Security Agreement is not intended to limit or restrict in any way the rights and remedies of Agent under the Loan Agreement but rather is intended to facilitate the exercise of such rights and remedies.

8. Release of Security Agreement. Upon the payment and performance in full of the Obligations, and the full and final termination of any commitment to extend any loan or financial accommodations under the Loan Agreement, this Security Agreement shall terminate, and Agent shall execute and deliver any document reasonably requested by Borrower, at Borrower’s sole cost and expense, as shall be necessary to evidence termination of the security interest granted by Borrower to Agent hereunder.

9. Duties of Borrower. Except as otherwise permitted by the Loan Agreement, the Borrower shall have the duty to the extent commercially reasonable and in Borrower’s good faith business judgment: (i) to file and prosecute diligently any trademark or service mark applications pending as of the date hereof or hereafter until all Obligations shall have been paid in full and the Loan Agreement has been terminated, (ii) to make application on trademarks and service marks, (iii) to preserve and maintain all rights in the Intellectual Property

4

owned by Borrower (including, but not limited to, with respect to Trademarks, the filing of affidavits of use and, incontestability, where applicable, under §§8 and 15 of the Lanham Act (15 U.S.C. § 1058, 1065) and renewals and, to the extent commercially reasonable, initiating opposition or cancellation proceedings or litigation against users of the same or confusingly similar marks who seriously threaten the validity or rights of Borrower in its Trademarks), and (iv) to ensure that the Intellectual Property owned by Borrower is and remains enforceable. Any and all costs and expenses incurred in connection with Borrower’s obligations under this Section 9 shall be borne by Borrower.

10. Agent’s Right to Sue. Upon the occurrence and during the continuance of an Event of Default, Agent shall have the right, but shall in no way be obligated, to bring suit in its own name to enforce the Intellectual Property, and, if Agent shall commence any such suit, Borrower shall, at the request of Agent, do any and all lawful acts and execute any and all proper documents and instruments reasonably required by Agent in aid of such enforcement.

11. Waivers. No course of dealing between or among Borrower and Agent, nor any failure to exercise, nor any delay in exercising, on the part of Agent, any right, power or privilege hereunder or under the Loan Agreement shall operate as a waiver thereof; nor shall any single or partial exercise of any right, power or privilege hereunder or thereunder preclude any other or further exercise thereof or the exercise of any other right, power or privilege.

12. Severability. The provisions of this Security Agreement are severable, and if any clause or provision shall be held invalid and unenforceable in whole or in part in any jurisdiction, then such invalidity or unenforceability shall affect only such clause or provision, or part thereof, in such jurisdiction, and shall not in any manner affect such clause or provision in any other jurisdiction, or any other clause or provision of this Security Agreement in any jurisdiction.

13. Modification. This Security Agreement cannot be altered, amended or modified in any way, except as specifically provided in Section 5 hereof or by a writing signed by Borrower and Agent.

14. Cumulative Remedies; Power of Attorney. All of Agent’s rights and remedies with respect to the Intellectual Property, whether established hereby or by the Loan Agreement, or by any other agreements or by law shall be cumulative and may be exercised singularly or concurrently. The Borrower hereby authorizes Agent upon the occurrence and during the continuance of an Event of Default, to make, constitute and appoint any officer or agent of Agent as Agent may select, in its sole discretion, as Borrower’s true and lawful attorney-in-fact, with power to, upon written notice to Borrower, (i) endorse Borrower’s name on all applications, documents, papers and instruments necessary or desirable for Agent in the use of the Intellectual Property, or (ii) take any other actions with respect to the Intellectual Property as Agent deems to be in the best interest of Agent, or (iii) grant or issue any exclusive or non-exclusive license under the Intellectual Property to any person or entity, or (iv) assign, pledge, sell, convey or otherwise transfer title in or dispose of any of the Intellectual Property to any person or entity. Borrower hereby ratifies all that such attorney shall lawfully do or cause to be done by virtue hereof. This power of attorney being coupled with an interest shall be irrevocable until all Obligations shall have been paid in full and the Loan Agreement has been terminated.

5

15. Binding Effect; Benefits. This Security Agreement shall be binding upon Borrower and its respective successors and permitted assigns, and shall inure to the benefit of Agent and its successors, nominees and assigns; provided, however, Borrower shall not assign this Security Agreement or any of Borrower’s obligations hereunder without the prior written consent of Agent.

16. Headings; Counterparts. Paragraph headings used herein are for convenience only and shall not modify the provisions which they precede. This Security Agreement may be signed in one or more counterparts, but all of such counterparts shall constitute and be deemed to be one and the same instrument. A signature hereto sent or delivered by facsimile or other electronic transmission shall be as legally binding and enforceable as a signed original for all purposes.

17. Further Assurances. The Borrower agrees to execute and deliver such further agreements, instruments and documents, and to perform such further acts, as Agent shall reasonably request from time to time in order to carry out the purpose of this Security Agreement and agreements set forth herein. Borrower acknowledges that a copy of this Security Agreement will be filed by Agent with the United States Patent and Trademark Office and, if applicable, the United States Copyright Office or Library of Congress, at the sole cost and expense of Borrower.

18. Survival of Representations. All representations and warranties of Borrower contained in this Security Agreement shall survive the execution and delivery of this Security Agreement.

19. GOVERNING LAW. THIS SECURITY AGREEMENT SHALL BE DEEMED TO HAVE BEEN MADE IN THE STATE OF ILLINOIS AND THE VALIDITY OF THIS SECURITY AGREEMENT, ITS CONSTRUCTION, INTERPRETATION AND ENFORCEMENT, AND THE RIGHTS AND OBLIGATIONS OF PARTIES HEREUNDER, SHALL BE DETERMINED UNDER, GOVERNED BY AND CONSTRUED IN ACCORDANCE WITH THE INTERNAL LAWS OF THE STATE OF ILLINOIS, WITHOUT REGARD TO CONFLICTS OF LAW OR CHOICE OF LAW PRINCIPLES.

20. SUBMISSION TO JURISDICTION. EACH OF THE BORROWER HEREBY IRREVOCABLY AND UNCONDITIONALLY:

(i) SUBMITS FOR ITSELF AND ITS PROPERTY IN ANY LEGAL ACTION OR PROCEEDING RELATING TO THIS AGREEMENT, OR FOR RECOGNITION AND ENFORCEMENT OF ANY JUDGMENT IN RESPECT THEREOF, TO THE NON-EXCLUSIVE GENERAL JURISDICTION OF THE COURTS OF THE STATE OF ILLINOIS, THE COURTS OF THE UNITED STATES OF AMERICA FOR THE NORTHERN DISTRICT OF ILLINOIS AND APPELLATE COURTS FROM ANY THEREOF; AND

(ii) CONSENTS THAT ANY SUCH ACTION OR PROCEEDING MAY BE BROUGHT IN SUCH COURTS AND WAIVES TO THE FULLEST EXTENT PERMITTED BY LAW IN CONNECTION WITH ANY SUCH ACTION OR PROCEEDING ANY OBJECTION THAT SUCH PARTY MAY NOW OR HEREAFTER

6

HAVE TO THE VENUE OF ANY SUCH ACTION OR PROCEEDING IN ANY SUCH COURT OR THAT SUCH ACTION OR PROCEEDING WAS BROUGHT IN AN INCONVENIENT COURT AND AGREES NOT TO PLEAD OR CLAIM THE SAME.

21. JURY TRIAL WAIVER. EACH OF THE BORROWER AND AGENT HEREBY IRREVOCABLY AND KNOWINGLY WAIVES (TO THE FULLEST EXTENT PERMITTED BY LAW), AS SEPARATELY BARGAINED FOR CONSIDERATION TO AGENT, ANY RIGHT TO A TRIAL BY JURY IN ANY ACTION OR PROCEEDING (INCLUDING, WITHOUT LIMITATION, ANY COUNTERCLAIM) ARISING OUT OF THIS AGREEMENT OR ANY TRANSACTIONS RELATED HERETO, AND AGREE THAT ANY SUCH ACTION OR PROCEEDING SHALL BE TRIED BEFORE A COURT AND NOT A JURY. EACH OF THE BORROWER AND AGENT ACKNOWLEDGES THAT SUCH PARTY HAS HAD THE OPPORTUNITY TO CONSULT WITH INDEPENDENT LEGAL COUNSEL REGARDING THE MEANING AND EFFECT OF THIS PARAGRAPH.

[SIGNATURE PAGE FOLLOWS]

7

IN WITNESS WHEREOF, the Borrower has duly executed this Intellectual Property Security Agreement as of the date first written above.

EXHIBIT A

TRADEMARKS

Trademark Registrations

Trademark Applications

EXHIBIT B

COPYRIGHTS

None.

EXHIBIT C

PATENTS

Patent Registrations