Exhibit 10.4

POLYCOM, INC.

CONSULTING AGREEMENT

This Consulting Agreement (this “Agreement”) is made and entered into as of May 10, 2010 (the “Effective Date”) by and between Polycom, Inc., a Delaware corporation (the “Company”), and Robert C. Hagerty (“Consultant”) (each herein sometimes referred to individually as a “Party,” or collectively as the “Parties”).

Whereas, Consultant has served as the President and Chief Executive Officer of the Company since 1998 and Chairman of the Board of Directors since 2000, and accordingly Consultant has deep domain knowledge and understanding of the Company, and its business and market; and

Whereas, effective May 10, 2010, Consultant ceased serving as an employee of the Company and as the Company’ President and Chief Executive Officer, and resigned his position as Chairman and a member of the Board of Directors of the Company; and

Whereas, effective May 10, 2010, the Parties intend that Consultant incurred a “separation from service” within the meaning of Section 409A of the Internal Revenue Code of 1986, as amended, and the final regulations and any guidance promulgated thereunder and any applicable state law equivalents (as each may be amended or promulgated from time to time) (“Section 409A”); and

Whereas, the Company desires to retain Consultant as an independent contractor to advise the Board of Directors and the Chief Executive Officer on such strategic and other matters as the Board of Directors or Chief Executive Officer may request, and Consultant is willing to perform such advisory services on the terms described in this Agreement;

Now, therefore, in consideration of these premises and the mutual promises made herein, the Company and Consultant hereby agree as follows:

1. Services and Compensation

Consultant shall perform the services described in Exhibit A (the “Services”) for the Company, and the Company agrees to pay Consultant the compensation described in Exhibit A for Consultant’s performance of the Services.

2. Confidential Information and Ownership of Inventions

Consultant reaffirms and agrees to execute, observe and abide by the terms of his Proprietary Information and Invention Agreement dated January 14, 2007 (the “Proprietary Information Agreement”), specifically including the provisions therein regarding nondisclosure of the Company’s trade secrets and confidential and proprietary information.

3. Conflicting Obligations

A. Consultant represents and warrants that Consultant has no agreements, relationships, or commitments to any other person or entity that conflict with the provisions of this Agreement, Consultant’s obligations to the Company under this Agreement, or Consultant’s ability to perform the Services. Consultant will not enter into any such conflicting agreement during the term of this Agreement. Consultant’s violation of this Section 3 will be considered a material breach under Section 5.B.

B. Consultant explicitly agrees that while he is a consultant he shall not, other than on behalf of the Company or with the prior written consent of the Company, (i) serve as a partner, employee, independent contractor, consultant, advisor, officer, director, proprietor, manager, agent, associate, or (ii) directly or indirectly own (except for passive ownership of one percent (1%) or less of any entity whose securities have been registered under the Securities Act of 1933 or Section 12 of the Securities Exchange Act of 1934), purchase, invest in, organize, or take preparatory steps for the organization of, or (iii) directly or indirectly build, design, finance, acquire, lease, control, operate, manage, invest in, work, consult for, or otherwise affiliate with the following companies and any successor in interest to the business and assets of such companies whether by merger, purchase of assets, purchase of stock, or otherwise: Konftel, Huawei’s video unit, Plantronics, Cisco, Logitech, Radvision, Teleris, Vidyo, ClearOne Communications, Ascom, Mitel, Yamaha, Aastra, LG Nortel, Snom, Thomson, Siemens, Hitachi Cable, Ericsson, Motorola, Samsung, Vocera, Alcatel-Lucent, Avaya and Aethra. Consultant represents that he (i) is familiar with the foregoing covenant not to compete, (ii) is fully aware of his obligations hereunder, and (iii) acknowledges the reasonableness and necessity of this covenant, including the reasonableness in duration and scope, as necessary for the Company to avoid the actual or threatened misappropriation of the Company’s trade secrets and confidential information.

4. Return of Company Materials

Upon the termination of this Agreement, Consultant will immediately deliver to the Company, and will not keep in Consultant’s possession, recreate, or deliver to anyone else, any and all Company property, including, but not limited to, Confidential Information (as defined in the Proprietary Information Agreement), tangible embodiments of the Inventions (as defined in the Proprietary Information Agreement), all devices and equipment belonging to the Company, all electronically-stored information and passwords to access such property, and any reproductions of any of the foregoing items that Consultant may have in Consultant’s possession or control.

5. Term and Termination

A. Term. The term of this Agreement will begin on the Effective Date of this Agreement and will continue until the earlier of (i) one (1) year after the Effective Date, or (ii) Employee becoming employed as an employee or otherwise performing services for another entity in excess of twenty four (24) hours per week, or (iii) termination as provided in Section 5.B. Upon mutual written agreement, the parties may elect to renew this Agreement for an additional one-year term.

B. Termination. The Company may terminate this Agreement immediately and without prior notice if Consultant is unable or refuses to perform the Services, and either Party may terminate this Agreement immediately and without prior notice if the other Party is in breach of any material provision of this Agreement.

C. Survival. Upon any termination, all rights and duties of the Company and Consultant toward each other shall cease except:

(1) The Company will pay, within thirty (30) days after the effective date of termination, all amounts owing to Consultant for Services completed and accepted by the Company prior to the termination date and related reimbursable expenses, if any, submitted in accordance with the Company’s policies; and

-2-

(2) Section 2 (Confidentiality), Section 3 (Conflicting Obligations), Section 4 (Return of Company Materials), Section 5 (Term and Termination), Section 6 (Independent Contractor), Section 7 (Noninterference), Section 8 (Arbitration and Equitable Relief), and Section 9 (Miscellaneous) will survive termination or expiration of this Agreement in accordance with their terms.

6. Independent Contractor; Tax Consequences

A. It is the express intention of the Company and Consultant that Consultant perform the Services as an independent contractor to the Company. Nothing in this Agreement shall in any way be construed to constitute Consultant as an agent or employee of the Company. Without limiting the generality of the foregoing, Consultant is not authorized to bind the Company to any liability or obligation or to represent that Consultant has any such authority. Consultant acknowledges and agrees that Consultant is obligated to report as income all compensation received by Consultant pursuant to this Agreement. Consultant agrees to and acknowledges the obligation to pay all self-employment and other taxes on such income. The Company and Consultant agree that Consultant will receive no Company-sponsored benefits from the Company.

B. The Company makes no representations or warranties with respect to the tax consequences of the payments and any other consideration provided to Consultant under the terms of this Agreement. Consultant agrees and understands that he is responsible for payment, if any, of local, state, and/or federal taxes on the payments and any other consideration provided hereunder by the Company and any penalties or assessments thereon. Consultant agrees to indemnify and hold harmless the Company and its affiliates and their directors, officers and employees from and against all taxes, losses, damages, liabilities, costs and expenses, including attorneys’ fees and other legal expenses, arising from or in connection with (i) any obligation imposed on the Company to pay withholding taxes or similar items, (ii) any determination by a court or agency that the Consultant is not an independent contractor.

C. The provisions of this Agreement and all compensation provided for under this Agreement are intended to comply with or be exempt from the requirements of Section 409A so that none of the payments to be provided hereunder will be subject to the additional tax imposed under Section 409A, and any ambiguities or ambiguous terms herein will be interpreted to be exempt or to so comply. It is the intent of the parties that all payments under this Agreement qualify for an exemption from Section 409A or meet the Section 409A requirements regarding time and form of payment. Each installment or payment payable under this Agreement is intended to constitute a separate payment for purposes of Section409A. Any such amount that is paid on or before March 15^th of the calendar year following the calendar year in which it ceases to be subject to a substantial risk of forfeiture (within the meaning of Section 409A) is intended to qualify as a “short term deferral” under Treasury Regulation Section 1.409A-1(b)(4).

7. Noninterference

To the fullest extent permitted under applicable law, from the date of this Agreement until twelve (12) months after the termination of this Agreement for any reason (the “Restricted Period”), Consultant will not, without the Company’s prior written consent, either directly or indirectly solicit, induce, recruit or encourage any of the Company’s employees with whom he has worked and/or about whom he has received material, confidential information during his engagement hereunder, to leave their employment, or take away such employees, or attempt to solicit, induce, recruit, encourage or take away employees of

-3-

the Company, either for himself or any other person or entity. During the period of time he is a Consultant, Consultant will not, whether for Consultant’s own account or for the account of any other person, firm, corporation or other business organization, intentionally interfere with any person who is or during the period of Consultant’s engagement by the Company was a partner, supplier, customer or client of the Company or its affiliates.

8. Arbitration and Equitable Relief

A. Arbitration. In consideration of Consultant’s consulting relationship with Company, its promise to arbitrate all disputes related to Consultant’s consulting relationship with the Company and Consultant’s receipt of the compensation paid to Consultant by Company, Consultant agrees that any and all controversies, claims, or disputes with anyone (including Company and any employee, officer, director, shareholder or benefit plan of the Company in their capacity as such or otherwise), arising out of, relating to, or resulting from Consultant’s consulting relationship with the Company or the termination of Consultant’s consulting relationship with the Company, including any breach of this Agreement, shall be subject to binding arbitration under the arbitration rules set forth in California Code of Civil Procedure Section 1280 through 1294.2, including Section 1281.8 (the “Act”) and pursuant to California Law. Consultant further understands that this Agreement to arbitrate also applies to any disputes that the Company may have with Consultant. Except as provided by the Act and this Agreement, arbitration shall be the sole, exclusive, and final remedy for any dispute between Consultant and the Company, and neither Consultant nor the Company will be permitted to pursue court action regarding claims that are subject to arbitration. This agreement to arbitrate shall apply to disputes regarding Consultant’s consulting relationship, and shall not otherwise supersede Consultant’s prior agreement to arbitrate employment-related disputes in his Amended Severance Agreement dated February 8, 2010.

B. Procedure. Consultant agrees that any arbitration will be administered by Judicial Arbitration & Mediation Services, Inc. (“Jams”) pursuant to its employment arbitration rules & procedures (the “Jams Rules”). Consultant agrees that the arbitrator shall have the power to decide any motions brought by any party to the arbitration, and that the arbitrator shall issue a written decision on the merits. Consultant also agrees that the arbitrator have the power to award any remedies available under applicable law, and that the arbitrator may award attorneys’ fees and costs to the prevailing party, except as prohibited by law. Consultant agrees that the decree or award rendered by the arbitrator may be entered as a final and binding judgment in any court having jurisdiction thereof. To the extent that the Jams Rules conflict with California law, California law shall take precedence. Consultant further agrees that any arbitration under this Agreement shall be conducted in Santa Barbara County, California.

C. Availability of Injunctive Relief. In accordance with Rule 1281.8 of the California Code of Civil Procedure, the Parties agree that any Party may also petition the court for injunctive relief where either Party alleges or claims a violation of any agreement regarding intellectual property, confidential information or noninterference. In the event either Party seeks injunctive relief, the prevailing Party shall be entitled to recover reasonable costs and attorneys’ fees.

9. Miscellaneous

A. Governing Law; Consent to Personal Jurisdiction. This Agreement shall be governed by the laws of the State of California, without regard to California’s conflicts of law rules. To the extent that any lawsuit is permitted under this Agreement, the Parties hereby expressly consent to the personal and exclusive jurisdiction and venue of the state and federal courts located in California.

-4-

B. Assignability. This Agreement will be binding upon Consultant’s heirs, executors, assigns, administrators, and other legal representatives, and will be for the benefit of the Company, its successors, and its assigns. Consultant may not sell, assign or delegate any rights or obligations under this Agreement. Company may assign this Agreement and its rights and obligations under this Agreement to any successor to all or substantially all of Company’s relevant assets, whether by merger, consolidation, reorganization, reincorporation, sale of assets or stock.

C. Entire Agreement. This Agreement constitutes the entire agreement and understanding between the Parties with respect to the subject matter herein and supersedes all prior written and oral agreements, discussions, or representations between the Parties. Consultant represents and warrants that he is not relying on any statement or representation not contained in this Agreement.

D. Severability. If a court or other body of competent jurisdiction determines any provision of this Agreement, or portion thereof, to be invalid or unenforceable, such provision will be enforced to the maximum extent permissible so as to effect the intent of the Parties, and the remainder of this Agreement will continue in full force and effect.

E. Modification, Waiver. No modification of or amendment to this Agreement, nor any waiver of any rights under this Agreement, will be effective unless in a writing signed by the Parties. Waiver by the Company of a breach of any provision of this Agreement will not operate as a waiver of any other or subsequent breach.

F. Notices. Any notice or other communication required or permitted by this Agreement to be given to a Party shall be in writing and shall be deemed given (i) if delivered personally or by commercial messenger or courier service, (ii) when sent by electronic mail or by confirmed facsimile, or (iii) if mailed by U.S. registered or certified mail (return receipt requested), to the Party at the Party’s address written below or at such other address as the Party may have previously specified by like notice. If by mail, delivery shall be deemed effective three business days after mailing in accordance with this Section 10.F.

(2) If to Consultant, to the address for notice on the signature page to this Agreement or, if no such address is provided, to the last address of Consultant provided by Consultant to the Company.

G. Attorneys’ Fees. In any court action at law or equity that is brought by one of the Parties to this Agreement to enforce or interpret the provisions of this Agreement, the prevailing Party will be entitled to reasonable attorneys’ fees, in addition to any other relief to which that Party may be entitled.

-5-

IN WITNESS WHEREOF, the Parties hereto have executed this Consulting Agreement as of the date first written above.

-6-

EXHIBIT A

SERVICES AND COMPENSATION

1. Contact. Consultant’s principal contact information:

Name: Robert C. Hagerty

Email: Robert.hagerty@polycom.com

Phone: 925.963.2692

2. Services. Consultant agrees to advise the Company’s Board of Directors and Chief Executive Officer, at reasonable times, in matters related to the Company’s business and market and to make good faith efforts to meet with the Company, as appropriate as reasonably requested.

3. Compensation and Expenses.

A. Subject to Section 5 of the Agreement, in consideration for the services to be performed by Consultant under this Agreement, the Company agrees, to pay Consultant $41,666.66 per month for twelve months, payable monthly on the last business day of each month commencing May 2010 through April 2011. If this Agreement is terminated during a calendar month, Consultant will receive a pro-rata payment for the Services rendered prior to such termination with respect to such calendar month, and Consultant will be entitled to no further payments pursuant to this Agreement (except to the extent permitted under subsection B below).

B. The Company will reimburse Consultant, in accordance with Company policy, for all reasonable travel and expenses incurred by Consultant in performing the Services pursuant to this Agreement.

This Exhibit A is accepted and agreed upon as of May 10, 2010.