Exhibit 10.1

RELINQUISHMENT OF RIGHTS AGREEMENT

This Relinquishment of Rights Agreement (this “Agreement”) is made and entered into as of this 3^rd day of April, 2012, by and among the various stockholders identified on the signature page hereto (collectively, the “Stockholders”) and Taylor Capital Group, Inc., a Delaware corporation (the “Company”).

RECITALS

WHEREAS, each of the Stockholders and the Company are subject to that certain Share Restriction Agreement, dated November 30, 1998, as amended (the “Share Restriction Agreement”), which such Share Restriction Agreement provides for certain restrictions on the transfer of any equity securities of the Company (the “Company Stock”) held by the Stockholders;

WHEREAS, pursuant to the Share Restriction Agreement, any Stockholder receiving a Bona Fide Offer (as such term is defined in the Share Restriction Agreement) to purchase his, her or its shares of Company Stock must first offer to sell such shares to the other Stockholders, and, if such other Stockholders do not exercise their options to purchase the shares, the Company may at its option, for a period of fifteen (15) days, elect to purchase such shares at the same price and on the same terms as contained in the Bona Fide Offer (such option, the “Company Option”);

WHEREAS, Section 2.9 of the Company’s Third Amended and Restated By-laws, as such document may be amended or restated from time to time (the “By-laws”), grants certain of the Stockholders the right to collectively nominate two directors to the Company’s Board of Directors so long as they beneficially own in the aggregate fifteen percent (15%) or more, and the right to nominate one additional director so long as they beneficially own in the aggregate twenty-five percent (25%) or more, of the total voting power of the outstanding stock entitled to vote generally in the election of the Company’s directors (the “Nomination Rights”); and

WHEREAS, the Company and the Stockholders believe it to be in their respective best interests for the Stockholders to relinquish their Nomination Rights and any other rights pursuant to a written agreement between any such Stockholder and the Company that may exist granting the Stockholders, in their capacity as Stockholders, structural rights with respect to the corporate governance of the Company (collectively, the “Governance Rights”) in exchange for the Company’s relinquishment of the Company Option under the Share Restriction Agreement, upon the terms and conditions set forth in this Agreement; provided, that such Governance Rights shall not include rights of any Stockholder as a stockholder of the Company under the law, the By-laws or other constitutional documents of the Company that are generally applicable to other stockholders or to any registration rights that may be held by such Stockholders;

NOW, THEREFORE, in consideration of the premises and the agreements set forth below, and for other good and valuable consideration, the receipt and sufficiency of which are hereby acknowledged, the parties agree as follows:

ARTICLE I

Covenants, Representations and Warranties of the Stockholders

Section 1.1 Waiver of Governance Rights. Each of the Stockholders hereby forever waives the Governance Rights to which such Stockholder or any of such Stockholder’s affiliates, heirs, assigns, or successors is otherwise entitled. Each of the Stockholders acknowledges that, upon execution of this Agreement, he, she or it will forfeit, on his, her or its behalf and that of any of his, her or its affiliates, heirs, assigns, or successors, any and all Governance Rights.

Section 1.2 Sufficiency of Waiver. Each of the Stockholders hereby represents and warrants to the Company that the Stockholders party hereto collectively: (a) represent all members of the Taylor Family as such term is defined in Section 2.9 of the Company’s By-laws; and (b) have the requisite power and authority to enter into and perform their respective obligations under this Agreement such that this Agreement constitutes a legal, valid and binding obligation of each of the Stockholders enforceable in accordance with its terms, subject to general principles of equity.

Section 1.3 Share Ownership and Voting Covenant. Each of the Stockholders hereby: (a) represents and warrants that such Stockholder is the record or beneficial owner (other than shares held in a fiduciary capacity for others) of that number of shares of Company Stock set forth in the Stockholder’s signature block on the signature page hereto (the “Shares”); and (b) agrees to vote (or cause to be voted) all of such Shares in favor of any proposal considered at a meeting of the Company’s stockholders to (i) amend Section 2.9 of the By-laws to remove the Nomination Rights (such proposal, the “By-laws Amendment”), (ii) amend Paragraph A of Article SEVENTH of the Company’s Third Amended and Restated Certificate of Incorporation, as such document may be amended or restated from time to time, to remove the requirement that any amendment to the By-laws affecting the Nomination Rights receive the affirmative vote of eighty-five percent (85%) of the Company’s stockholders (such proposal, the “Certificate Amendment”), (iii) approve the Company’s Fourth Amended and Restated Certificate of Incorporation, substantially in the form attached as Appendix A to the Company’s preliminary proxy statement filed with the Securities and Exchange Commission on March 30, 2012 and (iv) approve the Company’s Fourth Amended and Restated By-laws, substantially in the form attached as Appendix B to the Company’s preliminary proxy statement filed with the Securities and Exchange Commission on March 30, 2012; provided, however, that nothing in this Agreement shall prevent the Stockholder (or any representative of the Stockholder) from discharging his, her or its fiduciary duties as a member of the Board of Directors of the Company.

Section 1.4 Further Assurances. Each of the Stockholders hereby agrees: (a) to furnish upon request to the Company such further information; (b) to execute and deliver to the Company such other documents; and (c) to do such other acts and things; all as the Company may reasonably request for the purpose of carrying out the intent of this Agreement and the documents referred to in this Agreement.

Section 1.5 Release. Each of the Stockholders hereby knowingly and voluntarily releases, waives and forever discharges the Company, its affiliates, assigns, successors, subsidiaries, directors and officers, of and from any and all claims, known or unknown, which such Stockholder had, has or may have against the Company pursuant to the Share Restriction Agreement.

- 2 -

ARTICLE II

Covenants, Representations and Warranties of the Company

Section 2.1 Relinquishment of Company Option. The Company hereby agrees to cause its authorized representative to execute, on behalf of the Company, Amendment Number Two to the Share Restriction Agreement, in the form attached hereto as Exhibit A (the “Share Restriction Agreement Amendment”), to remove the Company Option from the Share Restriction Agreement, immediately following the execution of this Agreement.

Section 2.2 Release. The Company hereby knowingly and voluntarily releases, waives and forever discharges each of the Stockholders and all affiliates, heirs, assigns, or successors of each of the Stockholders from any and all claims, known or unknown, which the Company had, has or may have against the Stockholders pursuant to the Share Restriction Agreement.

Section 2.3 Further Assurances. The Company hereby agrees: (a) to furnish upon request of the Stockholders such further information; (b) to execute and deliver to the Stockholders such other documents; and (c) to do such other acts and things; all as the Stockholders may reasonably request for the purpose of carrying out the intent of this Agreement and the documents referred to in this Agreement.

ARTICLE III

Termination

Section 3.1 Termination. This Agreement will terminate effective upon the earlier of: (a) such time as (i) the Share Restriction Agreement Amendment has become effective, (ii) the By-laws Amendment has become effective under the General Corporation Law of the State of Delaware or the applicable law of any such other jurisdiction under which the Company may be organized at the time the By-laws Amendment receives approval of its stockholders, and (iii) no Stockholder holds any Governance Rights; and (b) such other time as may be mutually agreed upon in writing by the Company and the Stockholders.

Section 3.2 Effect of Termination. In the event that this Agreement shall be terminated pursuant to Section 3.1, all further obligations of the parties under this Agreement shall terminate without further liability of any party to another. A termination under Section 3.1 shall not relieve any party of any liability for a breach of this Agreement, or be deemed to constitute a waiver of any available remedy (including specific performance if available) for any such breach. Nothing in this Section 3.2 shall relieve either party to this Agreement of liability for a breach of a covenant or obligation under this Agreement prior to the termination of this Agreement pursuant to Section 3.1.

- 3 -

ARTICLE IV

Miscellaneous Provisions

Section 4.1 Notice. Any notices or other communications required or permitted to be given under the terms of this Agreement must be in writing and will be deemed to have been delivered: (a) upon receipt, when delivered personally; (b) upon receipt, when sent by facsimile (provided confirmation of transmission is mechanically or electronically generated and kept on file by the sending party); or (c) one business day after deposit with an overnight courier service, in each case addressed: (x) if to the Company, to its office at 9550 West Higgins Road, Rosemont, Illinois 60018, Attention: Corporate Secretary; or (y) if to any Stockholder, to the applicable address for receipt of notice for such Stockholder as provided for in the Share Restriction Agreement; or (z) to such other address as the Company or any such Stockholder, as the case may be, shall have designated by notice similarly given.

Section 4.2 Entire Agreement. This Agreement hereby embodies the entire agreement and understanding of the parties hereto with respect to the subject matter hereof and supersedes all prior and contemporaneous oral or written agreements, representations, warranties, contracts, correspondence, conversations, memoranda and understandings between or among the parties or any of their agents, representatives or affiliates relative to such subject matter, including any term sheets, emails or draft documents.

Section 4.3 Assignment; Binding Agreement. This Agreement and the various rights and obligations arising hereunder shall inure to the benefit of and be binding upon the parties hereto and their successors and assigns.

Section 4.4 Counterparts. This Agreement may be executed in multiple counterparts, and on separate counterparts, each of which shall be deemed an original, but all of which taken together shall constitute one and the same instrument. Any counterpart or other signature hereupon delivered by facsimile or other electronic transmission shall be deemed for all purposes as constituting good and valid execution and delivery of this Agreement by such party.

Section 4.5 Specific Performance. The parties hereto agree that irreparable damage would occur in the event that any of the provisions of this Agreement were not performed in accordance with their specific terms or were otherwise breached. It is accordingly agreed that the parties shall be entitled to specific performance of the terms and provisions hereof, in addition to any other remedy to which they are entitled at law or in equity.

Section 4.6 Governing Law; Jurisdiction; Jury Trial. All questions concerning the construction, validity, enforcement and interpretation of this Agreement shall be governed by the internal laws of the State of Delaware, without giving effect to any choice of law or conflict of law provision or rule (whether of the State of Delaware or any other jurisdictions) that would cause the application of the laws of any jurisdictions other than the State of Delaware. Each party hereby irrevocably submits to the exclusive jurisdiction of the state and federal courts sitting in the State of Delaware for the adjudication of any dispute hereunder or in connection herewith or with any transaction contemplated hereby or discussed herein, and hereby irrevocably waives, and agrees not to assert in any suit, action or proceeding, any claim that it is not personally subject to the jurisdiction of any such court, that such suit, action or proceeding is

- 4 -

brought in an inconvenient forum or that the venue of such suit, action or proceeding is improper. Each party hereby irrevocably waives personal service of process and consents to process being served in any such suit, action or proceeding by mailing a copy thereof to such party at the address for such notices to it under this Agreement and agrees that such service shall constitute good and sufficient service of process and notice thereof. Nothing contained herein shall be deemed to limit in any way any right to serve process in any manner permitted by law. EACH PARTY HEREBY IRREVOCABLY WAIVES ANY RIGHT IT MAY HAVE, AND AGREES NOT TO REQUEST, A JURY TRIAL FOR THE ADJUDICATION OF ANY DISPUTE HEREUNDER OR IN CONNECTION WITH OR ARISING OUT OF THIS AGREEMENT OR ANY TRANSACTION CONTEMPLATED HEREBY.

Section 4.7 No Third Party Beneficiaries or Other Rights. Nothing herein shall grant to or create in any person not a party hereto, or any such person’s dependents or heirs, any right to any benefits hereunder, and no such party shall be entitled to sue any party to this Agreement with respect thereto.

Section 4.8 Waiver; Consent. This Agreement may not be changed, amended, terminated, augmented, rescinded or discharged (other than in accordance with its terms), in whole or in part, except by a writing executed by the parties hereto. No waiver of any of the provisions or conditions of this Agreement or any of the rights of a party hereto shall be effective or binding unless such waiver shall be in writing and signed by the party hereto claimed to have given such waiver or consented thereto. Except to the extent otherwise agreed in writing, no waiver of any term, condition or other provision of this Agreement, or any breach thereof shall be deemed to be a waiver of any other term, condition or provision or any breach thereof, or any subsequent breach of the same term, condition or provision, nor shall any forbearance to seek a remedy for any noncompliance or breach be deemed to be a waiver of a party’s rights and remedies with respect to such noncompliance or breach.

Section 4.9 Construction; Interpretation; Certain Terms. The headings contained in this Agreement are for reference purposes only and shall not affect in any way the meaning or interpretation of this Agreement. Section, recital and party references are to this Agreement unless otherwise stated. The words “hereof,” “herein,” “hereunder” and words of similar import shall refer to this Agreement as a whole and not to any particular section or provision of this Agreement, and reference to a particular section of this Agreement shall include all subsections thereof. The term “including” as used in this Agreement shall mean including, without limitation, and shall not be deemed to indicate an exhaustive enumeration of the items at issue. All terms and words used in this Agreement, regardless of the number or gender in which they are used, shall be deemed to include any other number and any other gender as the context may require. No party, nor its counsel, shall be deemed the drafter of this Agreement for purposes of construing the provisions of this Agreement.

Section 4.10 Severability. If any one or more of the provisions contained herein, or the application thereof in any circumstance, is held invalid, illegal or unenforceable, the validity, legality and enforceability of any such provision in every other respect and of the remaining provisions contained herein shall not be affected or impaired thereby.

[Remainder of page intentionally left blank]

- 5 -

IN WITNESS WHEREOF, each of the parties hereto has caused this Relinquishment of Rights Agreement to be executed as of the date first above written.

[Signature Page 1 to Relinquishment of Rights Agreement]

[Signature Page 2 to Relinquishment of Rights Agreement]

EXHIBIT A

AMENDMENT NUMBER TWO TO SHARE RESTRICTION AGREEMENT

(See Exhibit 10.2)