Exhibit 10.1

ESQUIRE FINANCIAL HOLDINGS, INC.

320 Old Country Road, Suite 101

Garden City, New York 11530

December 23, 2014

CJA Private Equity Financial Restructuring Master Fund I, LP

c/o Gapstow Capital Partners

654 Madison Avenue, Suite 601

New York, NY 10065

Ladies and Gentlemen:

This letter will confirm our agreement that pursuant to and effective as of your purchase of capital stock of Esquire Financial Holdings, Inc. (the “Company”), a Delaware corporation and parent company of Esquire Bank (the “Bank”), CJA Private Equity Financial Restructuring Master Fund I, LP, a Cayman Islands limited partnership (the “Investor”), shall be entitled to the following contractual rights, in addition to any other rights specifically provided to the Investor pursuant to that certain Subscription Agreement dated as of December 23, 2014 by and between the Company and the Investor, including any amendments or supplements thereto, and such other agreements, instruments and certificates delivered in connection therewith (collectively, the “Subscription Documents”):

1.           Right to Designate Board Member. Effective on and after the first anniversary of the date hereof, as long as the Investor, together with its affiliates (as such term is defined under the Home Owners’ Loan Act of 1933, as amended (“HOLA”) (“Affiliates”), beneficially owns at least 4.0% of the total outstanding capital stock of the Company, and subject to any required approvals or non-objections of the Board of Governors of the Federal Reserve System (whether acting directly or through the Federal Reserve Bank of New York in such reserve bank’s regulatory capacity), the Federal Deposit Insurance Corporation, the Office of the Comptroller of the Currency and/or any other regulatory body having jurisdiction over the Company or the Bank (collectively, the “Regulators”), the Company shall use its reasonable best efforts, subject to the fiduciary duties of the directors of the Company, to cause one person nominated by the Investor to be elected to serve on the Board of Directors of the Company, and any direct or indirect subsidiary thereof, including the Bank (collectively, the “Board”), which efforts shall include, without limitation, if consistent with the fiduciary duties of the directors of the Company, soliciting proxies for the Investor’s nominee in the same manner as it does for the Company’s other nominees. Any director nominated by the Investor pursuant to this paragraph 1 shall be entitled to indemnification rights in his or her capacity as a member of the Board in a manner no less favorable as provided to other members of the Board. The Board representative of the Investor shall receive compensation from the Company equal in form and value to compensation paid to other Board members generally (such compensation to be paid as directed by the Investor), it being understood that certain Board members may receive different compensation

for services as chairman, lead director or a committee chairman. In the event the Investor designates as its Board representative an individual who is not an employee of the Investor or any of its Affiliates, the Company shall, in negotiation with the Investor, provide such individual with reasonable stock incentive compensation as the Company customarily offers to other independent, outside directors, if any. To facilitate the in-person attendance of the Board representative of the Investor (as a member of the Board pursuant to this paragraph 1 or as an observer pursuant to paragraph 2 below), the Company shall reimburse the Investor for all reasonable travel expenses of such representative promptly upon receiving documentation thereof reasonably acceptable to the Company.

2.           Board Observer Rights. If the Investor is not represented on the Board (including during such time as regulatory approval of the person nominated by the Investor as its Board representative is pending), as long as the Investor (together with its Affiliates) beneficially owns at least 4.0% of the total outstanding capital stock of the Company, the Company shall allow a representative of the Investor to attend all meetings of the Board in a non-voting observer capacity and, in this respect, shall give such representative copies of all notices, minutes, consents and other materials that it provides to its directors at the same time and in the same manner as provided to such directors; provided, however, that (i) such representative shall agree, in writing, to hold in confidence and trust all information so provided; (ii) the representative may be excluded from access to any material or meeting or portion thereof if the Board determines in good faith, upon advice of counsel, that access to such material or attendance at such meeting would adversely affect the attorney-client privilege between the Company or the Bank and its counsel or would conflict with applicable banking laws or regulations or if such material or meeting relates to relations or negotiations with the Investor or require the consent or non-objection of any Regulator; and (iii) such observer shall be excluded from all “executive sessions” of the Board if any other persons who are not members of the Board, other than counsel to the Company, are also excluded. For the avoidance of doubt, such representative shall not have access to any “confidential supervisory information” (as such term or relevant similar term is defined under the regulations of any Regulator).

3.           Terms of Agreements with Other Investors. The Company represents and warrants that it has not entered into, and does not currently intend to enter into, any agreement with any other investor who is not currently a stockholder of the Company or that provides rights to such investor related to its investment in capital stock of the Company, other than any such agreement as to which it has provided a copy to the Investor. If, during the eighteen-month period immediately after the date hereof, the Company enters into any such agreement with a party that agrees to purchase capital stock issued by the Company, then the Company shall promptly provide a copy of any such agreement to the Investor, and any terms of such agreement that are more favorable to such investor than the terms of the Subscription Documents shall be added and incorporated into this letter agreement, unless the Investor provides written notice to the Company that it elects to waive its rights to any such additional or modified terms.

4.           Capital Structure.

(a)           Preemptive Rights. If, following the consummation of the transactions contemplated by the Subscription Documents, and while the Investor (together with its

Affiliates) beneficially owns at least 4.0% of the total outstanding capital stock of the Company, the Company authorizes the issuance or sale of any securities, comparable or identical to the securities issued in this offering pursuant to the Subscription Documents, the Investor shall be entitled, in its sole discretion, to purchase shares of the Company’s capital stock such that the Investor would maintain its percentage ownership interest in the Company’s capital stock on a fully-diluted basis, subject to compliance with the applicable requirements of the Regulators. With respect to such rights described above (the “Preemptive Rights”), the Company shall give written notice of such proposed issuance or sale (including the terms and conditions thereof) to the Investor at least thirty (30) days prior to the anticipated issuance or sale date, and the Investor shall have twenty (20) days from the receipt thereof to provide the Company with notice of the exercise of its Preemptive Rights with respect to such issuance or sale. The Preemptive Rights described herein (A) will terminate upon the registration of the Company’s common stock, par value $0.01 per share (“Common Stock”) on NASDAQ, NYSE or other national securities exchange, (B) shall not apply in a transaction approved by the Board that results in a Change of Control (as defined below), (C) shall not apply to the issuance of Common Stock, or options to purchase the same, pursuant to the Company Stock Plans (as defined in the Subscription Documents) and (D) shall not apply to shares of Common Stock, or options to purchase the same, issued to employees or directors of, or consultants or advisors to, the Company or any Company Subsidiary (as defined in the Subscription Documents) pursuant to a plan, agreement or arrangement approved by the Board (each such issuances described in clauses (B) through (D) above, an “Excluded Issuance”); provided, however, that in all cases, (1) the aggregate ownership percentage of the Investor and its Affiliates of any class of voting securities of the Company shall not exceed 9.95% (or 4.95%, if so elected by the Investor) and (2) the total equity ownership in the Company by the Investor and its Affiliates shall not exceed 24.95% of the Company’s total issued and outstanding capital stock. For purposes of this letter agreement a “Change of Control” means the acquisition by any person (including a group of related persons within the meaning of Rule 13d-2 of the Securities Exchange Act of 1934, as amended) of (1) more than fifty percent (50%) of the outstanding capital stock of the Company, (2) all or substantially all of the assets of the Company (including the sale of more than two-thirds (2/3) of the capital stock held by the Company in the Bank) or (3) a merger of the Company with or into any person, or of any person with or into the Company, immediately after which the shareholders of the Company (as measured immediately prior to completion of the transaction) own less than a majority of the combined capital stock or membership interests of the surviving entity.

(b)          Avoidance of Control. Notwithstanding anything to the contrary in this letter agreement, neither the Company nor any Company Subsidiary (as defined in the Subscription Documents) shall take any action (including any redemption, repurchase, or recapitalization of common stock, or securities or rights, options or warrants to purchase common stock, or securities of any type whatsoever that are, or may become, convertible into or exchangeable into or exercisable for common stock in each case, where the Investor is not given the right to participate in such redemption, repurchase or recapitalization to the extent of the Investor’s pro rata proportion) that would: (i) cause the Investor’s equity of the Company (together with equity owned by the Investor’s Affiliates) to exceed 24.95% of the Company’s total equity; or (ii) cause the Investor’s ownership of any class of voting securities of the Company (together with the ownership by the Investor’s Affiliates of voting securities of the Company) to exceed 9.95% (or 4.95%, if so elected by the Investor), in each case without the

prior written consent of the Investor (which consent shall not be unreasonably withheld), or to increase to an amount that would constitute “control” under the HOLA or any rules or regulations promulgated thereunder (or any successor provisions) or otherwise cause the Investor to “control” the Company under and for purposes of the HOLA or any rules or regulations promulgated thereunder (or any successor provisions). Notwithstanding anything to the contrary in this letter agreement, the Investor (together with its Affiliates) shall not have the ability to own or control (A) more than 24.95% of the Company’s total equity or (B) in excess of 9.95% (or 4.95%, if so elected by the Investor) of any class of voting securities of the Company. In the event either the Company or the Investor breaches its obligations under this paragraph 4(b) or believes that it is reasonably likely to breach such an obligation, it shall promptly notify the other party hereto and shall cooperate in good faith with such party to modify ownership or make other arrangements or take any other action, in each case, as is necessary to cure or avoid such breach.

(c)          Dilutive Issuances. In the event the Company shall, at any time during the six-month period following the date hereof (provided that during such period the Investor, together with its Affiliates, beneficially owns at least 4.0% of the total outstanding capital stock of the Company), issue, sell or exchange any shares of Company Stock (other than in an Excluded Issuance) for a consideration per share less than $12.50 per share (the consideration for any such issuance, the “Dilutive Price”, and any such issuance, sale or exchange, a “Dilutive Transaction”), then and thereafter successively upon the consummation of any Dilutive Transaction, the Company shall issue and deliver to the Investor certificates representing such number of additional shares of Company Stock in an amount such that, when added to the number of shares issued to the Investor on the date hereof, the Investor has received an aggregate number of shares equal to the amount the Investor would have received if the Investor had purchased shares at a per share purchase price equal to the Dilutive Price for an aggregate purchase price equal to the aggregate purchase price paid for such shares pursuant to the Subscription Documents. The Investor may elect to receive such shares in the form of the Company’s Series B Non-Voting Preferred Stock (“Non-Voting Preferred Stock”). In all cases (i) the shares of Common Stock held by the Investor will be limited to 9.95% (or 4.95%, if the Investor so elects) of the Company’s total issued and outstanding shares of Common Stock (inclusive of all shares owned the Investor, including any that may have been acquired outside of the purchase effected by way of the Subscription Documents), which, as of the date hereof, is the only class of voting securities of the Company, and (ii) the Investor’s total equity ownership in the Company will be limited to 24.95% of the Company’s total issued and outstanding capital stock (inclusive of all shares owned by the Investor, including any that may have been acquired outside of the purchase effected by way of the Subscription Documents).

(d)          Exchange Rights. In the event the Investor acquires in excess of 4.95% of the voting securities held by all stockholders of the Company on a fully-diluted basis, the Investor shall have the right, but not the obligation, from time to time, in its sole discretion but subject to the conditions set forth below, to exchange any shares of Common Stock held by the Investor for shares of the Non-Voting Preferred Stock in order to reduce its ownership of voting Common Stock to 4.95% of the voting securities held by all stockholders of the Company on a fully-diluted basis. The shares of Non-Voting Preferred Stock shall be issued and authorized by the Company prior to the closing of the transactions contemplated by the Subscription Documents and shall have such rights, powers and preferences as set forth in a Certificate of

Designations of Series B Non-Voting Preferred Stock substantially in the form attached as Exhibit A hereto. Any shares of Non-Voting Preferred Stock received by the Investor or any Affiliate of the Investor pursuant to this paragraph shall not be convertible by the Investor or any Affiliate of the Investor into shares of Common Stock, and any such shares shall be subject to restrictions set forth in the Certificate of Designations of Series B Non-Voting Preferred Stock, including restrictions on transfer contained therein that are intended to have such shares qualify as non-voting shares under the applicable requirements and policies of the Regulators. In the event the Investor’s exercise of the exchange right set forth herein would cause any other stockholder of the Company to own or control either (i) 25% or more of any class of voting securities of the Company or (ii) 25% or more of the total equity of the Company, the Investor shall (A) refrain from exercising such right and (B) work together with the Company and such other stockholder in good faith in order to manage such other stockholder’s ownership limitations for the purpose of allowing the Investor to exercise such right; provided, however, that the Investor shall only be required to refrain from exercising such right if the Company and such other stockholder also use good faith efforts to manage such ownership limitations for such purpose.

(e)          Transferee Exchange Rights. Any transferee of the Investor’s shares of Non-Voting Preferred Stock shall have the right to exchange any shares of Non-Voting Preferred Stock held by such person for shares of Common Stock on a one-to-one basis; provided, however, that neither the Investor nor any Affiliate of the Investor shall have or shall exercise such right. Any such exchange shall be effected by way of an Exchange Agreement in form and substance substantially as set forth on Exhibit B hereto. Notwithstanding the foregoing, the Company may deny such exchange to the extent it would be inconsistent with, or in violation of, the requirements of any Regulator with respect to the restrictions on the transfer of the Non-Voting Preferred Stock that are required in order to preserve the non-voting classification for regulatory purposes of the Non-Voting Preferred Stock. Any such denial shall be imposed by the transmittal by the Company of written notice to such holder specifying in reasonable detail the reason for such denial. Such notice may be dispatched by first class mail, by electronic transmission, or by any other means reasonably designed and in good faith intended to provide prompt delivery to an executive officer (or equivalent) of, or legal counsel to, such holder.

5.           Registration Rights. In addition to, and not exclusive of, any other registration rights granted to the Investor pursuant to the Subscription Documents, the Investor shall be entitled to exercise “piggyback” registration rights to participate in the registration of shares pursuant to all registration statements proposed to be filed by the Company (except for the registration of securities (a) to be offered pursuant to an employee benefit plan on Form S-8 or pursuant to a registration made on Form S-4 or any successor forms then in effect or (b) in a transaction relating solely to the sale of debt or convertible debt instruments). The rights and obligations of the Investor and the Company in respect of such registration rights shall be set forth in a Registration Rights Agreement in form and substance substantially as set forth on Exhibit C hereto.

6.           Regulatory Approval. The Company and the Investor shall cooperate to obtain the appropriate approvals from the Regulators in accordance with this letter agreement and the Subscription Documents. If necessary, the Investor shall agree to certain passivity commitments

imposed by the Regulators, provided, that the terms and conditions of such commitments are customary and are not deemed by the Investor (in its sole discretion) to be unreasonable and provided, further, that the Investor shall not be required to agree to any restrictions, conditions or commitments imposed or otherwise required by any Regulator that are determined by the Investor (in its sole discretion) to be unduly burdensome.

7.           Miscellaneous. The validity, construction and interpretation of this letter agreement and the rights and duties of the parties hereunder shall be governed by and construed in accordance with laws of the State of Delaware without regard to its conflicts of laws provisions. This letter agreement (together with the Subscription Documents) constitutes the entire agreement among the parties hereto, and supersedes any and all prior representations, agreements and understandings, whether written or oral, with respect to the subject matter hereof. This letter agreement shall not be modified, amended or waived, in whole or in part, except by written agreement of both parties. The provisions hereof shall be binding upon, and shall inure to the benefit of, the parties hereto and their successors and assigns. Each of the parties hereto shall, at the request of the other party, execute, deliver and acknowledge without any consideration, such additional documents, instruments or certificates or do or cause to be done such other things as are reasonably necessary or desirable to make effective the agreements and transactions contemplated by this letter agreement. This letter agreement may be executed and delivered (including by facsimile or other electronic transmission) in multiple counterparts, each of which shall constitute an original and all of which together shall be deemed to be one and the same instrument.

[Remainder of page intentionally left blank. Signature page follows.]

******************************************************

ACKNOWLEDGED AND AGREED:

CJA PRIVATE EQUITY FINANCIAL

RESTRUCTURING MASTER FUND I, LP

GP I, Ltd, its General Partner

By: Christopher J. Acito & Associates

GP, LLC, its Managing Member

[Signature Page to Investor Rights Letter Agreement]