Exhibit 10.4

VOTING AGREEMENT

VOTING AGREEMENT (this “Agreement”), dated as of November 12, 2013, by and among the investment entities managed or advised by Senator Investment Group LP, a Delaware limited partnership, as set forth on the signature pages hereto (each, a “Backstop Investor” and collectively, the “Backstop Investors”), and David Levin (the “Shareholder”). Each of the Backstop Investors and the Shareholder is referred to as a “party” and collectively as the “parties.” Capitalized terms used but not defined in this Agreement have the meanings ascribed to them in the Purchase Agreement (defined below).

RECITALS

A. As of the date hereof, the Shareholder is the record and beneficial owner of such number of shares of common stock (the “Common Stock”), par value $0.01, of Trade Street Residential, Inc. (the “Company”), as set forth opposite its name on Schedule I hereto (“Company Shares”, and together with any other Company Shares owned or acquired (including, after the date hereof) by the Shareholder, being collectively referred to herein as the “Shareholder Shares”).

B. The Company intends to launch a rights offering (the “Rights Offering”) pursuant to which the Company is distributing at no charge to the holders of Common Stock transferable subscription rights to purchase shares of Common Stock at a specified cash subscription price per whole share.

C. In connection with the Rights Offering, the Company intends to enter into a standby purchase agreement (the “Standby Purchase Agreement”) with the Backstop Investors, pursuant to which the Backstop Investors have committed to purchase from the Company, subject to the terms and conditions of the Standby Purchase Agreement, at the Rights Offering subscription price, all of the unsubscribed shares of Common Stock in the Rights Offering (minus the aggregate dollar value of the Management Purchase Commitment (as defined below)) such that the gross proceeds to the Company from the sale of shares of Common Stock in the Rights Offering will be $100.0 million. This commitment by the Backstop Investors is referred to as the “Backstop Commitment.”

D. In addition, the Backstop Investors have agreed to purchase $50.0 million of Common Stock in addition to shares of Common Stock purchased pursuant to the Backstop Commitment, which is referred to as the “Additional Commitment.”

E. In connection with the Backstop Commitment and pursuant to the Standby Purchase Agreement, the Company has agreed to enter into a stockholders’ agreement with the Backstop Investors (the “Stockholders Agreement”), pursuant to which the Backstop Investors will be granted, among other rights, certain board designation rights, negative control rights and a right to seek liquidity with respect to the shares of Common Stock it owns.

F. The proposed adoption of the Backstop Commitment, the Additional Commitment and the Stockholders Agreement by the shareholders of the Company are collectively referred to as the “Proposals.”

G. As a condition to the Backstop Commitment and the Additional Commitment, the Company’s Chairman and Chief Executive Officer, Michael Baumann, and Vice-Chairman and President, David Levin, have agreed to purchase shares of Common Stock that are subject to their basic subscription right at the subscription price of the Rights Offering for an aggregate commitment based on the number of Common Stock that each individual owns as of the record date (the “Management Purchase Commitment” and together with the Standby Purchase Agreement, the Stockholders Agreement or any other agreement, document, certificate or other instrument delivered in connection with the transactions contemplated hereby and thereby, the “Transaction Documents”), pursuant to a purchase agreement to be concurrently executed with this Agreement.

H. As a condition to their willingness to enter into the Standby Purchase Agreement, the Backstop Investors have required that the Shareholder enter into this Agreement.

I. The Backstop Investors have asked for the Shareholder to vote in favor of, or consent to, the Proposals and the other transactions contemplated by the Standby Purchase Agreement and to enter into this Agreement, and the Shareholder has agreed, subject to the terms hereof, to vote in favor of, or give its consent to, the Proposals and is willing to enter into this Agreement.

AGREEMENT

In consideration of the foregoing and other good and valuable consideration, the receipt and sufficiency of which are hereby acknowledged, the parties, intending to be legally bound, agree as follows:

1. Agreements of the Shareholder.

(a) Voting. From the date hereof until any termination of this Agreement in accordance with its terms, at any annual, special or other meeting of the shareholders of the Company, however called (or any action by written consent in lieu of a meeting) or any adjournment thereof, the Shareholder shall vote all of its Shareholder Shares (or cause them to be voted), in person or by proxy, or, as applicable, execute written consents in respect thereof, (i) in favor of the Proposals and the other transactions contemplated by the Transaction Documents, (ii) against any action or agreement (including, without limitation, any amendment of any agreement) that would result in a breach of any representation, warranty, covenant, agreement or other obligation of the Company in any of the Transaction Documents, or of the Shareholder contained in this Agreement, (iii) unless the Standby Purchase Agreement is previously terminated in accordance with its terms, against any Acquisition Transaction and against any other action, agreement or transaction that is intended, or could reasonably be expected to impede, interfere with, delay, postpone, discourage, frustrate the purposes of or adversely affect the Rights Offering, the Proposals or the other transactions contemplated by the Transaction Documents or this Agreement or the performance by the Company of its obligations under the Transaction Documents or by the Shareholder of its obligations under this Agreement, and (iv) in favor of any adjournment or postponement of the annual, special or other meeting of the shareholders recommended by the Board of Directors of the Company if there are not sufficient votes for adoption of the

2

Proposals on the date on which such meeting is initially held or scheduled, as applicable. Any such vote shall be cast, or consent shall be given, as applicable, by the Shareholder in accordance with such procedures relating thereto so as to ensure that it is duly counted, including for purposes of determining that a quorum is present and for purposes of recording the results of such vote or consent.

(b) Restriction on Transfer; Proxies; Non-Interference; etc. From the date hereof until any termination of this Agreement in accordance with its terms, the Shareholder shall not directly or indirectly (i) offer, sell, transfer, tender, give, hypothecate, pledge, encumber, assign or otherwise dispose of, or enter into any contract, option or other arrangement or understanding with respect to the offer, sale, transfer, tender, gift, hypothecation, pledge, encumbrance, assignment or other disposition of, or grant a proxy or power of attorney with respect to, or create or permit to exist any security interest, lien, claim, pledge, option, right of first refusal, agreement, limitation on such Shareholder’s voting rights, charge or other encumbrance of any nature whatsoever (“Encumbrance”) with respect to any of its Shareholder Shares (or any right, title or interest thereto or therein), (ii) deposit any of its Shareholder Shares into a voting trust or grant any proxies or enter into a voting agreement, power of attorney or voting trust with respect to any of its Shareholder Shares, (iii) take any action that would make any representation or warranty of the Shareholder set forth in this Agreement untrue or incorrect or have the effect of preventing, disabling or delaying the Shareholder from performing any of its obligations under this Agreement or (iv) agree (whether or not in writing) to take any of the actions referred to in the foregoing clauses (i), (ii) or (iii) of this Section 1(b). Notwithstanding the foregoing, the Shareholder may make transfers of its Shareholder Shares by will, gift, operation of Laws, for estate planning purposes, to an affiliate of the Shareholder or to another shareholder of the Company, in each case, in which the transferee (other than another Shareholder, for whom such transferred Company Shares shall constitute additional Shareholder Shares of the Shareholder to whom such Company Shares were transferred) agrees to be bound by all terms of this Agreement pursuant to a written agreement in a form reasonably satisfactory to the Backstop Investors that the transferee shall be bound by, and the Shareholder Shares so transferred, assigned or sold shall remain subject to this Agreement.

(c) Information for Proxy Statement; Publication. The Shareholder hereby authorizes the Company to publish and disclose, in the registration statement related to the Rights Offering, the proxy statement related to the Proposals and any other filing with the SEC or other governmental authority in connection with the Rights Offering or the Proposals, or the transactions contemplated by the Transaction Documents, the Shareholder’s identity and ownership of Shareholder Shares and the nature of the Shareholder’s commitments, arrangements and understandings under this Agreement. The Shareholder represents and warrants to each Backstop Investor that none of the information relating to the Shareholder and his, her or its affiliates provided in writing by or on behalf of the Shareholder or his, her or its affiliates specifically for inclusion in any such filing will, at the time that such filing is first made or distributed, contain any untrue statement of a material fact or omit to state any material fact required to be stated therein or necessary in order to make the statements therein, in light of the circumstances under which they were made, not misleading. The Shareholder shall not issue any press release or make any other public statement with respect to this Agreement, the Rights Offering, the Proposals or the other transactions contemplated by the

3

Transaction Documents, without the prior written consent of the Backstop Investors and the Company, except as may be required by applicable Law (including as may be required for the Shareholder to comply with its obligations under the Exchange Act) and in case any such requirement under applicable Law arises, after giving the Backstop Investors and the Company reasonable opportunity to comment on any such press release or statement and including all reasonably requested comments of the Backstop Investors and the Company, provided that the ultimate discretion whether or not to include any such comments shall remain with the Shareholder and provided that the Shareholder shall not disclose the identity of the Backstop Investors.

(d) Proxy. The Shareholder hereby irrevocably appoints as its proxy and attorney-in-fact, the Company or its designee to vote or execute written consents with respect to the Shareholder Shares in accordance with Section 1(a) and, in the discretion of the Company or its designee, with respect to any proposed postponements or adjournments of any annual or special meetings of the stockholders of the Company at which any of the matters described in Section 1(a) is to be considered. This proxy is coupled with an interest, was given to secure the obligations of the Shareholder under Section 1(a), was given as an additional inducement for the Backstop Investors to enter into the Transaction Documents and shall be irrevocable, and the Shareholder shall take such further action or execute such other instruments as may be necessary to effectuate the intent of this proxy and hereby revokes any proxy previously granted by the Shareholder with respect to the Shareholder Shares. The power of attorney granted by the Shareholder herein is a durable power of attorney and shall survive the dissolution, bankruptcy, death or incapacity of the Shareholder. The Company may terminate this proxy with respect to the Shareholder at any time at its sole election by written notice provided to the Shareholder.

2. Representations and Warranties of the Shareholder. The Shareholder hereby represents and warrants to each Backstop Investor as follows:

(a) Authority. The Shareholder has all necessary legal capacity, power and authority to execute and deliver this Agreement and to perform the Shareholder’s obligations under this Agreement. This Agreement has been duly executed and delivered by the Shareholder and, assuming due and valid authorization, execution and delivery hereof by the Backstop Investors, constitutes a valid and binding obligation of the Shareholder, enforceable against the Shareholder in accordance with its terms, subject to (i) laws of general application relating to bankruptcy, insolvency and the relief of debtors, and (ii) rules of law governing specific performance, injunctive relief and other equitable remedies.

(b) Consents and Approvals; No Violations. Other than filings under the Exchange Act, no notices, reports or other filings are required to be made by the Shareholder with, nor are any consents, registrations, approvals, permits or authorizations required to be obtained by the Shareholder from, any governmental authority or any other Person or entity, in connection with the execution and delivery of this Agreement by the Shareholder. The execution, delivery and performance of this Agreement by the Shareholder does not, and the consummation by the Shareholder of the transactions contemplated hereby will not (i) conflict with or violate any federal, state or local law, statute, ordinance, rule, regulation, order, judgment or decree applicable to such Shareholder or by which the Company Shares are bound or affected or (ii) result in a violation or breach of, or constitute (with or without due

4

notice or lapse of time or both) a default (or give rise to any right of termination, cancellation, modification or acceleration) (whether after the giving of a notice or the passage of time of both) under any contract to which the Shareholder is a party or which is binding on the Shareholder or the Shareholder’s assets or properties and will not result in the creation of any lien or any encumbrance on any of the assets or properties of the Shareholder, including, without limitation, on the Company Shares.

(c) Ownership of Shares. The Shareholder owns, beneficially and of record, all of its Company Shares, and is entitled to vote, without restriction, on all matters brought before holders of capital stock of the Company. The Shareholder owns all of its Company Shares free and clear of any proxy, voting restriction, adverse claim, security interest, pledge, option, right of first refusal, agreement, limitation on such Shareholder’s voting rights, lien, charge or other encumbrance of any nature whatsoever (other than proxies and restrictions in favor of the Company pursuant to this Agreement and transfer restrictions under the applicable securities Laws). Without limiting the foregoing, except for proxies and restrictions in favor of the Company pursuant to this Agreement and transfer restrictions under applicable securities Laws, the Shareholder has sole voting power and sole power of disposition with respect to all of its Company Shares, with no restrictions on the Shareholder’s rights of voting or disposition pertaining thereto and no Person other than the Shareholder has any right to direct or approve the voting or disposition of any of its Company Shares. As of the date hereof, the Shareholder does not own, beneficially or of record, any voting securities of the Company other than the Company Shares which constitute its Shareholder Shares.

(d) Brokers. No broker, investment banker, financial advisor or other Person is entitled to any broker’s, finder’s, financial advisor’s or other similar fee or commission that is payable by the Company or any of its subsidiaries in connection with the Rights Offering, the Proposals or the other transactions contemplated by the Transaction Documents based upon arrangements made by or on behalf of the Shareholder.

3. Company Cooperation. The Company hereby covenants and agrees that it will not, and the Shareholder irrevocably and unconditionally acknowledges and agrees that the Company will not (and waives any rights against the Company in relation thereto), recognize any Encumbrance or agreement on any of the Shareholder Shares subject to this Agreement unless the provisions of Section 1(b) have been complied with.

4. Termination. This Agreement shall terminate on the first to occur of (a) in respect of the Shareholder, the mutual written agreement of the Backstop Investors and the Shareholder, (b) the termination of the Standby Purchase Agreement in accordance with its terms or (c) the time the Company obtains the required approval of its shareholders of the Proposals and the Transaction Documents and all of the transactions contemplated hereby and thereby. Notwithstanding the foregoing, (x) nothing herein shall relieve any party from liability for any breach of this Agreement, and (y) the provisions of this Section 4 and Section 5 of the Agreement shall survive any termination of this Agreement.

5

5. Miscellaneous.

(a) Expenses. Except as otherwise expressly provided in this Agreement, all costs and expenses incurred in connection with the transactions contemplated by this Agreement shall be paid by the party incurring such costs and expenses.

(b) Additional Shares. Until any termination of this Agreement in accordance with its terms, the Shareholder shall promptly notify the Company and the Backstop Investors of the number of Company Shares, if any, as to which the Shareholder acquires record or beneficial ownership after the date hereof. Any Company Shares as to which a Shareholder acquires record or beneficial ownership after the date hereof and prior to termination of this Agreement shall be Shareholder Shares of the Shareholder for purposes of this Agreement. Without limiting the foregoing, in the event of any share split, share dividend or other change in the capital structure of the Company affecting the Company Shares, the number of Company Shares constituting Shareholder Shares shall be adjusted appropriately and this Agreement and the obligations hereunder shall attach to any additional shares of Company Shares or other voting securities of the Company issued to the Shareholder in connection therewith.

(c) Definition of “Beneficial Ownership”. For purposes of this Agreement, “beneficial ownership” with respect to (or to “own beneficially”) any securities shall mean having “beneficial ownership” of such securities (as determined pursuant to Rule 13d-3 under the Exchange Act), including pursuant to any agreement, arrangement or understanding, whether or not in writing.

(d) Further Assurances. From time to time, at the reasonable request of the Backstop Investors and without further consideration, the Shareholder shall execute and deliver such additional documents and take all such further action as may be reasonably required to consummate and make effective, in the most expeditious manner practicable, the transactions contemplated by this Agreement.

(e) Entire Agreement; No Third Party Beneficiaries. This Agreement constitutes the entire agreement, and supersedes all prior agreements and understandings, both written and oral, among the parties, or any of them, with respect to the subject matter hereof. This Agreement is not intended to and shall not confer upon any Person other than the parties any rights hereunder.

(f) Assignment; Binding Effect. Neither this Agreement nor any of the rights, interests or obligations hereunder shall be assigned by any of the parties (whether by operation of law or otherwise) without the prior written consent of the other parties, except that the Backstop Investors may assign its rights and interests hereunder to any of its subsidiaries or to any entity that will acquire any or all of any of the Backstop Investors’ assets by merger, stock purchase, asset purchase or otherwise. Subject to the preceding sentence, this Agreement shall be binding upon and shall inure to the benefit of the parties and their respective successors and permitted assigns. Any purported assignment not permitted under this Section 5(f) shall be null and void.

6

(g) Amendments; Waiver. This Agreement may not be amended or supplemented, except by a written agreement executed by the parties. Any party may (A) waive any inaccuracies in the representations and warranties of any other party or extend the time for the performance of any of the obligations or acts of any other party or (B) waive compliance by the other party with any of the agreements contained herein. Notwithstanding the foregoing, no failure or delay by any party hereto in exercising any right hereunder shall operate as a waiver thereof nor shall any single or partial exercise thereof preclude any other or further exercise thereof or the exercise of any other right hereunder. Any agreement on the part of a party to any such extension or waiver shall be valid only if set forth in an instrument in writing signed on behalf of such party.

(h) Severability. If any term or other provision of this Agreement is determined by a court of competent jurisdiction to be invalid, illegal or incapable of being enforced by any rule of law or public policy, all other terms, provisions and conditions of this Agreement shall nevertheless remain in full force and effect. Upon such determination that any term or other provision is invalid, illegal or incapable of being enforced, the parties hereto shall negotiate in good faith to modify this Agreement so as to effect the original intent of the parties as closely as possible to the fullest extent permitted by applicable Law in an acceptable manner to the end that the transactions contemplated hereby are fulfilled to the extent possible.

(i) No Ownership Interest. Nothing contained in this Agreement shall be deemed to vest in any Backstop Investor any direct or indirect ownership or incidence of ownership of or with respect to any Shareholder Shares. All rights, ownership and economic benefits of and relating to the Shareholder Shares shall remain vested in and belong to the applicable Shareholder, and the Backstop Investors shall have any authority to exercise any power or authority to direct Shareholder in the voting of any of the Shareholder Shares, except as otherwise specifically provided herein, or in the performance of a Shareholder’s duties or responsibilities as a shareholder of the Company.

(j) Capacity as Shareholder. The Shareholder has entered into this agreement solely in its capacity as the owner of the Shareholder Shares. Notwithstanding anything to the contrary herein, nothing herein shall limit or affect any actions taken by the Shareholder or any other person in their capacity as an officer or director of the Company, and no action taken in such capacity shall constitute a breach of this Agreement by the Shareholder.

(k) Counterparts. This Agreement may be executed in two or more separate counterparts, each of which shall be deemed to be an original but all of which taken together shall constitute one and the same agreement. This Agreement shall become effective when each party shall have received counterparts hereof signed by the other parties.

(l) Descriptive Headings. Headings of Sections and subsections of this Agreement are for convenience of the parties only, and shall be given no substantive or interpretive effect whatsoever. Except as otherwise indicated, all references in this Agreement to “Sections,” and “Schedules” are intended to refer to Sections of this Agreement and Schedules to this Agreement.

7

(m) Notices. All notices, requests and other communications to any party hereunder shall be in writing (including facsimile transmission) and shall be given,

if to the Company, to:

Trade Street Residential, Inc.

19950 West Country Club Drive, Suite 800

Aventura, Florida 33180

Attention: Chief Executive Officer

Facsimile: (786) 248-3679;

with copy to (which shall not constitute notice):

Morrison & Foerster LLP

2000 Pennsylvania, NW, Suite 6000

Washington, DC 20006-1888

Attention: John A. Good

Facsimile: (202) 785-7522;

if to a Backstop Investor, to:

Senator Investment Group LP

510 Madison Ave.

New York, New York 10022

Attention: Evan Gartenlaub, General Counsel

with copy to (which shall not constitute notice):

Schulte Roth & Zabel LLP

919 Third Avenue

New York, New York 10022

Attention: Eleazer N. Klein

Facsimile: (212) 593-5955;

if to the Shareholder, as stated in Schedule I hereto

or such other address or facsimile number as such party may hereafter specify for the purpose by notice to the other parties. All such notices, requests and other communications shall be deemed received on the date of receipt by the recipient thereof if received prior to 5:00 P.M. in the place of receipt and such day is a business day in the place of receipt. Otherwise, any such notice, request or communication shall be deemed not to have been received until the next succeeding business day in the place of receipt.

(n) Drafting. The parties have participated jointly in the negotiation and drafting of this Agreement and, in the event an ambiguity or question of intent or interpretation arises, this Agreement shall be construed as jointly drafted by the parties and no presumption or burden of proof shall arise favoring or disfavoring any party by virtue of the authorship of any provision of this Agreement.

8

(o) Governing Law; Enforcement; Jurisdiction.

(i) THIS AGREEMENT SHALL BE GOVERNED BY AND CONSTRUED IN ACCORDANCE WITH THE LAWS OF THE STATE OF NEW YORK.

(ii) The parties hereto agree that any suit, action or proceeding seeking to enforce any provision of, or based on any matter arising out of or in connection with, this Agreement or the transactions contemplated hereby may only be brought in the United States District Court for the Southern District of New York, and each of the parties hereby consents to the jurisdiction of such courts (and of the appropriate appellate courts therefrom) in any such suit, action or proceeding and irrevocably waives, to the fullest extent permitted by Law, any objection which it may now or hereafter have to the laying of the venue of any such suit, action or proceeding in any such court or that any such suit, action or proceeding which is brought in any such court has been brought in an inconvenient forum. Process in any such suit, action or proceeding may be served on any party anywhere in the world, whether within or without the jurisdiction of any such court.

(iii) EACH OF THE PARTIES HERETO HEREBY IRREVOCABLY WAIVES ANY AND ALL RIGHT TO TRIAL BY JURY IN ANY LEGAL PROCEEDING ARISING OUT OF OR RELATED TO THIS AGREEMENT OR THE TRANSACTIONS CONTEMPLATED HEREBY.

(iv) The parties agree that irreparable damage would occur in the event that any of the provisions of this Agreement were not performed in accordance with their specific terms or were otherwise breached. It is accordingly agreed that the parties shall be entitled, without derogating from the applicable court’s general discretion to decline granting equitable relief, to an injunction or injunctions to prevent breaches of this Agreement and to enforce specifically the terms and provisions of this Agreement in the United States District Court for the Southern District of New York, this being in addition to any other remedy to which they are entitled at law or in equity.

[REMAINDER OF THIS PAGE INTENTIONALLY LEFT BLANK]

9

IN WITNESS WHEREOF, the parties have caused this Agreement to be duly executed as of the date first above written.

Schedule I