Exhibit 5.1

July 3, 2017

Immune Design Corp.

1616 Eastlake Ave. E., Suite 310

Seattle, Washington 98102

Ladies and Gentlemen:

You have requested our opinion as counsel to Immune Design Corp., a Delaware corporation (the “Company”), with respect to certain matters in connection with the offering by the Company of $50,000,000 of shares of the Company’s common stock, par value $0.001 per share (the “Shares”), pursuant to a Registration Statement on Form S-3 (No. 333-206324) (the “Registration Statement”), filed with the Securities and Exchange Commission (the “Commission”) under the Securities Act of 1933, as amended (the “Act”), the related prospectus dated December 29, 2015 (the “Base Prospectus”), and the prospectus supplement dated July 3, 2017, filed with the Commission pursuant to Rule 424(b) of the Rules and Regulations of the Act (together with the Base Prospectus, the “Prospectus”). The Shares are to be sold by the Company in accordance with that certain Sales Agreement, dated July 3, 2017, by and between the Company and Cowen and Company, LLC (the “Agreement”).

In connection with this opinion, we have examined and relied upon the Registration Statement, the Prospectus, the Agreement, the Company’s Amended and Restated Certificate of Incorporation and Amended and Restated Bylaws, each as currently in effect, and such other documents, records, certificates, memoranda and other instruments as in our judgment are necessary or appropriate to enable us to render the opinion expressed below. In rendering this opinion, we have assumed the genuineness and authenticity of all signatures on original documents; the genuineness and authenticity of all documents submitted to us as originals; the conformity to originals of all documents submitted to us as copies; and the accuracy, completeness and authenticity of certificates of public officials. As to certain factual matters, we have relied upon a certificate of an officer of the Company and have not sought independently to verify such matters.

We have assumed (i) that each sale of the Shares will be duly authorized by the Board of Directors of the Company, a duly authorized committee thereof or a person or body pursuant to an authorization granted in accordance with Section 152 of the General Corporation Law of the State of Delaware (the “DGCL”), (ii) that no more than 5,128,205 Shares will be sold under the Agreement at a price of not less than $9.75 per share, representing the last reported sale price of the Company’s common stock on the NASDAQ Global Market on June 30, 2017 and (iii) that the price at which the Shares are sold will equal or exceed the par value of the Shares. We express no opinion to the extent that future issuances of securities of the Company and/or anti-dilution adjustments to outstanding securities of the Company cause the number of shares of the Company’s common stock outstanding or issuable upon conversion or exercise of outstanding securities of the Company to exceed the number of Shares then issuable under the Agreement.

Our opinion herein is expressed solely with respect to the DGCL. We express no opinion as to whether the laws of any particular jurisdiction other than those identified above are applicable to

Cooley LLP    3175 Hanover Street    Palo Alto, CA    94304-1130

t: (650) 843-5000    f: (650) 849-7400    cooley.com

Immune Design Corp.

July 3, 2017

Page Two

the subject matter hereof. We are not rendering any opinion as to compliance with any federal or state law, rule or regulation relating to securities, or to the sale or issuance thereof.

On the basis of the foregoing, and in reliance thereon, we are of the opinion that the Shares, when sold and issued against payment therefor in accordance with the Agreement, the Registration Statement and the Prospectus, will be validly issued, fully paid and nonassessable.

We consent to the reference to our firm under the caption “Legal Matters” in the Prospectus and to the filing of this opinion as an exhibit to a Current Report on Form 8-K to be filed with the Commission for incorporation by reference into the Registration Statement.

Our opinion is based on these laws as in effect on the date hereof, and we disclaim any obligation to advise you of facts, circumstances, events or developments which hereafter may be brought to our attention and which may alter, affect or modify the opinion expressed herein.

Very truly yours,

Cooley LLP    3175 Hanover Street    Palo Alto, CA    94304-1130

t: (650) 843-5000    f: (650) 849-7400    cooley.com