Exhibit 10.4

NOTICE OF CONFIDENTIALITY RIGHTS: IF YOU ARE A NATURAL PERSON, YOU MAY REMOVE OR STRIKE ANY OR ALL OF THE FOLLOWING INFORMATION FROM ANY INSTRUMENT THAT TRANSFERS AN INTEREST IN REAL PROPERTY BEFORE IT IS FILED FOR RECORD IN THE PUBLIC RECORDS: YOUR SOCIAL SECURITY NUMBER OR YOUR DRIVER’S LICENSE NUMBER.

Cross Reference to:

Deed of Trust recorded under

Document No. 2012049828 and

Assignment of Leases and Rents

recorded under Document No.

2012049829, Official Public Records

of Travis County records

HC-2501 W WILLIAM CANNON DR, LLC,

a Delaware limited liability company,

as Grantor

AND

KEYBANK NATIONAL ASSOCIATION,

a national banking association,

as Agent for Lenders,

as Agent

FIRST AMENDMENT TO DEED OF TRUST, SECURITY AGREEMENT AND

ASSIGNMENT OF LEASES AND RENTS AND ASSIGNMENT OF LEASES AND

RENTS

WHEN RECORDED, RETURN TO:

McKenna Long & Aldridge LLP

303 Peachtree Street, N.E., Suite 5300

Atlanta, Georgia 30308

Attention: Brian T. Holmes, Esq.

FIRST AMENDMENT TO DEED OF TRUST, SECURITY AGREEMENT AND

ASSIGNMENT OF LEASES AND RENTS AND ASSIGNMENT OF LEASES AND

RENTS

THIS FIRST AMENDMENT TO DEED OF TRUST, SECURITY AGREEMENT AND ASSIGNMENT OF LEASES AND RENTS AND ASSIGNMENT OF LEASES AND RENTS (this “Amendment”) is made as of the 29^th day of June, 2012, by and between HC-2501 W WILLIAM CANNON DR, LLC, a Delaware limited liability company (“Grantor”), having a mailing address of 4211 W. Boy Scout Boulevard, #500, Tampa, Florida 33607, and KEYBANK NATIONAL ASSOCIATION, a national banking association (“KeyBank”), having a mailing address of 4900 Tiedeman Road, Brooklyn, Ohio 44144, Attn: Real Estate Capital Services, with a copy to KeyBank National Association, 1200 Abernathy Road, N.E., Suite 1550, Atlanta, Georgia 30328, Attn: Daniel Stegemoeller, as Agent (KeyBank, in its capacity as Agent, is hereinafter referred to as “Agent”) for itself and each other lender (collectively, the “Lenders”) which is or may hereafter become a party to the “Credit Agreement” (as hereinafter defined).

RECITALS

WHEREAS, Carter/Validus Operating Partnership, LP, a Delaware limited partnership (“Borrower”), KeyBank, as Agent and the Lenders from time to time a party thereto entered into that certain Credit Agreement dated as of March 30, 2012, as amended by that certain First Amendment to Credit Agreement dated as of May 8, 2012, and as further amended by that certain Second Amendment to Credit Agreement dated as of even date herewith (as the same may be further varied, amended, restated, renewed, consolidated, extended or otherwise supplemented from time to time, the “Credit Agreement”);

WHEREAS, in connection with the Credit Agreement, Grantor entered into that certain Deed of Trust, Security Agreement and Assignment of Leases and Rents dated as of March 30, 2012, and recorded under Document No. 2012049828 with the County Clerk, Travis County, Texas (the “Deed of Trust”) encumbering certain real property located in Travis County, Texas more particularly described on Exhibit “A” attached hereto and made a part hereof and certain personal property related thereto (said real and personal property, the “Property”);

WHEREAS, in connection with the Credit Agreement, Grantor entered into that certain Assignment of Leases and Rents dated as of March 30, 2012, and recorded under Document No. 2012049829 with the County Clerk, Travis County, Texas (the “Assignment of Leases”);

WHEREAS, in connection with an increase of the Total Commitment from $30,000,000.00 to $40,000,000.00 under the Credit Agreement, the parties hereto desire to amend the Deed of Trust and the Assignment of Leases to reflect said increase.

NOW, THEREFORE, for and in consideration of the sum of TEN and NO/100 DOLLARS ($10.00), and other good and valuable consideration, the receipt and sufficiency of which is hereby acknowledged, the parties hereto do hereby covenant and agree as follows:

1. Definitions. Capitalized terms used herein that are not otherwise defined herein shall have the meanings set forth in the Credit Agreement.

2. Modification of the Deed of Trust. Grantor and Agent do hereby modify and amend the Deed of Trust by deleting in its entirety subparagraph (a) appearing on page 6 of the Deed of Trust, and inserting in lieu thereof the following subparagraph (a):

        “(a) The debt evidenced by (i) those certain Revolving Credit Notes made by Borrower in the aggregate principal amount of Forty Million and No/100 Dollars ($40,000,000.00) to the order of Lenders, and (ii) that certain Swing Loan Note made by Borrower in the principal amount of Ten Million and No/100 Dollars ($10,000,000.00) to the order of KeyBank, each of which has been issued pursuant to the Credit Agreement and each of which is due and payable in full on or before March 30, 2015, unless extended as provided in the Credit Agreement; and (iii) each other note as may be issued under the Credit Agreement, each as originally executed, or if varied, extended, supplemented, consolidated, amended, replaced, renewed, modified or restated from time to time as so varied, extended, supplemented, consolidated, amended, replaced, renewed, modified or restated (collectively, the “Note”);”

3. Modification of the Assignment of Leases. Grantor and Agent do hereby modify and amend the Assignment of Leases by deleting in its entirety subparagraph (a) appearing on page 2 of the Assignment of Leases, and inserting in lieu thereof the following subparagraph (a):

        “(a) The debt evidenced by (i) those certain Revolving Credit Notes made by Borrower in the aggregate principal amount of Forty Million and No/100 Dollars ($40,000,000.00) to the order of Lenders, and (ii) that certain Swing Loan Note made by Borrower in the principal amount of Ten Million and No/100 Dollars ($10,000,000.00) to the order of KeyBank, each of which has been issued pursuant to the Credit Agreement and each of which is due and payable in full on or before March 30, 2015, unless extended as provided in the Credit Agreement, and (iii) each other note as may be issued under the Credit Agreement, each as originally executed, or if varied, extended, supplemented, consolidated, amended, replaced, renewed, modified or restated from time to time, as so varied, extended, supplemented, consolidated, amended, replaced, renewed, modified or restated (collectively, the “Note”);”

4. References to Deed of Trust and Assignment of Leases. All references in the Loan Documents to the Deed of Trust and the Assignment of Leases shall be deemed a reference to the Deed of Trust and Assignment of Leases, as modified and amended herein.

5. Ratification. Except as hereinabove set forth, all terms, covenants, and provisions of the Deed of Trust and the Assignment of Leases remain unaltered and in full force and effect, and the parties hereto do hereby expressly ratify and confirm the Deed of Trust and the Assignment of Leases as modified and amended herein. Nothing in this Amendment or in any other document executed in connection herewith shall be deemed or construed to constitute, and there has not otherwise occurred, a novation, cancellation, satisfaction, release, extinguishment, or substitution of the indebtedness evidenced by the Notes or the other obligations of Grantor under the Loan Documents.

3

6. Amendment as Loan Document. This Amendment shall constitute a Loan Document.

7. Counterparts. This Amendment may be executed in any number of counterparts which shall together constitute but one and the same agreement.

8. ENTIRE AGREEMENT. THIS AMENDMENT AND THE OTHER LOAN DOCUMENTS (A) INTEGRATE ALL THE TERMS AND CONDITIONS SET FORTH IN OR INCIDENTAL TO THE LOAN DOCUMENTS, (B) SUPERSEDE ALL ORAL NEGOTIATIONS AND PRIOR WRITINGS WITH RESPECT TO THE SUBJECT MATTER HEREOF, AND (C) ARE INTENDED BY THE PARTIES AS THE FINAL EXPRESSION OF THE AGREEMENT WITH RESPECT TO THE TERMS AND CONDITIONS SET FORTH IN THE LOAN DOCUMENTS AND AS THE COMPLETE AND EXCLUSIVE STATEMENT OF THE TERMS AGREED TO BY THE PARTIES. IN THE EVENT OF ANY CONFLICT BETWEEN THE TERMS, CONDITIONS AND PROVISION OF THIS AMENDMENT AND ANY OTHER LOAN DOCUMENTS, THE TERMS, CONDITIONS AND PROVISIONS OF THIS AMENDMENT SHALL PREVAIL.

9. Miscellaneous. THIS AMENDMENT SHALL BE CONSTRUED AND ENFORCED IN ACCORDANCE WITH THE LAWS OF THE STATE WHERE THE PROPERTY IS LOCATED. Grantor shall pay all reasonable fees and expenses of Agent’s counsel in connection with the execution and delivery of this Amendment. This Amendment shall be binding upon and shall inure to the benefit of the parties hereto and their respective permitted successors, successors-in-title and assigns as provided in the Credit Agreement.

[Remainder of Page Intentionally Left Blank]

4

IN WITNESS WHEREOF, the parties hereto have hereto set their hands and affixed their seals as of the day and year first above written.

ACKNOWLEDGMENT

This instrument was acknowledged before me on                     , 2012, by                     , as                      of Carter Validus Mission Critical REIT, Inc., a Maryland corporation, which is the general partner of Carter/Validus Operating Partnership, LP, a Delaware limited partnership, which is the sole member of HC-2501 W WILLIAM CANNON DR, LLC, a Delaware limited liability company, on behalf of said limited liability company.

[Signatures Continue on Next Page]

[First Amendment to Deed of Trust and Assignment of Leases

Travis County, TX]

This instrument was acknowledged before me on                     , 2012, by                     , as                      of KeyBank National Association, a national banking association, on behalf of said national banking association.

[First Amendment to Deed of Trust and Assignment of Leases

Travis County, TX]

EXHIBIT “A”

Units 301, 401, 501, STONEGATE COMMONS OFFICE CONDOMINIUMS, a condominium project in Travis County, Texas, according to the Declaration of Covenants, Conditions and Restrictions recorded under Document Nos. 2004233570 and amended under Document Nos. 2005050752, 2005170928, 2005229466, 2006110880, and 2006240381, all of the Official Public Records of Travis County, Texas, together with an undivided interest in and to the common elements appurtenant thereto.

A-1