Exhibit 5.1

December 9, 2009

(212) 351-4035

Capital One Financial Corporation

1680 Capital One Drive

McLean, Virginia 22102

Registration Statement on Form S-3 (Registration No. 333-159085)

Warrants to Purchase Common Stock of Capital One Financial Corporation

Ladies and Gentlemen:

We have examined: (i) the Registration Statement on Form S-3 (Registration No. 333-159085) filed with the Securities and Exchange Commission (the “Commission”) under the Securities Act of 1933, as amended (the “Securities Act”), on May 8, 2009 of Capital One Financial Corporation (the “Company”) for the registration of the Company’s senior debt securities, subordinated debt securities, junior subordinated debt securities, preferred stock, depositary shares, common stock, purchase contracts, warrants and units (the “Registration Statement”); (ii) the prospectus dated May 8, 2009 forming a part thereof, together with the documents incorporated therein by reference; (iii) the preliminary prospectus supplement dated December 1, 2009, in the form filed with the Commission pursuant to Rule 424(b) under the Securities Act on December 1, 2009; and (iv) the final prospectus supplement dated December 3, 2009 in the form filed with the Commission pursuant to Rule 424(b) under the Securities Act on December 4, 2009 in connection with the offering and sale by the United States Department of the Treasury (“Treasury”) of 12,657,960 warrants issued November 14, 2008 (the “Warrants”), representing the right to purchase an aggregate of up to that same number of shares (the “Warrant Shares”) of the Company’s common stock, par value $0.01 per share.

We have examined originals, or photostatic or certified copies, of such records of the Company and certificates of officers of the Company and of public officials and such documents as we have deemed relevant and necessary as the basis for the opinion set forth below. In our examination, we have assumed the genuineness of all signatures, the legal capacity and competency of all natural persons, the authenticity of all documents submitted to us as originals and the conformity to original documents of all documents submitted to us as copies. As to any facts material to this opinion, we have relied to the extent

Capital One Financial Corporation

December 9, 2009

Page 2

we deemed appropriate and without independent investigation upon statements and representations of officers and other representatives of the Company and others.

Based upon the foregoing examination and in reliance thereon, and subject to the assumptions stated and in reliance on statements of fact contained in the documents that we have examined, we are of the opinion that the Warrants are legal, valid and binding obligations of the Company, enforceable in accordance with their terms, and that when issued, delivered to and paid for by the exercising holder of the Warrants in accordance with the terms of the Warrants, the Warrant Shares will be validly issued, fully paid and non-assessable.

We render no opinion herein as to matters involving the laws of any jurisdiction other than the State of New York, the General Corporation Law of the State of Delaware (the “DGCL”) and the United States of America. This opinion is limited to the current laws of the State of New York, the DGCL and the current federal laws of the United States, and to the current judicial interpretations thereof and to the facts as they exist on the date hereof. We are not admitted to practice law in the State of Delaware, but we are generally familiar with the DGCL, as currently in effect, and have made such inquiries as we considered necessary to render our opinion. We assume no obligation to revise or supplement our opinion should the present laws, or the interpretation thereof, be changed in respect of any circumstances or events that occur subsequent to the date hereof.

We consent to the filing of this opinion as an exhibit to the Company’s Current Report on Form 8-K filed with the Commission on December 9, 2009, and we further consent to the use of our name under the caption “Validity of the Common Stock” in the Registration Statement and the prospectus that forms a part thereof. In giving these consents, we do not thereby admit that we are within the category of persons whose consent is required under Section 7 of the Securities Act or the Rules and Regulations of the Commission.

Very truly yours,

/s/ Gibson, Dunn & Crutcher LLP