Exhibit 5.1

March 20, 2012

Capital One Financial Corporation

1680 Capital One Drive

McLean, Virginia 22102

Ladies and Gentlemen:

We have examined the Registration Statement on Form S-3, File No. 333-159085 (the “Registration Statement”), of Capital One Financial Corporation, a Delaware corporation (the “Company”), filed with the Securities and Exchange Commission (the “Commission”) pursuant to the Securities Act of 1933, as amended (the “Securities Act”), in connection with the offering by the Company of 24,442,706 shares of the Company’s common stock, par value $0.01 per share (the “Shares”).

We have examined the originals, or photostatic or certified copies, of such records of the Company and certificates of officers of the Company and of public officials and such other documents as we have deemed relevant and necessary as the basis for the opinions set forth below. In our examination, we have assumed the genuineness of all signatures, the legal capacity and competency of all natural persons, the authenticity of all documents submitted to us as originals and the conformity to original documents of all documents submitted to us as copies.

Based upon the foregoing examination and in reliance thereon, and subject to the assumptions stated and in reliance on statements of fact contained in the documents that we have examined, we are of the opinion that the Shares, when issued against payment therefor, will be validly issued, fully paid and non-assessable.

We consent to the filing of this opinion as an exhibit to the Registration Statement, and we further consent to the use of our name under the captions “Validity of the Common Stock” and “Certain Legal Matters” in the Registration Statement and the prospectus that forms a part thereof. In giving these consents, we do not thereby admit that we are within the category of persons whose consent is required under Section 7 of the Securities Act or the Rules and Regulations of the Commission.

Very truly yours,

/s/ Gibson, Dunn & Crutcher LLP