Exhibit 5.1

Via Email and Overnight Delivery

B. Riley & Co., LLC

4675 MacArthur Court Suite 1500

Newport Beach, CA 92660

Ladies and Gentlemen:

We have acted as counsel to BofI Holding, Inc., a Delaware corporation (the “Company”), in connection with the sale to you of an aggregate of 1,226,276 shares (the “Shares”) of common stock, par value $0.01 per share, of the Company pursuant to the terms of the Underwriting Agreement, dated March 30, 2010 (the “Underwriting Agreement”), by and between the Company and you. This opinion is being delivered to you pursuant to Section 6(e) of the Underwriting Agreement. Capitalized terms used herein have the respective meanings ascribed thereto in the Underwriting Agreement unless otherwise defined herein.

In connection with this opinion, we have examined and relied upon the originals, or copies certified to our satisfaction, of such records, documents, certificates and other instruments as in our judgment are necessary or appropriate to enable us to render the opinion expressed below. As to certain factual matters, we have relied upon the certificates and representations of the officers of the Company provided in connection with the offering of the Shares, including the copy of the certificate attached hereto, and have not sought to independently verify such matters.

In rendering this opinion, we have assumed the genuineness and authenticity of all signatures on original documents; the authenticity of all documents submitted to us as originals; the conformity to originals of all documents submitted to us as copies; the accuracy, completeness and authenticity of certificates of public officials; and the due authorization,

B. Riley & Co., LLC

April 5, 2010

Page 2

Our File No.: 6322-2

execution and delivery of all documents by parties other than the Company, where authorization, execution and delivery are prerequisites to the effectiveness of such documents.

Whenever a statement herein is qualified by “known to us,” “to our knowledge” or similar phrase, it is intended to indicate that, during the course of our representation of the Company, no information that would give us current actual knowledge of the inaccuracy of such statement has come to the attention of those attorneys in this firm who have rendered legal services in connection with the representation described in the introductory paragraph of this opinion letter. We have not, however, undertaken any further independent investigation to determine the accuracy of any such statement, and you acknowledge we have no duty to make such investigation. Any limited inquiry undertaken by us during the preparation of this opinion letter should not be regarded as such an investigation; no inference as to our knowledge of any matters bearing on the accuracy of any such statement should be drawn from the fact of our representation of the Company in connection with this transaction.

We are members of the bar of the State of California. We do not express any opinion herein on any laws other than the laws of the State of California, the General Corporation Law of the State of Delaware and the federal law of the United States of America. Our opinion is based on these laws as in effect on the date hereof. We express no opinion as to whether the laws of any jurisdiction are applicable to the subject matter hereof.

Based upon the foregoing, and in reliance thereon, and subject to the qualifications herein stated, we are of the opinion that:

1. The Company is a corporation validly existing and in good standing under the laws of the State of Delaware, with full corporate power to own, lease and operate its properties and conduct its business as described in the Registration Statement, the General Disclosure Package and the Prospectus, to execute and deliver the Underwriting Agreement and to issue, sell and deliver the Shares as contemplated therein.

2. The Company is duly qualified to do business as a foreign corporation in the State of California. To our knowledge, the State of California is the only state within the United States in which the Company owns any material property or conducts any material business and in which the Company is required to be qualified as a foreign corporation.

3. The Company’s authorized capital consists of (a) 25,000,000 shares of Common Stock, par value $ .01 per share and (b) 1,000,000 shares of Preferred Stock, par value $ .01 per share.

B. Riley & Co., LLC

April 5, 2010

Page 3

Our File No.: 6322-2

4. The Underwriting Agreement has been duly authorized, executed and delivered by the Company.

5. The Shares have been duly authorized and, when issued and delivered upon payment therefor in accordance with the terms of the Underwriting Agreement, will be validly issued, fully paid and non-assessable.

6. The Shares are free of statutory and, to our knowledge, contractual preemptive rights, resale rights, rights of first refusal and restrictions upon voting or transfer.

7. Each of the Registration Statement, the Base Prospectus included in the General Disclosure Package and the Prospectus (except as to the financial statements and schedules and other financial data contained therein, as to which we express no opinion), as of its date, complied as to form in all material respects with the requirements of the Securities Act. To our knowledge, the Company is not a party to any contract, agreement or document of a character that is required to be filed as an exhibit to, or incorporated by reference in, the Registration Statement or described in the Registration Statement or the Prospectus that has not been so filed, incorporated by reference or described as required.

8. The Registration Statement was declared effective under the Securities Act on January 7, 2010. The Prospectus Supplement was filed by the Company with the Commission pursuant to Rule 424(b) under the Securities Act on April 1, 2010. To our knowledge, no stop order proceedings with respect thereto are pending or threatened under the Securities Act and any required filing of the Prospectus and any supplement thereto pursuant to Rule 424 under the Securities Act has been made in the manner and within the time period required by such Rule 424.

9. Other than the filing of the Prospectus Supplement with the Securities and Exchange Commission pursuant to Rule 424(b) and the filing of a Current Report on Form 8-K disclosing, among other things, the entry into the Underwriting Agreement, no approval, authorization, consent or order of or filing with any governmental authority of the United States of America, or the States of Delaware or California is required in connection with the issuance and sale of the Shares and consummation by the Company of the transactions contemplated in the Underwriting Agreement (except we express no opinion as to any necessary qualification under the state securities or blue sky laws of the various jurisdictions in which the Shares are being offered).

10. The execution and delivery of the Underwriting Agreement by the Company, and the consummation by the Company of the transactions contemplated thereby to be consummated

B. Riley & Co., LLC

April 5, 2010

Page 4

Our File No.: 6322-2

on the date hereof, do not violate (a) the Company’s Certificate of Incorporation or Bylaws, (b) any statute, law, rule or regulation known to us that is applicable to the Company or its property or (c) any judgment, decree or order of any court or any other agency of government known to us that is applicable to the Company or its property.

11. The execution and delivery of the Underwriting Agreement by the Company, and the consummation by the Company of the transactions thereunder, do not and will not conflict with, result in any breach or violation of or constitute a default under (nor constitute any event which with notice, lapse of time or both would result in any breach of or constitute a default under) any indenture, mortgage, deed of trust, bank loan or credit agreement, other evidence of indebtedness, any license, lease, contract or other agreement or instrument included as an exhibit to the Registration Statement or as an exhibit to any other report filed by the Company with the Commission that is incorporated by reference into the Registration Statement.

12. To our knowledge, there are no actions, suits, claims, investigations or proceedings pending against the Company which are required to be described in the Registration Statement, the General Disclosure Package or the Prospectus but are not so described.

13. The information in the Registration Statement and the Prospectus under the headings “Description of Common Stock,” and “Underwriting,” insofar as such statements constitute a summary of documents or matters of law, and those statements in the Registration Statement and the Prospectus that are descriptions of contracts, agreements or other legal documents or of legal proceedings, or refer to statements of law or legal conclusions, are accurate in all material respects and present fairly the information required by the rules of the Commission to be shown.

14. No person has the right, pursuant to the terms of any contract, agreement or other instrument described in or filed as an exhibit to the Registration Statement or otherwise known to us, to cause the Company to register under the Securities Act any shares of Common Stock or shares of any other capital stock or other equity interest of the Company, or to include any such shares or interest in the Registration Statement or the offering contemplated thereby, whether as a result of the filing or effectiveness of the Registration Statement or the sale of the Shares as contemplated thereby or otherwise.

This opinion is solely for your information and is not to be relied on by any other person or quoted in whole or in part or otherwise referred to, nor is it to be filed with any governmental agency or other person without our prior written consent.

B. Riley & Co., LLC

April 5, 2010

Page 5

Our File No.: 6322-2

This opinion is based upon information furnished to us as of the date hereof and upon the laws, regulations, judicial decisions and other legal authority in effect as of the day of this opinion. We undertake no responsibility to update this opinion with respect to matters occurring after the date hereof or with respect to matters which come to our attention after the date hereof.