Exhibit 99.4

FORM OF LETTER TO NOMINEE HOLDERS WHOSE

CLIENTS ARE BENEFICIAL HOLDERS

RUTH’S HOSPITALITY GROUP, INC.

Common Stock, par value $0.01 per share

Offered Pursuant to Subscription Rights

Distributed to Stockholders

of Ruth’s Hospitality Group, Inc.

January 21, 2010

To Securities Dealers, Commercial Banks,

    Trust Companies and Other Nominees:

This letter is being distributed to securities dealers, commercial banks, trust companies and other nominees in connection with the offering (the “Rights Offering”) by Ruth’s Hospitality Group, Inc. (the “Company”) of shares of the Company’s common stock, par value $0.01 per share (the “Common Stock”), pursuant to transferable subscription rights (the “Rights”) distributed to all holders of record (“Recordholders”) of shares of Common Stock at 5:00 p.m., New York City time, on January 20, 2010 (the “Record Date”). The Rights and Common Stock are described in the Company’s prospectus supplement dated January 21, 2010 (the “Prospectus Supplement”).

In the Rights Offering, the Company is offering an aggregate of approximately 14,000,000 shares of Common Stock, as described in the Prospectus Supplement.

The Rights will expire, if not exercised, at 5:00 p.m., New York City time, on February 9, 2010, unless extended in the sole discretion of the Company (as it may be extended, the “Expiration Date”), as described in the Prospectus Supplement.

As described in the Prospectus Supplement, each beneficial owner of Common Stock registered in your name or in the name of your nominee is entitled to one Right for each share of Common Stock owned by such beneficial owner as of 5:00 p.m., New York City time, on the Record Date.

Each Right entitles the holder thereof to purchase 0.579232 shares of Common Stock at the cash price of $2.50 per share (the “Subscription Price”) and consists of a basic subscription right and an oversubscription right. The basic subscription right entitles the holder thereof to purchase 0.579232 shares of Common Stock at the Subscription Price for each Right held. If the holder exercises the basic subscription right in full, the oversubscription right entitles the holder thereof to purchase, at the Subscription Price, additional shares of Common Stock that are offered but not purchased by other subscription right holders; provided that the Company may, but will not be obligated to, sell any shares of Common Stock pursuant to the exercise of an oversubscription right to the extent the Company raises more than $25.0 million in gross proceeds through the exercise of basic subscription rights and, if applicable, the exercise of oversubscription rights. If oversubscription requests exceed shares available, your client will receive a pro rata allocation of the available shares based on the number of shares your client purchased under its basic subscription right. Any excess subscription payments will be returned, without interest or penalty, as soon as practicable after the completion of the Rights Offering.

Your clients will not receive fractional shares of Common Stock or cash in lieu of fractional shares of Common Stock as a result of their exercise of Rights pursuant to the Rights Offering. Your clients may only exercise their subscription rights to purchase, at the subscription price, a whole number of shares of Common

Stock, rounded down to the nearest whole number your clients are otherwise entitled to purchase. For example, if your client owned 100 shares of Common Stock as of 5:00 p.m., New York City time, on the Record Date, your client would receive 100 Rights, which would entitle your client to purchase 57 shares (57.9232 rounded down to the nearest whole share) at the subscription price of $2.50 per share through the exercise of basic subscription rights.

The Common Stock is traded on the NASDAQ Global Select Market (“NASDAQ”) under the symbol “RUTH.” The Rights are transferable, and are listed on NASDAQ under the symbol “RUTHR.” The Rights will be evidenced by Rights certificates (the “Rights Certificates”), which will be transferable until the close of business on the last NASDAQ trading day preceding the Expiration Date, at which time they will cease to have value for trading purposes.

We are asking persons who hold shares of Common Stock beneficially and who have received the Rights distributable with respect to those shares through a broker, dealer, commercial bank, trust company or other nominee, as well as persons who hold certificates of Common Stock directly and prefer to have such institutions effect transactions relating to the Rights on their behalf, to contact the appropriate institution or nominee and request it to effect the transactions for them. In addition, we are asking beneficial owners who wish to obtain a separate Rights Certificate to contact the appropriate nominee as soon as possible and request that a separate Rights Certificate be issued.

The Company will pay all fees charged by the Subscription Agent, other than fees associated with the sale or transfer of Rights, and the fees charged by the Information Agent. You are responsible for paying any other commissions, fees, taxes or other expenses incurred in connection with the exercise of the Rights. Neither we nor the Subscription Agent nor the Information Agent will pay such expenses.

Enclosed are copies of the following documents:

1. Prospectus Supplement;

2. Instructions as to Use of Ruth’s Hospitality Group, Inc. Subscription Rights Certificates (including a Notice of Guaranteed Delivery for Rights Certificates Issued by Ruth’s Hospitality Group, Inc.);

3. A form of letter which may be sent to your clients for whose accounts you hold shares of Common Stock registered in your name or the name of your nominee, with an attached form of instruction;

4. Notice of Guaranteed Delivery for Rights Certificates Issued by Ruth’s Hospitality Group, Inc.;

5. Nominee Holder Certificate for use if exercising rights for more than one client; and

6. A return envelope addressed to American Stock Transfer & Trust Company, LLC, the Subscription Agent.

Your prompt action is requested. To exercise Rights, you should deliver the properly completed and signed Rights Certificate (or Notice of Guaranteed Delivery if you are following the Guaranteed Delivery Procedures), with payment in full of the Subscription Price for each share of Common Stock subscribed for, to the Subscription Agent, as indicated in the Prospectus Supplement. The Subscription Agent must receive the Rights Certificate or Notice of Guaranteed Delivery with payment of the Subscription Price, including final clearance of any checks, prior to 5:00 p.m., New York City time, on the Expiration Date. FAILURE TO RETURN THE PROPERLY COMPLETED RIGHTS CERTIFICATE WITH THE CORRECT PAYMENT WILL RESULT IN YOUR NOT BEING ABLE TO EXERCISE YOUR RIGHTS. A Rights holder cannot revoke the exercise of its Rights unless the terms of the Rights Offering are amended in a material manner. Rights not exercised prior to the Expiration Date will expire.

2

Additional copies of the enclosed materials may be obtained from Georgeson Inc., the Information Agent. The Information Agent’s telephone number is (800) 676-0281. Any questions or requests for assistance concerning the Rights Offering should be directed to the Information Agent.

NOTHING IN THE PROSPECTUS SUPPLEMENT OR IN THE ENCLOSED DOCUMENTS SHALL CONSTITUTE YOU OR ANY PERSON AS AN AGENT OF RUTH’S HOSPITALITY GROUP, INC., THE SUBSCRIPTION AGENT OR ANY OTHER PERSON MAKING OR DEEMED TO BE MAKING OFFERS OF THE SECURITIES ISSUABLE UPON VALID EXERCISE OF THE RIGHTS, OR AUTHORIZE YOU OR ANY OTHER PERSON TO MAKE ANY STATEMENTS ON BEHALF OF ANY OF THEM WITH RESPECT TO THE RIGHTS OFFERING EXCEPT FOR STATEMENTS CONTAINED OR INCORPORATED BY REFERENCE INTO THE PROSPECTUS SUPPLEMENT AND THE ACCOMPANYING PROSPECTUS, OR IN ANY RELATED FREE WRITING PROSPECTUS THAT HAS BEEN OR WILL BE FILED BY THE COMPANY OR ON THE COMPANY’S BEHALF WITH THE SECURITIES AND EXCHANGE COMMISSION.

3