Exhibit 99.1

CASUAL MALE RETAIL GROUP, INC.

REPORTS FIRST QUARTER 2011 RESULTS

First Quarter Earnings $0.09 Per Diluted Share on comparable sales increase of 2.2%

CANTON, MA (May 19, 2011) — Casual Male Retail Group, Inc. (NASDAQ: “CMRG”), the largest retailer of big & tall men’s apparel and accessories, today reported operating results for the first quarter ended April 30, 2011.

First Quarter Highlights (1QFY11 vs. 1QFY10)

Sales

For the first quarter of fiscal 2011, total sales increased 0.9% to $95.8 million and comparable sales increased 2.2% as compared to the prior year’s first quarter. During the first quarter of fiscal 2011, comparable sales for our Retail channel increased 1.6% while comparable sales for our Direct channel increased 4.7%.

For the first quarter of fiscal 2011, comparable sales for our northeast and midwest regions were over 3% lower than comparable sales for our southern and western regions, principally due to adverse weather in those regions. As a result, comparable sales for the first quarter of fiscal 2011 were 1.5% lower than expected.

Gross Profit Margin

For the first quarter of fiscal 2011, gross margin increased 100 basis points to 46.9%. The increase was the result of a 90 basis point improvement in merchandise margin and a 10 basis point improvement in occupancy costs.

SG&A

On a dollar basis, SG&A expenses increased 4.2% to $37.1 million for the first quarter of fiscal 2011. As a percentage of sales, SG&A expenses increased 120 basis points to 38.7% in the first quarter of fiscal 2011 as compared to 37.5% in the first quarter of fiscal 2010.

As planned, our SG&A expenses for the first quarter of fiscal 2011 increased by $1.5 million, primarily due to payroll-related costs, including bonus accruals, modest salary increases, severance payments and 401k matching contributions.

Interest Expense

Net interest expense of $0.1 million for the first quarter of fiscal 2011 was flat with the prior year. Net interest expense for the first quarter of fiscal 2011 was primarily unused line fees because the Company had essentially no borrowings during the quarter.

Cash Flow

Free cash flow from operations (as defined below under Non-GAAP Measures) improved by $1.3 million to $1.2 million in the first quarter of fiscal 2011 as compared to $(0.1) million in the first quarter of fiscal 2010.

Balance Sheet & Liquidity

At April 30, 2011, the Company had no outstanding debt as compared to $12.0 million at end of the first quarter of fiscal 2010. In addition, the Company had full availability under its credit facility of $66.9 million.

Inventories decreased 1.1% to $97.6 million at the end of the first quarter of fiscal 2011 from $98.7 million at the end of first quarter of fiscal 2010.

Destination XL

The Company now expects to open 14-15 additional Destination XL™ stores during the second half of fiscal 2011. This is an increase from previously discussed 10-14 stores guidance for 2011. In addition, the Company anticipates opening 20–30 Destination XL stores in fiscal 2012. Depending on the real estate and market demographics for each of these store locations, we expect the size of each store to be between 6,000 to 12,000 square feet, to accommodate each market.

Fiscal 2011 Outlook

For the fiscal year ending January 28, 2012, the Company’s outlook for the year is unchanged from previous guidance, inclusive of the opening of 14-15 DXL™ stores:

Non-GAAP Measures

In addition to the financial measures prepared in accordance with generally accepted accounting principles (GAAP), the above discussion also refers to non-GAAP free cash flow of $1.2 million and $(0.1) million for the three months ended April 30, 2011 and May 1, 2010, respectively, and estimated non-GAAP free cash flow of $16.0 million for fiscal 2011. The presentation of non-GAAP free cash flow is not a measure determined by GAAP and should not be considered superior

to or as a substitute for net income or cash flows from operating activities or any other measure of performance derived in accordance with GAAP. In addition, all companies do not calculate non-GAAP financial measures in the same manner and, accordingly, "free cash flows" presented in this release may not be comparable to similar measures used by other companies. The Company calculates free cash flows as cash flow from operating activities less capital expenditures and less discretionary store asset acquisitions.

Investors are invited to listen to a broadcast of the Company’s conference call to discuss its earnings results for the first quarter of fiscal 2011. The conference call will broadcast live today, Thursday, May 19, 2011 at 9:00 a.m. Eastern Daylight Time and can be accessed at www.casualmalexl.com and then clicking on the investor relations icon. The call will be archived online within one hour after its completion. Participating in the call will be David Levin, President and Chief Executive Officer, and Dennis Hernreich, Executive Vice President, Chief Operating Officer and Chief Financial Officer.

During the conference call, the Company may discuss and answer questions concerning business and financial developments and trends. The Company's responses to questions, as well as other matters discussed during the conference call, may contain or constitute information that has not been disclosed previously.

Casual Male Retail Group, Inc., the largest retailer of big and tall men's apparel with operations throughout the United States, Canada and Europe, operates 439 Casual Male XL retail and outlet stores, 4 Destination XL stores, 16 Rochester Clothing stores, and direct-to-consumer businesses which include several catalogs and e-commerce sites. The Company is headquartered in Canton, Massachusetts, and its common stock is listed on the NASDAQ Global Market under the symbol "CMRG."

Certain information contained in this press release, including the Company's expectations regarding fiscal 2011, constitutes forward-looking statements under the federal securities laws. The discussion of forward-looking information requires management of the Company to make certain estimates and assumptions regarding the Company's strategic direction and the effect of such plans on the Company's financial results. The Company's actual results and the implementation of its plans and operations may differ materially from forward-looking statements made by the Company. The Company encourages readers of forward-looking information concerning the Company to refer to its prior filings with the Securities and Exchange Commission, including without limitation, its Annual Report on Form 10-K filed on March 18, 2011, that set forth certain risks and uncertainties that may have an impact on future results and direction of the Company.

Forward-looking statements contained in this press release speak only as of the date of this release. Subsequent events or circumstances occurring after such date may render these statements incomplete or out of date. The Company undertakes no obligation and expressly disclaims any duty to update such statements.

For additional information, contact:

Jeff Unger

Vice President of Investor Relations

Casual Male Retail Group, Inc.

561-482-9715

jeffunger@usa.net

[tables to follow]

CASUAL MALE RETAIL GROUP, INC.

CONSOLIDATED STATEMENTS OF OPERATIONS

(In thousands, except per share data)

CASUAL MALE RETAIL GROUP, INC.

CONSOLIDATED BALANCE SHEETS

April 30, 2011 and January 29, 2011

(In thousands)