Exhibit 99.1

Section 2 – Exhibit 99.1

[FOR IMMEDIATE RELEASE]

First Business Financial Services, Inc.

401 Charmany Drive

Madison, WI 53719

FIRST BUSINESS FINANCIAL SERVICES, INC. REPORTS 64 PERCENT INCREASE IN NET INCOME FOR THE FIRST QUARTER OF 2012

Year-over-Year Results Driven by a 6 Percent Increase in Top Line Revenue and Lower Credit Costs

Madison, WI – April 27, 2012 (GLOBE NEWSWIRE) – First Business Financial Services, Inc. (the “Company”) (NASDAQ: FBIZ), the parent company of First Business Bank and First Business Bank – Milwaukee, today reported strong first quarter results highlighted by growth in top line revenue, a meaningful reduction in non-performing assets, and continued capital strength.

Highlights for the quarter ended March 31, 2012 include:

The Company recorded first quarter 2012 net income of $2.2 million, or diluted earnings per common share of $0.84, compared to net income of $1.3 million, or diluted earnings per common share of $0.52, for the first quarter of 2011.

“The momentum of our efforts to grow top line revenue, generate in-market deposits and carefully manage credit risk drove another quarter of outstanding results for First Business Financial Services,” said Corey A. Chambas, President and Chief Executive Officer. “Our ability to consistently deliver quality earnings has also placed us in the enviable position of being financially and strategically equipped to invest in additional talent as a driver of future growth. We are adding talented individuals with niche commercial expertise today, enhancing our organic revenue and loan growth opportunities for tomorrow. As we remain focused on executing our strategic plan, we are confident the momentum we’ve achieved positions us well for continued growth in long-term shareholder value.”

Core Business Results

Net interest income increased $442,000, or 5.2%, to $8.9 million in the first quarter of 2012, compared to $8.5 million for the first quarter of 2011. The improvement principally resulted from a 45 basis point decline in the average rate paid on interest-bearing deposit balances, combined with a lesser decline in yields on average earning assets. The decrease in overall deposit funding costs was primarily caused by a 55 basis point reduction in the rate paid on brokered certificates of deposit, commensurate with lower current market rates, coupled with a 13% decline in average brokered certificate of deposit balances. The Company was able to actively reduce its overall usage of brokered certificates of deposit for funding as a result of its success in achieving 23.8% year-over-year growth in average in-market client deposits, which include all transaction accounts, money market accounts, and non-brokered certificates of deposit. Accordingly, interest expense for the first quarter decreased 15.7%, or $879,000, to $4.7 million, compared to $5.6 million for the first quarter of 2011. The decline in interest expense was partially offset by lower interest income as a result of lower market rates on securities purchased and lower average loan and lease balances. Net interest margin of 3.15% was down 8 basis points compared with the fourth quarter of 2011, but remained essentially flat as compared to 3.14% reported for the first quarter of 2011.

Non-interest income for the first quarter of 2012 was $1.9 million, an increase of $178,000, or 10.6%, compared with the same period of 2011. Service charges on deposits grew $106,000, or 28.4%, on increased deposit balances and transactions from commercial clients, while increased asset-based lending activity drove growth in loan fees of $67,000, up 20.2%, compared to the first quarter of 2011. Trust and investment services income grew $46,000, or 7.2%, compared to the same quarter of the prior year, primarily benefitting from an increase in assets under management from new client relationships, including a large client transaction executed in the fourth quarter of 2011.

Non-interest expense for the first quarter of 2012 was $6.8 million, representing a modest increase of $72,000, or 1.1%, compared to the same quarter in 2011. Compensation costs of $4.0 million were $268,000, or 7.2%, higher than the first quarter of 2011 due to merit increases on salaries, new positions filled in support of strategic initiatives, elevated social security taxes on non-equity incentive awards, and increased expenses related to the Company’s 401(k) employer match. Significant declines in FDIC insurance costs and collateral liquidation costs helped offset expense growth. FDIC insurance expense declined $172,000, or 22.7%, from the first quarter of 2011 due to the modification of the FDIC’s methodology for calculating an institution’s deposit insurance assessment base. The Company’s success in reducing non-performing loans drove a corresponding reduction of $134,000, or 55.4%, in collateral liquidation costs. Coupled with top line revenue growth of 6.1%, first quarter 2012 cost containment drove an improvement in the efficiency ratio to 61.8%, 423 basis points lower than in the prior year period.

The provision for loan and lease losses for the first quarter of 2012 was $504,000, representing a decline of $900,000, or 64.1%, compared to the same quarter of the prior year, primarily due to a reduction in net charge-offs of $665,000, from $873,000 to $208,000.

Asset Quality Continues to Improve

The ratio of non-performing assets to total assets fell 8 basis points from 2.04% at December 31, 2011 to 1.96% at March 31, 2012. The same measure fell 177 basis points from 3.73% at March 31, 2011. Non-performing assets decreased by 45.1%, or $18.7, million from March 31, 2011 to March 31, 2012, reflecting payoffs, paydowns, charge-offs and improved client performance causing a return to accrual status. These reductions were partially offset by continued additions of newly identified problem loans and leases. Annualized net charge-offs as a percent of average gross loans and leases declined 30 basis points to 0.10% for the three months ended March 31, 2012, compared to 0.40% for the same period of the prior year.

Total assets of $1.2 billion declined modestly by $15.1 million, or 1.3%, from December 31, 2011. Total assets grew $49.2 million, or 4.4%, from March 31, 2011. Compared to December 31, 2011, cash and cash equivalents increased $5.3 million while available-for-sale securities remained relatively flat. Growth in short-term investments was offset by a $19.4 million, or 2.3%, reduction in net loan and lease balances since December 31, 2011. The decline in net loans and leases is attributable to new loan originations being offset by amortization of the existing loan portfolio and the reduction of non-accrual loans and leases.

Dividend Maintained

During the first quarter of 2012 the Company’s Board of Directors approved a $0.07 quarterly cash dividend on its common stock, which was paid on April 15, 2012 to shareholders of record at the close of business on April 1, 2012. This maintained the Company’s annualized dividend at $0.28 per share, a level it has maintained for seventeen consecutive quarters.

Capital Strength

The Company’s capital ratios continue to improve and are in excess of the highest required regulatory benchmark levels. Total capital to risk-weighted assets was 13.56% as of March 31, 2012 as compared to 13.11% at December 31, 2011.

About First Business Financial Services, Inc.

First Business Financial Services (NASDAQ: FBIZ) is a $1.2 billion Wisconsin-based bank holding company that specializes in focused financial solutions for businesses, key executives, and high net worth individuals through its operating companies. It is the second largest Wisconsin-based commercial bank holding company listed on NASDAQ or New York Stock Exchange. Its companies include: First Business Bank—Madison; First Business Bank—Milwaukee; First Business Bank—Northeast; First Business Trust & Investments; First Business Equipment Finance, LLC; and First Business Capital Corp. For additional information, visit www.firstbusiness.com or call (608) 238-8008.

This press release includes “forward-looking” statements related to First Business Financial Services, Inc. (the “Company”) that can generally be identified as describing the Company’s future plans, objectives or goals. Such forward-looking statements are subject to risks and uncertainties that could cause actual results or outcomes to differ materially from those currently anticipated. These forward-looking statements are made pursuant to the safe harbor provisions of the Private Securities Litigation Reform Act of 1995. For further information about the factors that could affect the Company’s future results, please see the Company’s annual report on Form 10-K and other filings with the Securities and Exchange Commission.

SELECTED FINANCIAL CONDITION DATA

STATEMENTS OF INCOME

SELECTED FINANCIAL RATIOS

ASSET QUALITY RATIOS

NON-GAAP RECONCILIATIONS

CORE EARNINGS