Exhibit 23.1

Consent of Independent Registered Public Accounting Firm

We consent to the incorporation by reference in the Registration Statement on Form S-8 (File Nos. 333-48724, 333-05117, 333-05119, 333-74831, 333-40220, 333-44752, 333-88634 and 333-150510) and Form S-3 (File Nos. 333-135355, 333-155808 and 333-156369) of Pacific Capital Bancorp and in the related prospectus of our reports dated March 12, 2010, with respect to the consolidated financial statements of Pacific Capital Bancorp and the effectiveness of internal control over financial reporting of Pacific Capital Bancorp, included in this Annual Report on Form 10-K for the year ended December 31, 2009.

/s/ Ernst & Young LLP

Los Angeles, California

March 12, 2010