Exhibit 5.1

[Letterhead of Sullivan & Cromwell LLP]

March 9, 2011

Sterling Bancorp,

Ladies and Gentlemen:

In connection with the registration under the Securities Act of 1933 (the “Act”) of 4,025,000 shares (the “Securities”) of Common Shares, par value $1.00 per share, of Sterling Bancorp, a New York corporation (the “Company”), we, as your special counsel, have examined such corporate records, certificates and other documents, and such questions of law, as we have considered necessary or appropriate for the purposes of this opinion. Upon the basis of such examination, we advise you that, in our opinion, the Securities are validly issued, fully paid and nonassessable.

The foregoing opinion is limited to the Federal laws of the United States and the laws of the State of New York, and we are expressing no opinion as to the effect of the laws of any other jurisdiction.

We have relied as to certain factual matters on information obtained from public officials, officers of the Company and other sources believed by us to be responsible.

We hereby consent to the filing of this opinion as an exhibit to the Registration Statement and to the reference to us under the heading “Validity of the Common Shares” in the Prospectus Supplement. In giving such consent, we do not thereby admit that we are in the category of persons whose consent is required under Section 7 of the Act.