UNITED STATES
SECURITIES AND EXCHANGE COMMISSION

Form 10-K

ANNUAL REPORT PURSUANT TO SECTION 13 OR 15(d)
OF THE SECURITIES EXCHANGE ACT OF 1934

For the fiscal year ended                     December 31, 2004          

Commission file number                     0-15956          

Bank of Granite Corporation

Securities registered pursuant to Section 12(b) of the Act: None

Securities registered pursuant to Section 12(g) of the Act:

Common Stock, $1.00 par value

     Indicate by check mark if disclosure of delinquent filers pursuant to Item 405 of Regulation S-K is not contained herein, and will not be contained, to the best of registrant’s knowledge, in the definitive proxy or information statements incorporated by reference in Part III of this Form 10-K or any amendment to this Form 10-K. þ

     Indicate by check mark whether the registrant (1) has filed all reports required to be filed by Section 13 or 15(d) of the Securities Exchange Act of 1934 during the preceding 12 months (or for such shorter period that the registrant was required to file such reports), and (2) has been subject to such filing requirements for the past 90 days. Yes þ No o

     Indicate by check mark whether the registrant is an accelerated filer (as defined in Exchange Act Rule 12-b-2). Yes þ No o

     As of March 4, 2005, 13,281,106 shares of common stock, $1 par value, were outstanding. As of June 30, 2004, the aggregate market value of common stock held by non-affiliates was $258,532,008.

Documents Incorporated by Reference

     PART III: Definitive Proxy Statement dated March 18, 2005 as filed pursuant to Section 14 of the Securities Exchange Act of 1934 in connection with the 2005 Annual Meeting of Shareholders.

1

FORM 10-K CROSS-REFERENCE INDEX

2

PART I

ITEM 1 — BUSINESS

Bank of Granite Corporation (the “Company”) is a Delaware corporation that was organized June 1, 1987 as a bank holding company. The Company currently engages in no operations other than ownership and operation of Bank of Granite (the “Bank”), a state bank chartered under the laws of North Carolina on August 2, 1906 and Granite Mortgage, Inc. (“Granite Mortgage”), a mortgage bank chartered under the laws of North Carolina on June 24, 1985. Granite Mortgage was acquired by the Company on November 5, 1997. On July 15, 2003, the Company acquired First Commerce Corporation and its wholly owned subsidiary, First Commerce Bank (referred to herein collectively as “First Commerce”), and merged First Commerce Bank into the Bank on July 24, 2003. First Commerce Bank operated three banking offices in the Charlotte metropolitan area. The Company conducts its community banking business operations from 20 full-service offices and one loan-production office located in Caldwell, Catawba, Burke, Watauga, Wilkes, Mecklenburg and Forsyth counties in North Carolina. According to the Federal Deposit Insurance Corporation (the “FDIC”), the Bank ranked 10th in assets and 10th in deposits among North Carolina institutions as of September 30, 2004. The Company conducts its mortgage banking business operations from 13 offices in the Central and Southern Piedmont and Catawba Valley regions of North Carolina and in Hilton Head Island, South Carolina.

The Company conducts its business through three reportable business segments: Community Banking, Mortgage Banking and Other. The Community Banking segment offers a variety of loan and deposit products and other financial services. The Mortgage Banking segment originates, retains and sells mortgage loans. The Other segment includes activities at the holding company level such as corporate and shareholder relations and funding from the issuance of commercial paper and trust preferred securities. For financial information on the Company’s three business segments, see Note 20 “Operating Segments” of the “Notes to Consolidated Financial Statements.”

GENERAL BUSINESS

The Bank is an independent community bank. The Bank’s principal community banking activities include the taking of demand and time deposits and the making of loans, secured and unsecured, to individuals, associations, partnerships and corporations. The majority of its customers are individuals and small businesses. No material part of its business is dependent upon a single customer or a few customers whose loss would have an adverse effect on the business of the Bank. No material portion of the business of the Bank is seasonal.

Granite Mortgage’s principal mortgage banking activities include the origination and underwriting of mortgage loans to individuals. Granite Mortgage also sells mortgage servicing rights and appraisal services. Granite Mortgage specializes in government guaranteed mortgage products. The majority of its customers are individuals. No material part of its business is dependent upon a single customer or a few customers whose loss would have an adverse effect on the business of Granite Mortgage. The mortgage business is sensitive to changes in interest rates in the market. When rates decline, Granite Mortgage experiences an increase in its mortgage business. When rates rise, Granite Mortgage’s business declines. No material portion of the business of Granite Mortgage is seasonal.

3

GENERAL DESCRIPTION OF ECONOMIC AREAS

Prior to 2003, the Company conducted its community banking operations primarily in Caldwell, Catawba and Burke counties of North Carolina. This area was historically known as a center for the manufacture of fiber optic and coaxial cable, furniture, and apparel. When the economy began to weaken in 2001, these counties were significantly impacted with a sudden and sustained rise in their unemployment rates. All of these industries faced massive layoffs of their workforces. In 2003, as opportunities arose to expand and diversify its market areas, the Company did so by entering three new markets where the local economies were more diversified and growing. The Bank opened new banking offices in Watauga County (Boone) and Wilkes County (Wilkesboro) in April and acquired First Commerce Bank and its three banking offices in Mecklenburg County (Charlotte and Cornelius) in July. In 2004, the Bank opened a banking office in Forsyth County (Winston-Salem). In 2005, the Bank plans to convert its loan production office in Mecklenburg County (Matthews) to a full-service banking office. The relative unemployment rates and the population growth in Mecklenburg County and Forsyth County, each as shown in the tables below, formed the primary basis for the Company’s decision to expand into those new markets.

4

TERRITORY SERVED AND COMPETITION

The Bank operates community banking offices in the North Carolina cities of Granite Falls, Lenoir, Hudson, Newton, Morganton, Hickory, Boone, Wilkesboro, Charlotte, Cornelius, Conover and Winston-Salem, with a total of 20 full-service offices, as well as a loan-production office in Matthews. The Bank plans to convert its loan-production office in Matthews to a full-service banking office in 2005.

The Federal Deposit Insurance Corporation (the “FDIC”) collects deposit data from insured depository institutions as of June 30 of each year as presented below (deposit dollars in millions) .

The Bank’s community banking markets are highly competitive. In addition to competing with other large and small banks, which tend to be numerous, especially in the higher growth markets, the Bank also competes for both loan and deposit business with thrifts or savings institutions, credit unions, brokerage and insurance firms and other nonbank businesses, such as manufacturers and retailers.

The mortgage banking business is also highly competitive, with both bank and nonbank mortgage originators competing in the market. Granite Mortgage conducts its mortgage banking business from 13 offices in the North Carolina cities of Winston-Salem, Hickory, High Point, Lenoir, Morganton, Newton, Salisbury, Boone, Charlotte and Fayetteville, and from one office in Hilton Head Island, South Carolina.

The Company’s community banking and mortgage banking operations are both required to compete based on price in order to conduct business in each of the Company’s markets. However, the Company believes that its focus on and commitment to providing superior customer service is what distinguishes it from its competitors.

EMPLOYEES

As of December 31, 2004, the Bank had 253 and Granite Mortgage had 62 full-time equivalent employees. Each of the Bank and Granite Mortgage considers its relationship with its employees to be excellent.

5

SUPERVISION AND REGULATION

The following summaries of statutes and regulations affecting bank holding companies, banks and mortgage banks do not purport to be complete. Such summaries are qualified in their entirety by reference to such statutes and regulations.

The Company is a bank holding company within the meaning of the Bank Holding Company Act of 1956, as amended, and is required to register as such with the Board of Governors of the Federal Reserve System (the “Federal Reserve Board” or “FRB”).

A bank holding company is required to file with the FRB annual reports and other information regarding its business operations and those of its subsidiaries. It is also subject to examination by the Federal Reserve Board and is required to obtain Federal Reserve Board approval prior to acquiring, directly or indirectly, more than 5% of the voting stock of a bank, unless it already owns a majority of the voting stock of the bank. Furthermore, with limited exceptions, a bank holding company must engage only in the business of banking or managing or controlling banks or furnishing services to or performing services for its subsidiary banks. One of the exceptions to this prohibition is the ownership of shares of a company the activities of which the FRB has determined to be so closely related to banking or managing or controlling banks as to be a proper incident thereto.

The FRB has cease-and-desist powers over bank holding companies and non-banking subsidiaries where their action would constitute a serious threat to the safety, soundness or stability of a subsidiary bank.

Although the Company is not presently subject to any regulatory restrictions on dividends, the Company’s ability to pay dividends depends to a large extent on the amount of dividends paid by the Bank and any other subsidiaries. The Bank, as a North Carolina banking corporation, may pay dividends only out of undivided profits as determined pursuant to Section 53-87 of the North Carolina General Statutes. As of December 31, 2004, the Bank had undivided profits of approximately $94.1 million. Additionally, current federal regulations require that the Bank maintain a ratio of total capital to assets, as defined by regulatory authorities, in excess of 6%. As of December 31, 2004, this ratio was 13.20% for the Bank, leaving approximately $71.5 million of the Bank’s undivided profits available for the payment of dividends.

In an effort to achieve a measurement of capital adequacy that is more sensitive to the individual risk profiles of financial institutions, the various financial institution regulators mandate minimum capital regulations and guidelines that categorize various components of capital and types of assets and measure capital adequacy in relation to a particular institution’s relative levels of those capital components and the level of risk associated with various types of assets of that financial institution. The FDIC and the FRB statements of policy on “risk-based capital” require the Company to maintain a level of capital commensurate with the risk profile assigned to its assets in accordance with the policy statements. The capital standards call for minimum total capital of 8% of risk-adjusted assets. At December 31, 2004, the Company’s tier 1 ratio and total capital ratio to risk-adjusted assets were 16.0% and 17.2%, respectively. The Company’s leverage ratio at December 31, 2004 was 13.4%. The Company is in compliance with all regulatory capital requirements.

The Bank is subject to supervision and regulation, of which regular bank examinations are a part, by the FDIC and the North Carolina State Banking Commission (the “Banking Commission”). The Bank is a member of the FDIC, which currently insures the deposits of each member bank to a maximum of $100,000 per depositor. For this protection, each bank pays a semi-annual statutory assessment and is subject to the rules and regulations of the FDIC.

6

Federal banking laws applicable to all depository financial institutions, among other things, (i) afford federal bank regulatory agencies with powers to prevent unsafe and unsound banking practices; (ii) restrict preferential loans by banks to “insiders” of banks; (iii) require banks to keep information on loans to major shareholders and executive officers, and (iv) bar certain director and officer interlocks between financial institutions. The prohibitions against preferential loans and certain director and officer interlocks may inhibit the ability of the Bank and the Company to obtain experienced and capable officers and directors, to replace presently proposed officers and directors, or to add to their number.

The Company is an “affiliate” of the Bank within the meaning of the Federal Reserve Act, which imposes restrictions on loans by the Bank to the Company and on investments by the Bank in the stock or securities of the Company, which serve as security for loans by the Bank to any borrower. The Company is also subject to certain restrictions with respect to engaging in the business of issuing, underwriting and distributing securities.

Shareholders of banks (including bank holding companies which own stock in banks) may be compelled by bank regulatory authorities to invest additional capital in the event their banks experience either significant loan losses or rapid growth of loans or deposits. In addition, the Company may also be required to provide additional capital to any additional banks that it acquires as a condition to obtaining the approvals and consents of regulatory authorities in connection with such acquisitions.

Granite Mortgage, as a mortgage bank, is regulated by the Banking Commission. Because Granite Mortgage is a nonbank subsidiary of a bank holding company, it is also regulated by the FRB. In addition, because Granite Mortgage underwrites mortgages guaranteed by the government, it is subject to other audits and examinations as required by the government agencies or the investors who purchase the mortgages.

The Company cannot predict what other legislation might be enacted or what other regulation might be adopted or, if enacted or adopted, the effect thereof.

EFFECTS OF GOVERNMENTAL MONETARY POLICY AND ECONOMIC CONTROLS

The Company is directly affected by governmental monetary policy and by regulatory measures affecting the banking industry in general. Of primary importance is the FRB, whose actions directly affect the money supply and, in general, affect banks’ lending abilities by increasing or decreasing the cost and availability of bank credit in order to combat recession and curb inflationary pressures in the economy by open market operations in the United States government securities, changes in the discount rate on member bank borrowings, and changes in reserve requirements against bank deposits.

Deregulation of interest rates paid by banks on deposits and the types of deposits that may be offered by banks have eliminated minimum balance requirements and rate ceilings on various types of time deposit accounts. The effect of these specific actions and, in general, the deregulation of deposit interest rates have increased banks’ costs of funds and made them more sensitive to fluctuations in money market rates.

In view of changing conditions in the national economy and money markets, as well as the effect of actions by monetary and fiscal authorities, predictions as to possible future changes in interest rates, deposit levels, loan demand or the business and earnings of the Company are difficult and have very limited reliability.

7

INVESTMENT POLICIES

For a discussion of the Company’s investment policies, see “Investment Securities” of the “Management’s Discussion and Analysis of Financial Condition and Results of Operations” section of this annual report.

LOAN PORTFOLIO

For a discussion of the Company’s loan portfolio, see “Loans” and “Provisions and Allowances for Loan Losses” of the “Management’s Discussion and Analysis of Financial Condition and Results of Operations” section of this annual report.

AVAILABLE INFORMATION

Additional information about the Company and its business is available at the Company’s website, at www.bankofgranite.com. The Company’s filings with the Securities and Exchange Commission, including annual reports on Form 10-K, quarterly reports on Form 10-Q, current reports on Form 8-K, and amendments to those reports filed or furnished pursuant to Section 13(a) or 15(d) of the Exchange Act of 1934, are available, free of charge, on the Company’s website at www.bankofgranite.com under the heading “Investor Relations — SEC Filings.” These reports are available as soon as reasonably practicable after the Company electronically files such material with, or furnishes it to, the Securities and Exchange Commission. Information included on the Company’s website is not incorporated by reference into this annual report.

ITEM 2 — PROPERTIES

The Company indirectly owns and operates real estate through its ownership of the Bank and Granite Mortgage. The Bank occupies headquarters, operations and information technology center offices located in Granite Falls, North Carolina, which are owned. The Bank also owns its banking offices in the North Carolina cities and communities of Conover, Cornelius, Granite Falls, Hickory, Hudson, Lenoir, Morganton and Newton. The Bank leases its banking offices in the North Carolina cities and communities of Boone, Charlotte, Matthews, Wilkesboro and Winston-Salem. Management does not currently anticipate any problems with the renewal of its operating leases. Granite Mortgage occupies headquarters offices located in Winston-Salem, North Carolina, which are leased. Granite Mortgage also occupies mortgage origination offices in the North Carolina cities and communities of Boone, Charlotte, Conover, Fayetteville, Hickory, High Point, Lenoir, Morganton, Salisbury and Winston-Salem, and the South Carolina community of Hilton Head Island, all of which are leased. Management believes that the premises occupied by the Bank and Granite Mortgage are well-located and suitably equipped to serve and support the Company’s community banking and mortgage banking businesses. See also Note 5 “Premises and Equipment” of the “Notes to Consolidated Financial Statements.”

ITEM 3 — LEGAL PROCEEDINGS

There were no significant legal proceedings as of December 31, 2004.

ITEM 4 — SUBMISSION OF MATTERS TO A VOTE OF SECURITY HOLDERS

There were no matters submitted to a vote of shareholders in the fourth quarter of 2004.

8

PART II

The Company’s common stock, $1 par value, trades on The NASDAQ National Market® tier of The NASDAQ Stock Market® under the symbol GRAN. Price and volume information is contained in The Wall Street Journal® and most major daily newspapers in the NASDAQ section under the National Market System listings.

During 2004, the market participants making a market in the Company’s common stock with the highest volumes of Company shares traded were Goldman Sachs & Company, the National Stock Exchange, Knight Equity Markets, LP and Wachovia Capital Markets.

As of December 31, 2004, there were 13,316,102 shares outstanding, owned by approximately 2,700 shareholders of record and an estimated 4,200 holders of shares registered in street name or as beneficial owners. The following table presents the quarterly market sales prices and dividend information for the two years in the period ended December 31, 2004.

Quarterly Common Stock Market Price Ranges and Dividends

The following table sets forth information as of December 31, 2004 regarding shares of the Company’s common stock that may be issued upon exercise of options previously granted and currently outstanding under the Company’s stock option plans, as well as the number of shares available for the grant of options that had not been granted as of that date.

9

The Company purchases shares of its common stock in open-market and occasional privately negotiated transactions pursuant to publicly announced share repurchase programs. Share repurchase transactions for the most recent quarter ended December 31, 2004 are set forth below.

10

ITEM 6 — SELECTED FINANCIAL DATA

The following table presents selected consolidated historical financial data of the Company for the periods indicated. This financial data should be read in conjunction with the Company’s consolidated financial statements and notes thereto.

11

MANAGEMENT’S DISCUSSION AND ANALYSIS OF
FINANCIAL CONDITION AND RESULTS OF OPERATIONS

OVERVIEW

Management’s Discussion and Analysis is provided to assist in understanding and evaluating the Company’s results of operations and financial condition. The following discussion should be read in conjunction with the consolidated financial statements and related notes included elsewhere herein. In 1987, the Company was formed under a plan whereby all previously issued shares of the Bank’s stock were exchanged for shares of the Company’s stock. The Bank then became a wholly owned subsidiary of the Company. In 1997, the Company acquired Granite Mortgage, a mortgage bank, through a merger, which was accounted for as a pooling of interests. In July 2003, the Company acquired First Commerce Corporation and its subsidiary bank through a merger, which was accounted for as a purchase transaction. All information presented is consolidated data unless otherwise specified.

This discussion is intended to provide a general overview of the Company’s performance in 2004 and outlook for 2005. Readers seeking a more in-depth discussion of 2004 are invited to read the more detailed discussions below as well as the consolidated financial statements and related notes included under Item 8 of this annual report.

The Year 2004 in Review

Earnings declined in 2004 primarily due to (1) lower profits from mortgage originations, (2) higher provisions for loan losse