On June 14, 1904, a petition signed by more than 100 voters was presented to the Albany City Council to call a special election to vote on an annual tax of one and one-fourth mills on $100 valuation to establish and maintain a public library. The necessary procedure was carried out and the election was held on July 5, 1904, The result was 235 for and 45 against the tax.

The mayor and board of aldermen of the City of Albany at their meeting on July 6, 1904 appointed a Board of Directors known as the Library Board. The following nine men were appointed: Joshua B. Thomas, George C. Holden, Samuel W. Clark, Dale S. Flowers, Malcolm M. Campbell, J. W. Peery, Richard L. Whaley, George W. Doyle, and Charles H. S. Goodman.

The Library Board organized July 8, 1904. The president appointed a committee on by-laws. A motion was made and carried that sealed proposals be received by the secretary for a site for a library building, the site to be not less than 66 feet by 93 feet and not more than two blocks from the courthouse square. Lot 6, Block 1, Albany, Gentry County, Missouri was bought in July 1904 from Mr. James W. Ganor and wife for the sum of $1,325.00.

A telegram from Andrew Carnegie dated June 29, 1904 was received in response to communications from C. E. Ernst, city clerk of the City of Albany. The telegram stated that Mr. Carnegie would donate $10,000 to erect a library building if the City of Albany met certain conditions. On July 30, 1904 the necessary papers showing that the City of Albany had met the requirements were sent to Mr. Carnegie. Mr. Samuel W. Clark, Secretary of the Library Board, received a letter from Mr. Carnegie's secretary dated October 25, 1904 stating, "Mr. Carnegie will now authorize his cashier, Mr. R. A. Francis, Home Trust Company, Hoboken, N.J. to honor your calls for money to pay for the library building to the extent of ten thousand dollars."

On August 16, 1904 Mr. Eckel of the firm of Eckel and Mann Architects of St. Joseph, Mo., upon invitation, met with the Board of Directors in consultation in regard to plans for a library building. He made an offer to draw plans for a basement and one-story library building for $200. His offer was accepted. November 4, 1904 a motion was made to advertise for bids to build the Albany library building according to plans and specifications of Eckel and Mann Architects and to be completed on or before July 25, 1905. On November 25, 1905 a motion was made and carried to accept the bid of Louis Valin of Stanberry, Mo. to build the library according to plans for the sum of $8,976. Before the meeting was over the Library Board wanted changes which cost $95 more making a total of $9,071.

On July 10, 1905, on motion, it was ordered that the Library Board ask Mr. Carnegie for $2,000 additional to complete said library building, said amount being necessary on account of unforeseen foundation requirements. December 13, 1905 at the assurance of certain conditions by the Library Board, the city council passed a resolution required by Mr. Carnegie in order that his donation be increased in the sum of twenty-five hundred dollars.

On January 6, 1906 a motion was made and carried to ask for bids to plaster and partition off the basement into two rooms. This was done at the cost of $127 for the labor. The building and grounds cost $15,000. Also at this meeting, a committee was appointed to procure catalogues of books, magazines, and furniture.

From this time on, different committees worked to get the library ready for operation. February 3, 1906 Miss Lola Neil was elected librarian and janitor out of 14 applications. Her salary was to be $25 per month. A janitor was hired in November 1907. On March 1, 1906, the Carnegie Public Library was opened to the public.

In 1909 Miss Neil resigned as librarian and Miss May Erskine was elected librarian. She served one year then Miss Ophelia C. Hutchison was elected librarian for the year 1910. In 1911 Miss May Erskine was again elected librarian and then served until June 25, 1926. Mrs. Bea McFall was elected librarian and served until March 1, 1930. At that time the Library Board required an examination be given to the applicants for librarian. Mrs. Anita Atherton was elected librarian as a result of the first examination. She was librarian until she resigned and worked for her last day June 30, 1941. Another examination was given and Mrs. Mildred Osborne was elected librarian and served until August 10, 1942 when she resigned. Again, there was an examination for a librarian. Mrs. Linna E. Yeater was elected librarian and started service August 24, 1942. She was followed by Mrs. Helen Henton who served from November 1, 1964 until she retired on November 30, 1997. Mrs. Cheryl A. (Noble) Lang, who was promoted from assistant librarian on December 1, 1997, is the current librarian.

On March 1, 1930 the Dewey Decimal Classification System of books was started and completed in six months. The bookcases and library of Rev. J. Weston Mays was added to the Carnegie Public Library on September 1, 1930.

July 1, 1938 the Albany Public School made a proposition to the Carnegie Public Library for the library to shelve and care for the high school books; in return the public was to have access to the school books. The library accepted the proposition and the high school books were cared for by the Carnegie Public Library until June 1, 1953 when the agreement was ended.

Through the years, many additions and renovations were made to the interior of the library. A hardwood floor was laid, fluorescent lights installed, and new shelving installed. The original tables were refinished and new chairs purchased. Currently in the foyer is a display case that was donated to the library after Hardin's Drug Store closed.

Application was made by the library board to have the building listed on the National Register of Historic Places. Certification was received on March 1, 1990 and a commemorative bronze plaque was placed outside near the front door.