The Peabody Memorial Library is the first separate college library building in Texas. Located east of Austin Hall and the Old Main Memorial in central campus, it is the university's oldest building constructed after its founding in 1879.

The first library on the Sam Houston Normal campus was restricted to one room in the Main Building. Dubbed the Peabody Memorial Library, the growing student body soon found the twenty-year-old solitary room overcrowded and funds were raised for a new, separate structure. Designed by J. L. O'Connor and built in less than year, the tiny building continued to carry the name honoring Peabody and his Southern Education Fund.

Built at a cost of just over $9000, the building was constructed in the shape of a cross (65 x 85 ft.). Inside was the reading room, with decorated arched ceiling and fireplace; a cloakroom; and office space. The exterior was dotted with stained glass windows, similar to those found in the nearby Main Building. So handsome and pristine was the structure that soon President Henry Pritchett moved his personal office to the library; his successor, Harry Estill, chose against this, allowing Henry's brother, Joseph, the Dean of Men, to use the office instead.

It remained in use as a library for over twenty-five years until it too became congested. As before, funds were appropriated and, in 1929, the new Estill Library opened at the southern edge of the quadrangle. Unfortunately, the new library set off a series of uncertain years for the Peabody. Classroom and office space was always in need and the former library went through a multitude of uses: administrative offices, band rehearsal area, general storage, and the first studio space for KSHU-FM, which used the library as home until the nearby Communication Building was completed.

Leading up to the 1982 fire that destroyed the Main Building and severely damaged Austin Hall, there was talk of razing the Peabody Building. A sense of historic preservation prevailed after the loss of the Main Building, and the Peabody went through a series of renovations that cost over a half million dollars. After its rededication in 1991, the library not only regained its former splendor but also saw a return to its original use as a library - now for the university archives.

In October 1998, the Peabody was closed because an outbreak of mold was found in the collections. Brought upon by humidity and leaks in the roof discovered after recent rainy weather, university archives were temporarily relocated to the Gresham Library. Archives permanently moved to the Gresham Library in 2004.

In subsequent years the Peabody has become a social center similar to Austin Hall, available for receptions, meetings, and special events.

Incidentally, two items that were once located in the Memorial Hall of Old Main survived the fire and have since been on display within the Peabody. First, the statue of the goddess Athena spent most of the late 1990s in the Peabody, until it returned to the art department in 2004. In 2007, the 113-year-old refurbished Old Main organ was moved into the Peabody.

The marker on the building reads: "The first campus structure to be used exclusively for library purposes, this building was erected in 1902. Built with assistance from the Peabody Education Fund (a philanthropic program created by northern banker George Peabody soon after the Civil War), the library exhibits classical revival and Romanesque revival style influences. Designed by J. L. O'Connor, it served as a library until 1929 and has had other academic uses since that time."