Williams was born at Deerfield, Massachusetts, the son of Ephraim Williams and Emily (Trowbridge) Williams. He was educated at Harvard and at Trinity College, Hartford, where he graduated in 1835. Although his parents were Unitarian, Williams's time at Harvard convinced him to join the Episcopal faith. He was ordained deacon in 1838 and ordained priest in 1841. Williams held the rectorship of St. George's Church, Schenectady, New York, from 1842 to 1848, after which he became president of Trinity College, and at the same time professor of history and literature.
Bishop of Connecticut
In 1851 he was elected Assistant Bishop of Connecticut. He was the 53rd bishop in the ECUSA, and was consecrated by Bishops Thomas Church Brownell, John Henry Hopkins, and William Heathcote DeLancey. On the death of Bishop Brownell, in 1865, Williams succeeded him in the sole charge of the diocese. At the same time, from 1854 on, he held the office of dean of the Berkeley Divinity School at Middletown, and was its principal instructor in Church history and theology.
He succeeded Alfred Lee, of Delaware, in 1887, as presiding Bishop, and earned the reputation of a wise conservative leader in ecclesiastical affairs. In 1896, he was acknowledged as the senior bishop in the Anglican communion.
Among his published works are:
- Thoughts on the Gospel Miracles (1848)
- The English Reformation (Paddock Lectures, 1881)
- The World's Witness to Jesus Christ (Bedell Lectures, 1882)
- Studies in the Book of the Acts (1888)
- Batterson, Hermon Griswold (1884). A Sketch-book of the American Episcopate during one hundred years, 1783-1883. Philadelphia: J. B. Lippencott & Co.. http://books.google.com/books?id=TWY3AAAAMAAJ. Retrieved 2009-07-31.
- Johnson, Rossiter (1904). The Twentieth Century Biographical Dictionary of Notable Americans. X. Boston: The Biographical Society. http://books.google.com/books?id=Ou4UAAAAYAAJ. Retrieved 2009-07-31.
- Shepard, James (1907). History of Saint Mark's Church, New Britain, Conn.. New Britain, Conn.. http://books.google.com/books?id=RFAsAAAAYAAJ. Retrieved 2009-07-31.
- Documents by Williams from Project Canterbury