For his personal arms, His Excellency, Bishop Herzog, has selected a design that reflects his life and his heritage.

The design is composed of a silver (white) field on which are displayed symbols of three significant attributes of the bishop's life. In base are blue wavy bars to represent the waters of the Gulf of Mexico and the Mississippi River. Having served as a priest in the Diocese of Natchez-Jackson and the Diocese of Biloxi the mentioned bodies of water have been integral features in all that the Bishop has done in God's Holy Service. In the upper portion of the design are a quill, Proper (i.e.: as it appears in nature), with a brown sword placed in the form of an “X” (“per saltair”). These are two devices that are strongly associated with the life of Saint Paul, the Bishop's baptismal patron, who was both writer and put to death by the sword.

Across the center for Bishop Herzog's design is a bar, called a “fess,” that is made up of many small crosses that are of the many heraldic colors . . . “de colores . . . 'of the colors.'” This representation is employed to express that as a child of a marriage of mixed faiths (Lutheran and Catholic), ecumenism has been an important ministry for Bishop Herzog. The Church is made up of many denominations, races, colors and cultures and therefore we are “ONE IN THE LORD,” as expressed in the Bishop’s motto.

The achievement is completed with the external ornaments which are a gold processional cross, that is placed in back of and which extends above and below the shield, and the pontifical hat, called a “gallero,” with its six tassels, in three rows, on either side of the shield, all in green. These are the heraldic insignia of a prelate of the rank of bishop by instruction of The Holy See of March 31, 1969.