By heraldic tradition the arms of the bishop of a diocese, called the “Ordinary,” are joined, impaled, with the arms of his jurisdiction. In this case, these are the arms of the Diocese of Alexandria.

These arms are composed of a red field on which are placed a silver (white) cross between four silver (white) bells. The red background is used to signify the Red River that runs through the See City. The cross, of The Faith, divides the four bells that are taken from the arms of the Patriarchal See of Alexandria, in Egypt, for which the See City is named. Overall, at the center of the design is a black and gold (yellow) checky crescent, which is taken from the arms of the Spanish family “Xavier,” and this symbol is used throughout ecclesiastical heraldry as the classic charge for Saint Francis Xavier, titular of the cathedral-church in Alexandria.

For his personal arms, His Excellency, Bishop Herzog, has selected a design that reflects his life and his heritage.

The design is composed of a silver (white) field on which are displayed symbols of three significant attributes of the bishop's life. In base are blue wavy bars to represent the waters of the Gulf of Mexico and the Mississippi River. Having served as a priest in the Diocese of Natchez-Jackson and the Diocese of Biloxi the mentioned bodies of water have been integral features in all that the Bishop has done in God's Holy Service. In the upper portion of the design are a quill, Proper (i.e.: as it appears in nature), with a brown sword placed in the form of an “X” (“per saltair”). These are two devices that are strongly associated with the life of Saint Paul, the Bishop's baptismal patron, who was both writer and put to death by the sword.

Across the center for Bishop Herzog's design is a bar, called a “fess,” that is made up of many small crosses that are of the many heraldic colors . . . “de colores . . . 'of the colors.'” This representation is employed to express that as a child of a marriage of mixed faiths (Lutheran and Catholic), ecumenism has been an important ministry for Bishop Herzog. The Church is made up of many denominations, races, colors and cultures and therefore we are “ONE IN THE LORD,” as expressed in the Bishop’s motto.

The achievement is completed with the external ornaments which are a gold processional cross, that is placed in back of and which extends above and below the shield, and the pontifical hat, called a “gallero,” with its six tassels, in three rows, on either side of the shield, all in green. These are the heraldic insignia of a prelate of the rank of bishop by instruction of The Holy See of March 31, 1969.