Hay Castle is an 11th century earth and timber ringwork fortress and was probably founded by William fitz Osbern. The small knoll, crowned by an oval rampart, was scarped to create steeper sides and the castle may be the successor of or occupied simultaneously with Hay Tump. Bernard de Neufmarché may have founded the stone castle in the early 12th century, when adding the free-standing four storey residential keep, built to protect the northern entrance of the timber palisade. Replaced by a powerful unflanked polygonal curtain wall, with a finely built gateway, in the 13th century the Braose lords encased the gateway with a gatehouse and added a portcullis to defend the Norman gate. The walls of the keep have been extensively rebuilt, probably due to insecure foundations and the castle was damaged during several rebellions in the 13th century, with further damaged by Owain Glyndwr in 1401. In the 17th century, Howell Gwynne added the two storey Jacobean manor house to the north-west side of the castle and annexed the upper floors of the keep. Continuously occupied for the last 800 years, the house underwent restoration in 1910 but two severe fires in the 20th century have obliged further work. 3 miles north is Clifford Castle.