Castle Hill is a late 11th century earth and timber fortress, founded by Hugh de Grantmesnil. Built on a knoll, the oval flat-topped motte stood in a large elongated oval bailey, which was encased by a deep, wide rock-cut ditch, with a counterscarp bank. In 1176, King Henry II ordered the castle to be dismantled but buried in the centre of the motte, is the base of a 12th century high stone tower. Built directly on the bedrock and enclosed within a motte, the north wall of the tower is missing but there are still the remains of a stair and a doorway. In 1343 when a stone manor house and a chapel of the Ferrers family were mentioned, the castle had become a manorial complex. There are foundations and fragments of high walls in the southern bailey and more stonework may have been absorbed in the construction, of the late 15th century Old Hall. The eastern bailey ditch was backfilled, when a 16th century pleasure garden was built and 18th century images still show a building on the motte. 3 miles south is Kirby Muxloe Castle and 10 miles west is Shackerstone Castle.