Castle Hill is possibly a Norman earth and timber fortress, founded by Eustace fitz John. Standing on a steep natural oval knoll, are the remains of a medieval fortified manor house, with the grassy mounds of masonry foundations visible on the higher ground, to the north and east. Its layout is obscured by the encroachment of houses and the two small brick buildings, built for the Home Guard during World War II, apparently as ammunition stores. 3 north-east is Ayton Castle and 5 miles west is Wilton Hall.