Moat Hill was originally an early 12th century earth and timber ringwork, probably founded by the le Fleming family. Later in the 12th century the ringwork was infilled and converted into a motte, with a crescent shaped bailey to the north. In the late 12th or early 13th century, the motte was heightened and defended with a vertical timber revetment. The castle appears to have been abandoned in the mid 13th century when the le Flemings, possibly moved to the moated site of Aldingham Grange. Standing against the sea cliff, coastal erosion has destroyed part of the mound and its wide ditch. 2 miles north-west is Gleaston Castle and 6 miles south-west is Piel Castle.