Castle Hill is a 13th century earthwork enclosure fortress, founded by the Macduffs, Thanes of Fife. Standing on a low promontory overlooking the sea shore, it was thought to be the original castle of the de Vaux family. During the Wars of Independence with England, the castle was occupied from 1298, by the army of King Edward I under Bishop Beck. Abandoned by the English garrison in 1314, in the late 14th century the Lauder family possibly constructed a tower with a barmkin on the site. In the early 15th century the castle was abandoned for the secure stronghold of Bass Rock Castle and its lands given to North Berwick Priory. The oval summit of the mound is encased by a low bank of earth and stone, which is probably the remains of a turf-covered wall. Attached to the mainland by a narrow neck of land, a wide ditch which spreads down both sides cut off the castle. 3 miles east is Tantallon Castle and 3 miles west is Dirleton Castle.