Dirleton is notable for Dirleton Castle, a well-preserved medieval fortress, which today belongs to Historic Scotland. It is the caput of the feudal barony of Dirleton, said to be one of the oldest in Scotland (This barony did not, however, cover the entire parish). It was built in the middle of the twelfth century by a branch of the Anglo-Norman family of De Vaux, a family with its origins in Rouen, Normandy, which had settled at Dirleton during the reign of King Malcolm IV (1153–1165). They also held the manor of Golyn (Gullane) and parts of the lands of Fenton. In 1225 it is described as a “castellum”. In 1298 when King Edward I of England invaded Scotland, no place was able to resist him except Dirleton castle. After a resolute resistance it surrendered to forces under Antony Bek, Bishop of Durham. It was still in English possession in 1306. When Cromwell invaded Scotland in 1650 the castle was, after a gallant defence, taken by Lambert and by him partially dismantled and reduced to its present ruinous state.