The ancient family of Ramsay is of Anglo-Norman origin. Simundus de Ramesia was the first of the name to appear on record in Scotland. Originally from Huntingdonshire in England, he received lands in Lothian from David I and in 1140 witnessed a charter to the monks at Holyrood. He was founder of the main line and the first Ramsay to acquire land at Dawolsey.

There are several Ramsays on record who witnessed Lothian charters in the 13th century. William de Ramsay was witness to the Coldinghame Charter in 1198 and to another in 1236. His cousin, sir Nessus de Ramsay, affixed his seal to a charter by Alexander II in 1217. Nicholas de Ramsay was witness to a charter between 1250 and 1270. William's son, also William de Ramsay, was a member of the Council of Magnates of the Realm in 1255 and in 1260. He too witnessed a charter. His son, Sir William Ramsay de Dalwolsey, signed the Ragman Roll in 1296, and joined the forces of Sir Robert Bruce at the battle of Bannockburn in 1314. He signed the Declaration of Arbroath in 1320 asserting the independence of Scotland to the Pope.

Sir William's two sons include Sir Alexander Ramsay of Dalhousie and Sir William Ramsay of Inverieth. Sir Alexander, the elder son, was Warden of the Middle Marshes, in command of men of Lothian, and one of the Regent's chief commanders at Borough Muir, where England's ally, the Flemish Army, was defeated. He was present at the capture of Leuchars Castle, at St Andrews in 1335, and in June of 1338 he relieved Dunbar Castle and assisted the Countess of Dunbar in her struggle to maintain the stronghold against the English. His tragic story begins when he and his party of men recaptured Roxburgh Castle from the English in 1342. The titular constable of the Castle, Sir William Douglas, had several times tried unsuccessfully to retake it. For his brave and heroic feat, Sir Alexander was appointed constable of Roxburgh and Sheriff of Teviotdale. Sir William Douglas was so outraged by the appointments, that he sought revenge by capturing Sir Alexander and imprisoning him in the dungeon of Hermitage Castle where he was left to starve. Legend has it that he survived for seventeen days by eating small quantities of grain that fell through the cracks in the floor of the castle granary above the dungeon. His brother, Sir William Ramsay of Inverleith succeeded Sir Alexander at Dalhousie in 1342 and was famous for his raid around Norham Castle at the battle of Nesbitt Moor in 1355.

The family was then passed on through Sir Patrick Ramsay, Alexander Ramsay of Carnock, and his son Sir Alexander of Dalhousie who in 1400 successfully withheld a six month siege by English forces under Henry IV at Dalhousie Castle. Sir Alexander was killed two years later at Hamildon Hill and his great great grandson, Alexander was killed at Floddenin 1513. In August 1618, the family received Royal recognition when Sir George Ramsay was created a Lord of Parliament by the title of Lord Ramsay of Melrose, which he later had changed to Lord Ramsay of Dalhousie. Sir George's son, William, was created Earl of Dalhousie and Lord Ramsay of Keringtoun in June 1633. The earldom of Dalhousie was passed on to another George, 2nd Earl, then William, 3rd Earl, and to George, 4th Earl.