The antiquarian John Sime once referred to Kemp as an 'architectural artist', a reflection of his ambiguous professional status. Kemp originally trained and worked as a carpenter, but he had always been a keen draughtsman and developed an interest in architecture. Kemp particularly admired Gothic buildings and made studies of the medieval abbeys at Melrose, Dryburgh and Jedburgh in the Scottish borders. As an architect, he was largely self-taught. In 1836, he entered and won the competition to design a memorial in Edinburgh to honour the recently deceased writer Sir Walter Scott. This spectacular Neo-Gothic monument was to be Kemp's only architectural legacy to the city: he died in an accident in 1844 and never lived to see it completed.