Irvine was born in Lerwick in Shetland in June 1805. He lost his father aged only ten, and realised that survival in the world would have to come from his own hard work. He had left Shetland by 1826, arriving at London via Edinburgh. He studied at the Royal Academy and was awarded a medal in 1828. Fellow students included William Etty and Daniel Maclise, who he was particularly friendly with. In 1834 he was elected an associate of the Royal Scottish Academy, and also exhibited in London in 1838, specialising in portraits. He emigrated to Australia to join his son, and in 1863 he moved to Dunedin in New Zealand. He remained there until his death, aged eighty-three.