Towry was a very able administrator and became the Board's 'chief trouble-shooter... whenever there were difficulties to be resolved on the spot'. In May 1801 he and others were captured by a French privateer while on their way to Lisbon on such a mission, though soon released, and he remained in harness as a 'tough old bureaucrat' to his death. He appears to have had two sons by his first wife: one, George Henry Towry, became a promising captain in the Navy but died young. Charles George was a naval lieutenant of 1790. A daughter, Anne, was a notable beauty and in 1789, after three times refusing, married the less than handsome but brilliant barrister Edward Law, who in the early 1790s made his name successfully defending the impeached Warren Hastings. In 1802 Law became Lord Chief Justice of England and was ennobled as 1st Baron Ellenborough. Towry's numerous Ellenborough grandchildren included the eldest son, also Edward – a politician, Governor-General of India and 1st Earl of Ellenborough – and the Revd William Towry Law, later notable as a Catholic convert.