— Philip Stanhope, 4th Earl of Chesterfield British statesman and man of letters 1694 - 1773
Attributed to Chesterfield in his 1833 edition of Horace Walpole's letters to Sir Horace Mann by George Agar-Ellis, 1st Baron Dover. The quotation has been attributed to many others, such as Lord Chief Justice Campbell, William Henry Maule (in the form "You silly old fool, you don't even know the alphabet of your own silly old business"), Sir William Harcourt, Lord Pembroke, Lord Westbury, and to an anonymous judge, and said to have been spoken in court to Garter King at Arms, Rouge Dragon Pursuivant, or some other high-ranking herald, who had confused a "bend" with a "bar" or had demanded fees to which he was not entitled. George Bernard Shaw quotes it in Pygmalion (1912) in the form, "The silly people dont [sic] know their own silly business."
The quotation seems to appear first in Charles Jenner's The Placid Man: Or, The Memoirs of Sir Charles Beville (1770): "Sir Harry Clayton ... was perhaps far better qualified to have written a Peerage of England than Garter King at Arms, or Rouge Dragon, or any of those parti-coloured officers of the court of honor, who, as a great man complained on a late solemnity, are but too often so silly as not to know their own silly business." "Old Lord Pembroke" (Henry Herbert, 9th Earl of Pembroke) is said by Horace Walpole (in a letter of May 28, 1774 to the Rev. William Cole) to have directed the quip, "Thou silly fellow! Thou dost not know thy own silly business," at John Anstis, Garter King at Arms. Edmund Burke also quotes it ("'Silly man, that dost not know thy own silly trade!' was once well said: but the trade here is not silly.") in a "Speech in the Impeachment of Warren Hastings, Esq." on May 7, 1789. Chesterfield or Pembroke fit best in point of time.