Smith was born into a family of masons in Dalkeith, Scotland, and emigrated to Philadelphia in 1748 already trained as a builder rather than a gentleman-amateur. He rose fast: within a few years he was designing the city's most prominent commissions, among them the steeple of Christ Church, St Peter's Church, Carpenters' Hall and the building he raised for the College of New Jersey at Princeton. He worked exclusively in the Georgian idiom of his English pattern books and was a founding member of the Carpenters' Company of Philadelphia, the guild that trained the city's builders and set its construction standards. A member of the American Philosophical Society, he turned his skill to defence when war came, designing the chevaux-de-frise sunk in the Delaware River to block British ships. He died in 1777 at Fort Billingsport, New Jersey, while building barracks for the Continental Army.