Grace grew up in Darien, Connecticut, where actress Kate Bosworth was a middle school friend and actress Chloë Sevigny who later appeared with him in high school stage plays was sometimes his babysitter.