The death of Prince Philip at the age of 99 has the world reflecting on his remarkable life, and that includes looking back at his up-and-down relationship with his former daughter-in-law, Princess Diana. Prince Philip and Princess Diana once shared the special bond of knowing what it was like to marry the heir to Britain’s throne. That commonality wasn’t, ultimately, enough to keep them close. Anyone who watched The Crown‘s fourth season will recall the scenes of Philip and Diana happily stalking through the Scottish highlands together near Balmoral Castle, and those moments accurately reflect Philip’s feelings at the time. He thought Diana was everything Charles needed in a wife: young, English, fun, and interested in the outdoors. It’s possible Philip saw something of himself in Diana, too; they both were, while well-connected socially, outsiders to the unique world of the Windsors.