When
When – English Grammar Today – a reference to written and spoken English grammar and usage – Cambridge Dictionary
dictionary.cambridge.org
That 和 When 都是可以的。其中 When 可能比較有爭議，可以參考字典中 When 當成 pronoun 的例句：That was the week when we booked our holiday.
然而，when 在 corpus 的用法僅 that 用法的 1/10：https://netspeak.org/#q=remember+the+last+time+%5Bwhen+that%5D
結論：正常會用 that，使用 when 會使文章具有獨特風味，口說則是乾脆不用：https://youglish.com/pronounce/remember%20the%20last%20time%20I/english