The song is performed by Astaire on the foggy deck of the ferry from New Jersey to Manhattan. It is sung to Ginger Rogers, who remains silent listening throughout. No dance sequence follows, which was unusual for the Astaire-Rogers numbers. Astaire and Rogers did dance to it later in their last movie The Barkleys of Broadway (1949) in which they played a married couple with marital issues. The song, in the context of Shall We Dance, notes some of the things that Peter (Astaire) will miss about Linda (Rogers). The lyrics include "the way you wear your hat, the way you sip your tea", and "the way you hold your knife, the way we danced till three." Each verse is followed by the line "no, no, they can't take that away from me." The basic meaning of the song is that even if the lovers part, though physically separated the memories cannot be forced from them. Thus it is a song of mixed joy and sadness.