Gretchaninov started his musical studies rather late because his father, a businessman, had expected the boy to take over the family firm. Gretchaninov himself related that he did not see a piano until he was 14 and began his studies at the Moscow Conservatory in 1881 against his father's wishes and without his knowledge. His main teachers there were Sergei Taneyev and Anton Arensky. In the late 1880s, after a quarrel with Arensky, he moved to St. Petersburg where he studied composition and orchestration with Nikolai Rimsky-Korsakov until 1893. Rimsky-Korsakov immediately recognized Gretchaninov's extraordinary musical imagination and talent and gave him much extra time as well as considerable financial help. This allowed the young man, whose parents were not supporting him, to survive. Out of this came an important friendship, which only ended in 1908 with Rimsky's death. As such, it is not surprising that Rimsky's influence can be heard in Gretchaninov's early works, such as his String Quartet No.1, a prize-winning composition.

Around 1896, Gretchaninov returned to Moscow and was involved with writing for the theater, the opera, and the Russian Orthodox Church. His works, especially those for voice, achieved considerable success within Russia, while his instrumental works enjoyed even wider acclaim. By 1910, he was considered a composer of such distinction that the Tsar awarded him an annual pension.

Though Gretchaninov remained in Russia for several years after the Revolution, he ultimately chose to emigrate, first to France in 1925, and then to the United States in 1939. He remained in the U.S. the rest of his life and eventually became an American citizen. He died in New York at the age of 91 and is buried outside the church at Rova Farms, a Russian enclave in Jackson Township, Ocean County, New Jersey.

Gretchaninov wrote five symphonies, the first premiered by Rimsky-Korsakov; four string quartets, the first two of which won important prizes, two piano trios, sonatas for violin, cello, clarinet, piano and balalaika, several operas, song cycle Les Fleurs du Mal, op. 48 (setting lyrics by Baudelaire) and much other music.

Like Vladimir Rebikov, his position in the history of Russian music was mainly transitional, his earlier music belonging firmly in that earlier Romantic tradition while his later work is influenced by some of the streams that also affected Igor Stravinsky and Sergei Prokofiev.

Sketches for an unfinished sixth symphony from the 1940s are preserved at the New York Public Library at Lincoln Center.