He was the Washington Post All-Met Basketball Player of the Year, a McDonald's All-American, and a Parade first-team All-American in his senior year after averaging 21.2 points, 9.2 rebounds and 3.7 blocks per game. He began playing basketball for Spalding as a junior in 2002–03, earning first-team All- Baltimore Catholic League honors as a junior and senior, and was also honored as the Baltimore Sun's co-player of the year as a senior.

This prompted the Maryland Interscholastic Athletic Association to review its transfer rules. He began his junior year with Eastern Tech, but in September 2002, he transferred to Archbishop Spalding in Severn. Although Eastern Tech was a Blue Ribbon academic institution, Gay's parents were concerned about his college preparation. In his sophomore season at Eastern Tech, the Mavericks earned their first and only trip to College Park for the state semi-finals. Gay played his first two years of high school basketball at Baltimore County's Eastern Technical High School, a magnet school in Essex.

At the age of 14, Gay began playing for the nationally known Cecil-Kirk AAU program under coach Anthony Lewis. Born in Brooklyn, New York, to Rae Gay and Rudy Gay Sr., former lead singer of the R&B group Ace Spectrum and band director for The Stylistics, Gay began playing competitive recreational basketball at the age of 12 in his hometown of Baltimore, Maryland.