Al Gharafa Sports Club was established on 6 June 1979. They were originally known as Al-Ittihad. The club was officially renamed to its current form in 2004 to better represent the district of Al Gharafa.
As Al Gharafa was established relatively late compared to other Qatari clubs at the time, it was sent to the Qatari 2nd Division. It dominated the league and won the title on its debut in the 1979/80 season, The club ended up winning its first Stars League title in 1991–92, breaking the 16-year deadlock between the three dominant Qatari clubs Al-Arabi, Al Sadd, and Al-Rayyan. Al-Gharafa’s youth team also won the league a year later in the 1992/93 season. The club were runners-up in 1994 to Al-Arabi, before winning the Emir of Qatar Cup 4 times in a row, from 1995 til 1998.
Al Gharafa has one of the largest youth development programmes in the country. It recruits youth players for every age bracket available, Currently there are approximately 350 players in the clubs youth ranks, with 100 players in its academy, and 240 players in grass roots.