Their ancestor was a Pramukh (pronounced in Arabic as Barmak), a title borne by the high priest in the Buddhist temple of Nawbahār. He played an important role in the formation of the Abbasid dynasty. He is the first Barmakid about whom much is known. Khalid supported the revolt of As-Saffah, who was to become the first Abbassid caliph. In 747, he was put in charge of the distributions of war spoils from the revolt, then of the district of Dayr Qunna, and then in 749, was minister along with Abu Al-Jahm, and was chief of the land Tax bureau. He was also entrusted with the upbringing of the caliph's daughter.
Under the Caliphate of al-Mansur, he was appointed governor of Fars and, after helping obtain Prince Isa ibn Musa's renunciation of succession to the caliphate in 765, was governor of Tabaristan. He built the town of Mansura; and was later involved in the founding of Baghdad, and protested the use of material from Ctesiphon to build the new city
In 765 (775?), he lost favour with the Caliph due to political intrigues, and was given a heavy fine, for which Al-Khayzuran, the wife of prince Al-Mahdi helped him raise money. But following Kurdish uprisings in Mosul, Khalid was appointed governor of the city, which he was until Al-Mansur's death. Around the same time, his son Yahya ibn Khalid was appointed governor of Azerbaijan. Afterwards, his family gained more privilege under the caliphate of al-Mahdi, with Yahya's help. In 782, he was sent as guardian on an expedition led by Harun al-Rashid (then 20) against the Byzantine Empire, that went as far as the Bosphorus.