The district is named after Jhall Magsi, which is the name of the headquarters of the Magsi tribe, the major tribe within the district. The Magsi historically are branched off the Lashar tribes. The present Nawab of Magsi tribe is Zulfqar Ali Khan Magsi, ex-chief minister of Balochistan.
The district is divided into two sub-divisions: Gandawah and Jhall Magsi. Both sub-divisions have a separate and different administrative history.
Gandawah is a historical and ancient town in Balochistan. Its oldest name was Kandabil, which was renamed Gangaaba during Arab rule During the colonial period this name was changed to Gandawah. Dr. Akhtar Hameed Khan writes in his book titled "Ancient and famous cities of the Pakistan" that Gandawah was one of the provincial headquarters of the Bhil dynasty of India. Similarly, the encyclopaedia of Islam under alphabet "K", named Kandabil, has put light on this city. The writing is of Nabi Bux Khan Baloch. The script is supported and cited by many ancient historical sources. During the Arab rule, Gandawah was a very important town again. It was the winter headquarters of Khans during Kalat Khant-e-rule. Gandawah stayed the headquarters of Kachhi province and was part of Nayabati Balochistan. During colonial period it became the tehsil headquarters (Niabat). After independence and declaration of the “one unit”, Gandawah was one of the tehsil headquarters of Kalat district and Kalat division. In 1965, when Kachhi was notified as a separate district, Gandawah became part of the new district. Its position was raised to sub-division in 1971. Gandawah became the district headquarters of Jhall Magsi district, when it was notified on February 16, 1992.
Jhall Magsi, the other important town of the district, is also the headquarters of the Magsi tribe. It is purely a Baloch area and was part of the Kalat native states during the colonial period. The Magsi tribe enjoyed an important position during the role of the Khant-e of Kalat. It remained one of the native states with sufficient tribe autonomy during the colonial period. Since the Marshala of 1958 it was declared one sub tehsil of Gandawah. Jhall Magsi was raised to the position of subdivision in 1989.
The district is rich in archaeological sites and historical monuments. It has archaeological sites near Khanpur Bahltoor and Kotra (called Pingar Mari) named after Dalorai Dumb, a former Hindu king. Also important are Dumb Hazoor Bux, Tomb of Moti Ghoram, which is near old Khanpur, Tomb Altaz Khan near Panj Monah, Tomb Mian Sahib and Tomb Bhootani.
Many important saints (Pirs) are buried in the district. Among them are Muhammad Ayub Shah Bukhari (Gandawah), Sain Rakhil Shah and Sain Chizal Shah (Fatahpur) and Pir Chattal Shah near (Kotra). Their followers visit these places frequently. Pir Lakha is a famous place near Jhall Magsi, where bathing cures many skin diseases. Therefore, visitors are visiting Pir Lakha from far flung areas.