The Music Department began its independent activity within the library in 1923 under the leadership of philologist and composer Ladislav Vycpálek. He advocated placing the musical works scattered throughout the library collection in a separate music section under a call sign beginning with the number 59 and their proper cataloguing. Subsequently, he ensured that the legal deposit copies of musical works were delivered correctly to the library and recovered many undelivered copies. Collections of music manuscripts and manuscript correspondence of important personalities of musical culture were established in the same year that the department started its independent activities.
Under the leadership of his successor Marie Svobodová (1908-1995), who took over the post in 1945, the department acquired a reading room (the Old Mathematics Hall of the Klementinum), in which a reference library was established. As early as 1956, the Czechoslovak Republic, on the initiative of the Music Department of the National Library (then the State Library of the Czechoslovak Socialist Republic), joined the activities of the International Association of Music Libraries, Archives and Documentation Centres (IAML), founded in Paris in 1951. Particular focus was placed on participation in the leading project of this organisation - RISM (Répertoire International des Sources Musicales - International Inventory of Musical Sources), launched in Paris in 1952. The Czechoslovak National Group of the IAML was established in 1971.
Thanks to Svobodová's initiative, one of the library's special collections - the “Mozart Memorial” - came under the administration of the Music Department. The institution of the world's first Mozart collection, commemoratively housed in the Klementinum, was founded in 1837 on the 50th anniversary of the world premiere of Mozart's Don Giovanni in Prague.
After Jarmila Brožovská (1927-1994) took over the Music Department in 1965, it began building the Collective Music Catalogue as part of the central inventory of musical sources in the Czech Republic. Its task, related to the mission of RISM, was to map the preserved historical musical monuments in the country. For many years, Jitřenka Pešková (1930-2006), who served as head of the Music Department from 1977 to 1989, was the principal executive force behind this unique Czech musicology project. The information gathered in the Union Catalogue of Music prompted the establishment of the National Library's (at that time the State Library of the Czechoslovak Socialist Republic) specialist series of publications entitled Catalogus artis musicae in Bohemia et Moravia cultae - thematic catalogues of individual important church and secular collections preserved in our territory.
During the 1970s, the department's publishing activities further developed in methodology, cataloguing, and bibliography.
Julius Hůlek (*1947) was the head of the department between 1989 and 1997. From 1980 to 1998, he chaired the Czech National Group of the IAML, and in the field of international cooperation, he served as Vice President of the IAML from 1989 to 1992.
From 1997 to June 2023, the department was headed by Zuzana Petrášková (*1948), who had continuously coordinated the work for the RISM inventory of music sources for the Czech Republic since 1989. From 1999 to 2005, she was the Chair of the Czech National Group of the IAML. In 2016, she received the Z. V. Tobolka Medal for merit.