What is muscriptor?

MuScriptor is an AI model that listens to an audio recording of music and automatically writes out what each instrument is playing, a process called music transcription. It was originally built by the research groups Kyutai and Mirelo and trained on a huge collection of 170,000 songs spanning genres from classical music to heavy metal, and this particular repository is a fork that adds a number of practical improvements on top of that original work. The fork adds a web interface with controls for telling the model which instruments to expect and for adjusting how the AI generates its output, more reliable audio playback in the browser using proper instrument sound banks, and the ability to download the transcribed result as either a synthesized audio file or the original MIDI file, which is a standard format for representing musical notes digitally. It also adds a way to convert saved MIDI files back into playable audio without having to re-run the whole transcription process, plus improvements aimed at deploying the tool on cloud GPU servers. Before using it, someone needs a free Hugging Face account, since the model's weights are hosted there under a non-commercial license and must be downloaded on first use after accepting that license. Once set up, a person can run it either as a local web app with a single command, or from the command line to transcribe a specific audio file. The project ships three model sizes, small, medium, and large, trading off speed and accuracy, where the smallest is practical to run on a regular computer while the largest is most accurate but really wants a graphics card. Developers can also use it directly as a Python library, importing a transcription model and calling a function that streams back musical note events, such as when a note starts and ends and which instrument played it, or requesting the output directly as a MIDI file.