Google announced the release of Magika 1.0, the first stable version of its AI-driven open-source file type detection system. Already used at scale within Google services, Magika now introduces an entirely rewritten Rust-based engine, significantly expanded format support, and lightning-fast inference performance.
As you probably know, file-type detection has been a longstanding challenge in computing: existing tools, such as the classic file utility relies on handcrafted heuristics to infer formats, a process that requires constant rule maintenance and remains fragile in the face of evolving file structures and adversarial content.
Taking this into consideration, magika approaches the problem differently: under the hood, it uses a compact custom deep-learning model (just a few me…
Google announced the release of Magika 1.0, the first stable version of its AI-driven open-source file type detection system. Already used at scale within Google services, Magika now introduces an entirely rewritten Rust-based engine, significantly expanded format support, and lightning-fast inference performance.
As you probably know, file-type detection has been a longstanding challenge in computing: existing tools, such as the classic file utility relies on handcrafted heuristics to infer formats, a process that requires constant rule maintenance and remains fragile in the face of evolving file structures and adversarial content.
Taking this into consideration, magika approaches the problem differently: under the hood, it uses a compact custom deep-learning model (just a few megabytes in size) trained on millions of files and designed to run efficiently even on a single CPU.
In Google’s internal benchmarks, the system was able to identify files within milliseconds and achieved an average accuracy of ~99% across hundreds of file types. The just-released new Magika 1.0 version introduces some major improvements, with the main one being:
- Support for more than 200 content types (up from ~100 in earlier versions).
- A new core engine rewritten in Rust, offering high performance and memory safety.
- Native Rust command-line client, alongside Python and TypeScript modules for easier integration.
- Improved accuracy on challenging textual and programmer-oriented formats (e.g., differentiating JSON vs JSONL, C vs C++).
- Near-constant inference time independent of file size: once the model is loaded, it can process thousands of files with minimal delay. Magika file identification tool.
For reference, internally at Google, Magika is deployed in services such as Gmail, Drive, and Safe Browsing, where it routes hundreds of billions of files each week into the appropriate policy and security scanners. The good news is that, due to the tool’s open-source nature, developers and organisations outside of Google can now benefit from the same detection engine.
To install and start using Magika, users can run a simple shell command on Linux/macOS:
curl -LsSf https://securityresearch.google/magika/install.sh | shCode language: Bash (bash)
On Windows, a PowerShell installation script is provided. The magika Python package also includes the native Rust CLI client under the hood. For developers, Magika offers bindings in Python, JavaScript/TypeScript (via an npm package), Rust, and a work-in-progress Go library.
For all the changes in the new Magika 1.0, see the Google Open Source Blog. For more information on the tools themselves, visit this link.
Image credits: Google
Bobby Borisov
Bobby, an editor-in-chief at Linuxiac, is a Linux professional with over 20 years of experience. With a strong focus on Linux and open-source software, he has worked as a Senior Linux System Administrator, Software Developer, and DevOps Engineer for small and large multinational companies.