Meta releases PyTorch inference framework for edge devices

ExecuTorch 1.0, a PyTorch-native inference framework that allows developers to deploy AI applications to edge devices, has been introduced by the PyTorch Team at Meta.

ExecuTorch 1.0 was introduced October 22; ExecuTorch code can be found on GitHub.

With this open source, general purpose framework, developers can deploy PyTorch models directly to edge devices, including mobile, desktop, and embedded, without needing conversion or rewriting. ExecuTorch offers hardware support across CPU, GPU, and NPU, supporting platforms such as iOS, Android, embedded devices, and laptop computers. On-device AI deployment brings machine learning capabilities directly to users’ devices, enabling fast, real-time responses without cloud reliance, the PyTorch Team said. ExecuTorch’s approach also improves privacy by keeping data local and supports personalized features that work even without Internet access, according to the team.

The framework enables developers to take any PyTorch-based model from any domain—large language models (LLM), vision-language models (VLM), image segmentation, image detection, audio, and more—and deploy it directly onto edge devices without the need to convert to other formats or rewrite the model. The team said ExecuTorch already is powering real-world applications including Instagram, WhatsApp, Messenger, and Facebook, accelerating innovation and adoption of on-device AI for billions of users.

Traditional on-device AI examples include running computer vision algorithms on mobile devices for photo editing and processing. But recently there has been rapid growth in new use cases driven by advances in hardware and AI models, such as local agents powered by LLMs and ambient AI applications in smart glasses and wearables, the PyTorch Team said. However, when deploying these novel models to on-device production environments such as mobile, desktop, and embedded applications, models often had to be converted to other runtimes and formats. These conversions are time-consuming for machine learning engineers and often become bottlenecks in the production deployment process due to issues such as numerical mismatches and loss of debug information during conversion.

ExecuTorch allows developers to build these novel AI applications using familiar PyTorch tools, optimized for edge devices, without the need for conversions. A beta release of ExecuTorch was announced a year ago.