AI companies release tools for on-device machine learning and local processing

AI companies are shifting toward local processing with new tools designed for on-device machine learning. Perplexity has launched Portable Computer, an agentic AI system running locally on Linux with high-end Nvidia hardware, while Google's TensorFlow Lite enables mobile algorithms on Android and iOS. To address the complexity of local deployment, Liquid AI has open-sourced Pipette, a benchmarking suite validating model performance across specific hardware, quantization, and runtime configurations.

Pipette on-device benchmarking platform

• ▪Liquid AI, in partnership with Artificial Analysis, released Pipette, an open-source platform for benchmarking foundation models on edge devices.
• ▪Pipette measures on-device behavior as a full configuration consisting of model, quantization, runtime, and device across five performance metrics.
• ▪Pipette launched with a dataset covering more than 1,000 configurations, spanning over 30 models, and verified results from a MacBook Pro with M5 Max, an iPhone 17 Pro, and a Galaxy S26 Ultra.

On-device AI deployment drivers

• ▪On-device AI deployment is driven by industries such as consumer electronics, automotive, robotics, healthcare, finance, and defense where latency, privacy, or connectivity require local inference.
• ▪Model cards reporting quality under server-class, full-precision conditions rarely predict how the same model behaves on a phone, driving the need for localized system-level benchmarking.

TensorFlow Lite mobile machine learning

• ▪Google released a preview of TensorFlow Lite, an open-source software designed to run machine learning algorithms directly on mobile devices without connecting to the cloud.
• ▪TensorFlow Lite is designed to leverage existing hardware, such as Snapdragon processors inside smartphones, to run algorithms on Android and iOS.

Perplexity Portable Computer local AI

• ▪Perplexity Portable Computer runs tasks locally but can request user permission to use a cloud model for assistance when encountering tasks it cannot handle.
• ▪Perplexity launched Portable Computer, an AI tool designed to run agentic AI processing directly on a user's local machine without relying on cloud computing.
• ▪Perplexity Portable Computer packages a local AI model, inference engine, agent tools, app connectors, and an isolated sandbox into a single system.

High-end hardware requirements

• ▪At launch, Perplexity Portable Computer requires Linux and an Nvidia system with at least 24GB of VRAM, limiting its initial accessibility to high-end hardware.
• ▪Perplexity plans to release Windows support for Portable Computer in September 2026, while macOS support has not been announced.

Privacy through local processing

• ▪Local processing reduces cloud computing costs for providers and enables offline use cases where devices do not need network connectivity.
• ▪On-device machine learning tools like TensorFlow Lite and Perplexity Portable Computer aim to address user privacy concerns by keeping sensitive data on the local device.