Raspberry Pi AI HAT+ 2 ships: 40 TOPS for local generative AI

On January 15, Raspberry Pi announced the AI HAT+ 2 for Raspberry Pi 5. It carries a Hailo-10H accelerator rated at up to 40 TOPS, with 8 GB of dedicated on-board LPDDR4X memory, and went on sale immediately at $130. Unlike the usual announce-then-wait pattern, the board could be ordered the same day.

The biggest difference from its predecessor is what it can run: beyond computer vision, the AI HAT+ 2 runs language models and vision-language (VLM) workloads locally, and Raspberry Pi calls it its first AI product designed to fill the generative AI gap. In other words, the goal is not to help a Raspberry Pi see better but to let it converse, write code and describe what it sees.

Seen as a product line, this is a turn in Raspberry Pi’s AI strategy. The original AI HAT+, launched a little over a year earlier, was about making vision inference fast, aimed at cameras and machine vision; the AI HAT+ 2 changes the goal to running models — the same form factor serving a very different set of assumptions.

40 TOPS and 8 GB: what the two numbers weigh

Start with 40 TOPS. TOPS means trillions of operations per second and is the usual way to quote accelerator throughput — but the same number means different things at different precisions: the AI HAT+ 2’s 40 TOPS is an INT4 figure. Lower integer bit-widths pack more throughput into the same area at the cost of accuracy, so an edge chip’s performance figure has to be read together with its precision; comparing TOPS alone easily misleads.

The second number may matter more: 8 GB of dedicated memory. The earlier AI HAT+ offered 13 or 26 TOPS, aimed mainly at vision tasks, with no such separate memory; when running a language model, weights and KV cache both consume memory, and on-board capacity usually becomes the bottleneck sooner than peak throughput. Whether a model fits, and how long a context you can keep, is decided by that memory — which is why generative AI is on the table at all this time. Throughput decides whether something runs; memory decides how large — and the latter usually decides the experience.

What it runs out of the box

Language models available to install at launch include DeepSeek-R1-Distill (1.5 billion parameters), Llama3.2 (1 billion), Qwen2.5-Coder (1.5 billion), Qwen2.5-Instruct (1.5 billion) and Qwen2 (1.5 billion). More and larger models are being readied and should arrive soon after launch. The list shows a pattern: parameter counts cluster between 1 and 1.5 billion, matching the on-board memory.

In use, the official examples pair the hailo-ollama backend with the Open WebUI front end, giving a familiar chat interface in a browser, with all inference local. The examples cover general questions, coding tasks, French-to-English translation and a VLM describing a camera stream — that last one links camera to language model and shows best how this board differs from a pure vision accelerator. In engineering terms the examples matter not for their demos but for proving the deployment path: how a model is installed, how the backend starts, how the front end connects — all reproducible.

Expectations should stay realistic. Mainstream cloud models run from 500 billion to 2 trillion parameters, while edge models that fit in on-board memory typically run between 1 and 7 billion. Small models are not designed to match a large model’s knowledge but to work reliably within a constrained dataset — being dependable in one domain is more achievable than knowing everything. Every edge AI approach shares that premise: accept the ceiling on model size, then look for the most valuable tasks inside it.

Tunable and adjustable: LoRA and the camera path

Fine-tuning closes the gap. Vision models can be retrained on image datasets suited to the application; on the language side the AI HAT+ 2 supports LoRA (low-rank adaptation) fine-tuning — training a small set of adapter parameters while the base weights stay frozen, then compiling the adapter into the model. Hailo’s Dataflow Compiler handles that compilation, so generalise-then-specialise is a workable route. For industrial and educational settings that may matter more than the model list: only if you can adapt it can you fit your own data.

For developers already using the first AI HAT+, the transition is broadly smooth: vision models such as YOLO-based object detection, pose estimation and scene segmentation perform roughly on par with the 26 TOPS predecessor, thanks to the on-board memory, and integration with the camera software stack (libcamera, rpicam-apps, Picamera2) is unchanged. The board connects over PCIe, which is why it targets Raspberry Pi 5. It also means it cannot go on earlier Raspberry Pi models — the interface, not the software, sets compatibility.

It is not in the same class as an MCU-level NPU

The positioning deserves clarity: a Pi plus AI HAT is a Linux single-board computer with an external NPU. It is not in the same class as an MCU-level NPU (accelerator-equipped microcontrollers such as the STM32N6 or MCX N947) — power, boot complexity and latency are all higher, but the models it can run are far larger. The dividing line is clear: if a task needs millisecond boot and microamp standby, go to the MCU; if it needs a language model, a screen and networking, a single-board computer with an accelerator is the realistic choice.

The HAT form factor is worth noting too: it stacks on top of the main board and uses the existing connections and power to add function as a module you can attach when needed. For hobbyist and education settings that is far friendlier than redesigning a whole board, and it simplifies the upgrade path — keep the board, change the compute. Against other options of the moment, its value lies more in accessible compute than in maximum compute.

Price and the barrier to entry deserve a sentence too. $130 is not trivial for an individual developer, but against the GPU and full machine previously needed for the same class of model, it reduces the step to adding one add-on board. That explains the attention better than any single specification.

The significance is accessibility: entry-level edge generative AI now runs on a single-board computer hobbyists already know, and the all-in-one local LLM board has entered that ecosystem. It will not replace cloud models, but it makes the offline, nothing-uploaded, no-subscription route affordable — if data never leaves the device, there is nothing to upload. Its limits are clear, though: model size is capped, concurrency is modest, and a Raspberry Pi 5 is still the base — none of which fits a do-everything expectation.