Case study · AI hardware

Debugging an AI chip's software without the chip.

Lucia wanted to understand how quantization is implemented on Tenstorrent's AI accelerators. She does not own one of their cards. It turned out not to matter.

An abstract tensor grid with one dtype path highlighted in red

Setting up without hardware

pip install ttnn ships Tenstorrent's whole runtime, and ttsim, their open source simulator, is a 220 KB shared library that gives you a virtual Wormhole chip that runs the real kernels. A Docker container, a pip install and that simulator were enough to read, run and debug the stack.

What looked wrong

Reading through the quantization op, the same scale tensor, with the same dtype and the same numeric value, was accepted or rejected depending on parameters that had nothing to do with it: whether an axis was passed, and whether the zero point was a scalar or a tensor.

The cause: of four code paths, three widened the dtype before use and one did not.

Why the simulator was enough

The simulator cannot execute the quantization kernels themselves yet, a known limitation. It did not need to. That validation runs on the host, before any kernel is dispatched, so the failure was observable anyway.

Outcome

The report was accepted and a fix was opened against it. A small bug, which she rated low priority herself. The more interesting result is how low the barrier has become: the software of a modern AI chip is open to anyone willing to read it carefully.