on device

GGUF powers most local LLM runs, and its quant names look cryptic. What matters is decoding what Q4_K_M means and why mixed-precision k-quants outperform naive uniform quantization.

On-device AI gives up raw capability for privacy, offline use, and latency, under tight memory and battery limits. The signal is the small-quantized-model stack plus a hybrid router that sends the hard queries to the cloud without leaking private context.

Detecting walking, driving, or a fall from a sensor stream is windowed time-series classification, typically on a battery-bound device. What matters is windowing, temporal smoothing, and the on-device constraints most candidates overlook. Here is the design.

On-device AI is a genuine product surface (privacy, offline, latency), and it imposes hard constraints. The signal is the quantization plus format plus runtime stack and the tradeoffs you accept under tight memory and battery budgets.

Millions of landmarks exist, most with only a handful of photos, and the next photo could show something not in your catalog at all. A flat classifier collapses on the long tail and never says 'I don't know.' The interview is embeddings plus retrieval plus a confident refusal.