Example project · Builds & projects
A tiny AI storyteller (ESP32-S3)
A real 28.9-million-parameter TinyStories language model (slvDev/esp32-ai, MIT) running entirely on the emulated ESP32-S3: the 15.5 MB app image with the embedded model fills a 16 MB flash chip, the int8-staged weights and KV cache live in the 8 MB octal PSRAM, and both LX7 cores split every matrix row. The story streams to the big 2.8″ color TFT — your prompt in yellow, the tale in white, and a live tokens-per-second footer fed over real hardware SPI (every pixel costs honest bus time). Type a few words into the serial monitor and press Enter: the model tokenizes your prompt on-device and writes you a new story at its true hardware-class rate, ~8 tokens/second with the panel attached, courtesy of the JIT that compiles the hot int8 dot-product loop to WebAssembly on the fly.
What's on the bench
- Battery
- ESP32-S3
- TFT 2.8″ 240×320 (ST7789)
How it's wired
- Battery · pos→ESP32-S3 · vin
- Battery · neg→ESP32-S3 · gnd
- ESP32-S3 · 3v3→TFT 2.8″ 240×320 (ST7789) · vcc
- ESP32-S3 · gnd2→TFT 2.8″ 240×320 (ST7789) · gnd
- ESP32-S3 · g6→TFT 2.8″ 240×320 (ST7789) · clk
- ESP32-S3 · g7→TFT 2.8″ 240×320 (ST7789) · din
- ESP32-S3 · g5→TFT 2.8″ 240×320 (ST7789) · cs
- ESP32-S3 · g3→TFT 2.8″ 240×320 (ST7789) · dc
- ESP32-S3 · g4→TFT 2.8″ 240×320 (ST7789) · rst
The code
The exact ESP-IDF v5.3.5 project the bundled firmware was compiled from — shown in the simulator’s Code tab, where you can edit the sources and press Compile & run to rebuild the board’s firmware in the cloud sandbox.
CMakeLists.txt
cmake_minimum_required(VERSION 3.16) include($ENV{IDF_PATH}/tools/cmake/project.cmake) # Upstream ships at global -O3 (scripts/deploy.sh; matvec_i8_range carries the # one noinline the runtime needs). -ffp-contract=off on top: the host oracle # (host/build.sh) compiles the same llm.h with the same flag, so neither side # fuses a*b+c into an FMA the other did not — the story diff must compare # arithmetic, not compiler contraction choices. idf_build_set_property(COMPILE_OPTIONS "-O3;-ffp-contract=off" APPEND) project(ple-s3)
sdkconfig.defaults
# ple-s3 — deterministic 28.9M PLE TinyLM inference on the ESP32-S3 N16R8. CONFIG_IDF_TARGET="esp32s3" # 16 MB flash, custom table: one 15.875 MB factory app holding the embedded # 14.9 MB model. (esp32s3-memprobe/sdkconfig.psram is the base profile.) CONFIG_ESPTOOLPY_FLASHSIZE_16MB=y CONFIG_PARTITION_TABLE_CUSTOM=y CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions.csv" # 8 MB octal PSRAM (DevKitC-1 N16R8). SPIRAM_BOOT_INIT + the boot memtest # stay on: reaching app_main proves the octal bring-up and the full sweep. CONFIG_SPIRAM=y CONFIG_SPIRAM_MODE_OCT=y CONFIG_ESP32S3_DATA_CACHE_32KB=y # Full 240 MHz. The project CMakeLists appends -O3 -ffp-contract=off (see the # comment there); COMPILER_OPTIMIZATION_PERF keeps IDF's own default sane. CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ_240=y CONFIG_COMPILER_OPTIMIZATION_PERF=y # One console only: the default secondary USB-Serial-JTAG console duplicates # every byte in the emulator's merged serial stream. CONFIG_ESP_CONSOLE_SECONDARY_NONE=y # Token generation is a legitimate long CPU burst on the main task (and the # core-0 matvec worker); do not arm the task watchdog. # CONFIG_ESP_TASK_WDT_INIT is not set CONFIG_ESP_MAIN_TASK_STACK_SIZE=8192 # Keep DIRAM headroom: move IRAM-optional code to flash, ROM flash driver. CONFIG_FREERTOS_PLACE_FUNCTIONS_INTO_FLASH=y CONFIG_RINGBUF_PLACE_FUNCTIONS_INTO_FLASH=y CONFIG_HEAP_PLACE_FUNCTION_INTO_FLASH=y CONFIG_SPI_FLASH_ROM_IMPL=y # Pin the app version string so rebuilt artifacts don't embed the building # worktree's `git describe`. CONFIG_APP_PROJECT_VER_FROM_CONFIG=y CONFIG_APP_PROJECT_VER="bre101-1"
partitions.csv
# Name, Type, SubType, Offset, Size # One 15.875 MB factory app: code + the .incbin-embedded 14.9 MB model. nvs, data, nvs, 0x9000, 0x6000 phy_init, data, phy, 0xf000, 0x1000 factory, app, factory, 0x10000, 0xFE0000
main/CMakeLists.txt
idf_component_register( SRCS "ple_main.c" "model_blobs.S" PRIV_REQUIRES esp_driver_uart esp_driver_spi esp_driver_gpio vfs esp_timer ) # The .S embeds the assets via .incbin: hand it absolute paths, and rebuild # when the blobs change. target_compile_definitions(${COMPONENT_LIB} PRIVATE "MODEL_PATH=\"${CMAKE_CURRENT_SOURCE_DIR}/model/model.bin\"" "VOCAB_PATH=\"${CMAKE_CURRENT_SOURCE_DIR}/model/vocab.bin\"" "BTK_PATH=\"${CMAKE_CURRENT_SOURCE_DIR}/model/tokenizer.btk\"" ) set_property(SOURCE model_blobs.S APPEND PROPERTY OBJECT_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/model/model.bin" "${CMAKE_CURRENT_SOURCE_DIR}/model/vocab.bin" "${CMAKE_CURRENT_SOURCE_DIR}/model/tokenizer.btk" )
main/ple_main.c
/* * ple-s3 — the 28.9M-parameter Per-Layer-Embeddings TinyStories model * (slvDev/esp32-ai, MIT, (c) 2026 Viacheslav Sierbov) on the ESP32-S3, * ported from the upstream Arduino sketch to ESP-IDF for the Breadboard * emulator's N16R8 board profile (16 MB flash, 8 MB octal PSRAM). * * Memory placement follows upstream, by reads-per-token: * * flash PLE table + token embedding ~6 random rows (~450 B) per token, * read in place from the DROM window * PSRAM staged int8 core + head, read once per position * KV cache, logits * SRAM scratch + norm vectors touched many times per token * * Platform seams vs the upstream sketch (everything else, including the * dual-core row-split matvec, is kept): * * - the model is .incbin-embedded in the app image (main/model_blobs.S, * 16-byte aligned) instead of a separate mmap'd flash partition: the * emulator's engine seam loads one app image, and either way the bytes * are read through the same flash-cache window, * - console writes BLOCK (upstream drops story bytes when no USB host * drains the buffer — fatal to a byte-identical golden), * - an interactive UART0 prompt loop encodes typed ASCII prompts on-device * with upstream's runtime/bpe_tokenizer.h (their barista firmware's * encoder; TinyStories upstream is decode-only with a fixed prompt), * - no display, no RGB blink, task watchdog off in sdkconfig instead of * delay(0) feeds (generation is a legitimate long CPU burst), * - the whole generation loop lives in gen_core.h, shared verbatim with * host/ple_story.c: the host-native oracle byte-diffs the story stream. */ #include <stdint.h> #include <stdio.h> #include <string.h> #include "driver/uart.h" #include "driver/uart_vfs.h" #include "esp_heap_caps.h" #include "esp_timer.h" #include "freertos/FreeRTOS.h" #include "freertos/task.h" #include "tft.h" /* int8 activations, as the shipping upstream runtime: the staged int8 head * is the whole reason the model reaches ~10 tok/s on silicon. The host * oracle builds with the same flag. */ #define LLM_INT8_ACT 1 #include "gen_core.h" #define MAX_PROMPT_CHARS 256 /* Blobs embedded by model_blobs.S with 16-byte alignment. */ extern const uint8_t model_start[] asm("ple_model_start"); extern const uint8_t model_end[] asm("ple_model_end"); extern const uint8_t vocab_start[] asm("ple_vocab_start"); extern const uint8_t vocab_end[] asm("ple_vocab_end"); extern const uint8_t btk_start[] asm("ple_btk_start"); extern const uint8_t btk_end[] asm("ple_btk_end"); static PleRuntime rt; static size_t psram_used = 0, sram_used = 0; /* Allocations are strict: memory placement is part of the runtime * configuration, so a miss halts loudly instead of running elsewhere. */ static void *ps_alloc(size_t n) { void *p = heap_caps_malloc(n, MALLOC_CAP_SPIRAM); if (p) psram_used += n; return p; } static void *sram_alloc(size_t n) { void *p = heap_caps_malloc(n, MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT); if (p) sram_used += n; return p; } static void fatal(const char *what) { printf("[ple] FATAL: %s\n", what); for (;;) vTaskDelay(pdMS_TO_TICKS(1000)); } /* ---- dual-core int8 matvec, upstream's scheme unchanged: quantize once, * split the rows across both LX7 cores. Row results are independent, so the * split cannot change the numbers, only the wall time. Below 128 rows the * notify round trip costs more than the split saves. */ static TaskHandle_t worker_h, main_h; static const QT *job_t; static const int8_t *job_xq; static float job_xs; static float *job_y; static int job_split; static void worker_main(void *arg) { (void)arg; for (;;) { ulTaskNotifyTake(pdTRUE, portMAX_DELAY); matvec_i8_range(job_t, job_xq, job_xs, job_y, 0, job_split); xTaskNotifyGive(main_h); } } static void matvec_par(const QT *t, const float *x, float *y) { static int8_t xq[LLM_Q8_MAX_INPUT]; float xs; if (t->w8 == NULL || t->rows < 128) { MATVEC(t, x, y); return; } quantize_act(x, t->cols, xq, &xs); /* once; both cores read the result */ job_t = t; job_xq = xq; job_xs = xs; job_y = y; job_split = t->rows / 2; xTaskNotifyGive(worker_h); matvec_i8_range(t, xq, xs, y, job_split, t->rows); ulTaskNotifyTake(pdTRUE, portMAX_DELAY); } /* Copy the RMSNorm vectors from mapped flash to internal SRAM (upstream's * copy_norms_to_sram; numerics-neutral, they are read many times a token). */ static void copy_norms_to_sram(void) { Model *m = &rt.m; int D = m->c.dim, L = m->c.n_layers, P = m->c.ple_dim; const float **vecs[3 * LLM_MAX_LAYERS + 2]; int sizes[3 * LLM_MAX_LAYERS + 2], n_vec = 0; vecs[n_vec] = &m->ple_proj_norm; sizes[n_vec++] = P; for (int l = 0; l < L; l++) { vecs[n_vec] = &m->attn_norm[l]; sizes[n_vec++] = D; vecs[n_vec] = &m->ffn_norm[l]; sizes[n_vec++] = D; vecs[n_vec] = &m->ple_norm[l]; sizes[n_vec++] = D; } vecs[n_vec] = &m->out_norm; sizes[n_vec++] = D; for (int i = 0; i < n_vec; i++) { size_t bytes = (size_t)sizes[i] * sizeof(float); void *dst = sram_alloc(bytes); if (!dst) fatal("norm vector SRAM allocation"); memcpy(dst, *vecs[i], bytes); *vecs[i] = (const float *)dst; } printf("[ple] norms -> SRAM: %d vectors\n", n_vec); } /* Story bytes go straight to the UART0 console and always block: every byte * is emitted, which is what the byte-identical oracle diff needs. The TFT * console (tft.h) only OBSERVES the same bytes — it must never alter, * reorder or gate the UART stream. */ static void emit_console(const void *bytes, int len) { fwrite(bytes, 1, (size_t)len, stdout); tft_story_bytes(bytes, len); } static void console_init(void) { uart_driver_install(UART_NUM_0, 256, 0, 0, NULL, 0); uart_vfs_dev_use_driver(UART_NUM_0); uart_vfs_dev_port_set_rx_line_endings(UART_NUM_0, ESP_LINE_ENDINGS_CR); } static void read_prompt(char *buf, size_t cap) { if (fgets(buf, (int)cap, stdin) == NULL) { buf[0] = '\0'; return; } buf[strcspn(buf, "\r\n")] = '\0'; } /* One full story: encode, echo the region markers, generate, report. The * region between "[ple] story:" and "[ple] story done" is what the host * oracle reproduces byte-for-byte; the timing line stays outside it. */ static void run_story(const char *prompt) { static int ids[PLE_MAX_PROMPT_IDS]; int n = ple_encode(&rt, prompt, ids, PLE_MAX_PROMPT_IDS); if (n < 0) { printf("[ple] prompt rejected (ASCII only, max %d bytes)\n", BTK_MAX_INPUT_BYTES); return; } if (n == 0) { printf("[ple] prompt produced no tokens\n"); return; } printf("[ple] story (%d prompt tokens):\n\n", n); tft_story_begin(prompt); int64_t t0 = esp_timer_get_time(); int generated = ple_generate(&rt, ids, n); int64_t us = esp_timer_get_time() - t0; tft_story_end(); printf("\n\n[ple] story done: %d prompt + %d generated tokens\n", n, generated); printf("[ple] %.2f s, %.2f tok/s\n", us / 1e6, generated * 1e6 / (double)us); } void app_main(void) { setvbuf(stdout, NULL, _IONBF, 0); printf("\n[ple] PLE TinyLM 28.9M (slvDev/esp32-ai) on ESP32-S3\n"); tft_init(); /* bench TFT (BRE-100 example); console output is unaffected */ size_t model_size = (size_t)(model_end - model_start); const char *err = ple_bind_assets(&rt, model_start, model_size, vocab_start, (size_t)(vocab_end - vocab_start), btk_start, (size_t)(btk_end - btk_start)); if (err) fatal(err); Cfg *c = &rt.m.c; printf("[ple] model: Vin=%d Vout=%d D=%d L=%d H=%d F=%d P=%d S=%d group=%d (image %u B)\n", c->vocab, rt.m.out_vocab, c->dim, c->n_layers, c->n_heads, c->ffn, c->ple_dim, c->seq_len, c->group, (unsigned)rt.m.image_bytes); err = ple_alloc_scratch(&rt, sram_alloc, ps_alloc); if (err) fatal(err); copy_norms_to_sram(); int want = ple_stage_want(&rt); int staged = ple_stage(&rt, ps_alloc); if (staged != want) fatal("int8 staging PSRAM allocation"); printf("[ple] staged -> PSRAM: %d tensors int8, %.2f MB (sram %u B)\n", staged, psram_used / 1048576.0, (unsigned)sram_used); /* FNV-1a over the mapped image; upstream's deploy.sh prints the same value * for the file it flashes. The goldens pin it. */ printf("[ple] image fp=%08x\n", (unsigned)ple_fingerprint(&rt, model_start)); printf("[ple] free: sram %u KB | psram %.2f MB\n", (unsigned)(heap_caps_get_free_size(MALLOC_CAP_INTERNAL) / 1024), heap_caps_get_free_size(MALLOC_CAP_SPIRAM) / 1048576.0); main_h = xTaskGetCurrentTaskHandle(); if (xTaskCreatePinnedToCore(worker_main, "mv", 4096, NULL, 2, &worker_h, 0) == pdPASS) { rt.m.layer_matvec = matvec_par; rt.m.head_matvec = matvec_par; printf("[ple] dual-core: matvec worker on core 0\n"); } else { printf("[ple] dual-core worker failed; running single core\n"); } rt.emit_bytes = emit_console; printf("[ple] boot story (greedy, prompt \"Once upon a time\"):\n"); run_story("Once upon a time"); printf("[ple] boot story done\n"); /* The boot story above is greedy — the cross-platform oracle probe. * Interactive prompts sample (Gumbel-max, seeded per story): livelier * stories, still a pure function of the typed inputs. */ rt.sampled = 1; console_init(); static char line[MAX_PROMPT_CHARS]; for (;;) { printf("\nprompt> "); read_prompt(line, sizeof(line)); printf("[ple prompt: %s]\n", line); run_story(line[0] != '\0' ? line : "Once upon a time"); } }
main/llm.h
// Portable single-header inference for the PLE TinyLM. Same code runs on the // host (verify against PyTorch golden) and on the ESP32-S3 (mmap'd flash). No // dynamic arch: dims come from the model.bin header. Weights are bound in place // from `base` by default; platforms may relocate hot tensors and override the // output-head matvec. // // Matches src/model.py op-for-op: split-half RoPE, erf-GELU, SiLU-SwiGLU, // RMSNorm(weight * x * rsqrt(mean(x^2)+eps)), tied input/output embedding, and // the PLE input (RMSNorm(proj(x)/sqrt(D)) + table[tok]*sqrt(P)) / sqrt(2) // gated per layer as x += RMSNorm(ple_proj(gelu(ple_gate(x)) * ple_l)). #ifndef LLM_H #define LLM_H #include <stdint.h> #include <math.h> #include <string.h> #define LLM_MAGIC 0x00454C50u /* "PLE\0" */ #define LLM_FORMAT_VERSION 1u #define LLM_FLAG_TIED_HEAD 1u /* head is the token embedding */ #define LLM_FLAGS_KNOWN LLM_FLAG_TIED_HEAD #define LLM_HEADER_BYTES 56u /* version 1 */ #define LLM_HEADER_MAX 4096u /* sanity bound on a future header */ #define LLM_MAX_LAYERS 32 /* fixed per-layer arrays below */ #define RMS_EPS 1e-6f #define LLM_Q8_MAX_INPUT 4096 typedef struct { int vocab, dim, n_layers, n_heads, ffn, ple_dim, seq_len, group; float rope_theta; } Cfg; // A group-wise int4 tensor viewed in place: ragged packed nibbles (row-aligned // to a byte) + fp16 group scales. Per-tensor group. No padding. typedef struct { const uint8_t *codes; // rows*row_bytes, nibble = value+8, row_bytes=ceil(cols/2) const uint16_t *scales; // rows*n_groups fp16 int rows, cols, group, n_groups, row_bytes; /* Optional int8 staging: nibbles unpacked and group scales converted once, so * each matvec skips both. Numerics are unchanged - same codes, same scales, * same group sums. NULL selects the int4 path. */ const int8_t *w8; // rows*cols, values -7..7 const float *scale8; // rows*n_groups, half2float'd once } QT; // IEEE half -> float. static inline float half2float(uint16_t h) { uint32_t sign = (uint32_t)(h & 0x8000) << 16; uint32_t exp = (h >> 10) & 0x1F, man = h & 0x3FF, f; if (exp == 0) { if (man == 0) f = sign; else { exp = 127 - 15 + 1; while (!(man & 0x400)) { man <<= 1; exp--; } man &= 0x3FF; f = sign | (exp << 23) | (man << 13); } } else if (exp == 0x1F) { f = sign | 0x7F800000u | (man << 13); } else { f = sign | ((exp - 15 + 127) << 23) | (man << 13); } float out; memcpy(&out, &f, 4); return out; } typedef struct { Cfg c; QT tok_emb; // [V, D] input embedding QT out_head; // [Vout, D]; first Vout rows of tok_emb when tied int out_vocab; // logits produced per step QT ple_model_proj; // [L*P, D] const float *ple_proj_norm; // [P] QT ple_table; // [V, L*P] const float *attn_norm[LLM_MAX_LAYERS]; // [D] QT qkv[LLM_MAX_LAYERS]; // [3D, D] QT attn_proj[LLM_MAX_LAYERS]; // [D, D] const float *ffn_norm[LLM_MAX_LAYERS]; // [D] QT gate[LLM_MAX_LAYERS], up[LLM_MAX_LAYERS], down[LLM_MAX_LAYERS]; QT ple_gate[LLM_MAX_LAYERS]; // [P, D] QT ple_proj[LLM_MAX_LAYERS]; // [D, P] const float *ple_norm[LLM_MAX_LAYERS]; // [D] const float *out_norm; // [D] size_t image_bytes; // bytes consumed by the image, which is smaller than // the flash partition holding it // Optional platform overrides; NULL uses MATVEC. Separate hooks because a // tied 32,768-row head and a 66-row FFN matrix have different cost profiles. void (*head_matvec)(const QT *, const float *, float *); void (*layer_matvec)(const QT *, const float *, float *); } Model; // Advance a cursor over the file, binding one quant tensor. Reads the per-tensor // group prefix, then ragged codes + fp16 scales. static const uint8_t *bind_q(const uint8_t *p, QT *t, int rows, int cols) { int32_t group; memcpy(&group, p, 4); p += 4; t->rows = rows; t->cols = cols; t->group = group; t->n_groups = (cols + group - 1) / group; t->row_bytes = (cols + 1) / 2; t->codes = p; p += (size_t)rows * t->row_bytes; t->scales = (const uint16_t *)p; p += (size_t)rows * t->n_groups * 2; t->w8 = NULL; t->scale8 = NULL; // int4 path until staged return p; } static const uint8_t *bind_f(const uint8_t *p, const float **t, int n) { *t = (const float *)p; return p + (size_t)n * sizeof(float); } // Dequantize row r of a quant tensor into out[cols]. static inline void deq_row(const QT *t, int r, float *out) { const uint8_t *row = t->codes + (size_t)r * t->row_bytes; const uint16_t *sc = t->scales + (size_t)r * t->n_groups; for (int gi = 0; gi < t->n_groups; gi++) { int begin = gi * t->group; int end = begin + t->group; if (end > t->cols) end = t->cols; float scale = half2float(sc[gi]); int j = begin; if ((j & 1) && j < end) { out[j] = (float)((row[j >> 1] >> 4) - 8) * scale; j++; } for (; j + 1 < end; j += 2) { uint8_t byte = row[j >> 1]; out[j] = (float)((byte & 0xF) - 8) * scale; out[j + 1] = (float)((byte >> 4) - 8) * scale; } if (j < end) { uint8_t byte = row[j >> 1]; int code = (j & 1) ? (byte >> 4) : (byte & 0xF); out[j] = (float)(code - 8) * scale; } } } // y[row_begin:row_end] = W * x for a quant tensor W. Keeping the row range // explicit lets platforms parallelize the large output head without changing // any individual dot product. static inline void matvec_q_range(const QT *t, const float *x, float *y, int row_begin, int row_end) { for (int r = row_begin; r < row_end; r++) { const uint8_t *row = t->codes + (size_t)r * t->row_bytes; const uint16_t *sc = t->scales + (size_t)r * t->n_groups; float acc = 0.f; for (int gi = 0; gi < t->n_groups; gi++) { int begin = gi * t->group; int end = begin + t->group; if (end > t->cols) end = t->cols; float scale = half2float(sc[gi]); float group_acc = 0.f; int j = begin; if ((j & 1) && j < end) { group_acc += (float)((row[j >> 1] >> 4) - 8) * x[j]; j++; } for (; j + 1 < end; j += 2) { uint8_t byte = row[j >> 1]; group_acc += (float)((byte & 0xF) - 8) * x[j]; group_acc += (float)((byte >> 4) - 8) * x[j + 1]; } if (j < end) { uint8_t byte = row[j >> 1]; int code = (j & 1) ? (byte >> 4) : (byte & 0xF); group_acc += (float)(code - 8) * x[j]; } acc += group_acc * scale; } y[r] = acc; } } // y[rows] = W * x[cols], W a quant tensor [rows, cols]. static inline void matvec_q(const QT *t, const float *x, float *y) { matvec_q_range(t, x, y, 0, t->rows); } // ---- int8-activation path --------------------------------------------------- // Quantize x to int8 once per call, then per output row do int8*int8 -> int32 // group dot products, scaled by (x_scale * group_scale). The only approximation // against matvec_q is the int8 activations. Split into quantize + ranged-dot so // a parallel caller can quantize once and let both cores read the result. static inline void quantize_act(const float *x, int n, int8_t *xq, float *x_scale) { float xmax = 1e-8f; for (int j = 0; j < n; j++) { float a = fabsf(x[j]); if (a > xmax) xmax = a; } float inv = 127.f / xmax; for (int j = 0; j < n; j++) { int q = (int)lrintf(x[j] * inv); xq[j] = (int8_t)(q > 127 ? 127 : (q < -127 ? -127 : q)); } *x_scale = xmax / 127.f; } static inline void matvec_q8_range(const QT *t, const int8_t *xq, float x_scale, float *y, int row_begin, int row_end) { for (int r = row_begin; r < row_end; r++) { const uint8_t *row = t->codes + (size_t)r * t->row_bytes; const uint16_t *sc = t->scales + (size_t)r * t->n_groups; float acc = 0.f; for (int gi = 0; gi < t->n_groups; gi++) { int begin = gi * t->group, end = begin + t->group; if (end > t->cols) end = t->cols; int32_t g = 0; // group accumulator for (int j = begin; j < end; j++) { uint8_t byte = row[j >> 1]; int code = (j & 1) ? (byte >> 4) : (byte & 0xF); g += (code - 8) * (int)xq[j]; } acc += (float)g * half2float(sc[gi]); } y[r] = acc * x_scale; } } /* Scalar int8 dot product. */ static inline int32_t llm_dot_i8(const int8_t *a, const int8_t *b, int n) { int32_t acc = 0; for (int i = 0; i < n; i++) acc += (int32_t)a[i] * (int32_t)b[i]; return acc; } /* Same arithmetic as matvec_q8_range, reading pre-unpacked int8 weights. * * Keep out of line on ESP32-S3: with Arduino ESP32 3.3.10 at -O3, inlining * regressed inference from 95.0 to 155.2 ms/token. */ #if defined(__GNUC__) __attribute__((noinline)) #endif static void matvec_i8_range(const QT *t, const int8_t *xq, float x_scale, float *y, int row_begin, int row_end) { int g = t->group, ng = t->n_groups, cols = t->cols; for (int r = row_begin; r < row_end; r++) { const int8_t *w = t->w8 + (size_t)r * cols; const float *sc = t->scale8 + (size_t)r * ng; float acc = 0.f; for (int gi = 0; gi < ng; gi++) { int begin = gi * g, end = begin + g; if (end > cols) end = cols; acc += (float)llm_dot_i8(w + begin, xq + begin, end - begin) * sc[gi]; } y[r] = acc * x_scale; } } static inline void matvec_q8(const QT *t, const float *x, float *y) { static int8_t xq[LLM_Q8_MAX_INPUT]; float xs; quantize_act(x, t->cols, xq, &xs); if (t->w8 != NULL) { matvec_i8_range(t, xq, xs, y, 0, t->rows); return; } matvec_q8_range(t, xq, xs, y, 0, t->rows); } // llm_forward dispatches through MATVEC so one flag flips the whole model. #ifdef LLM_INT8_ACT #define MATVEC matvec_q8 #else #define MATVEC matvec_q #endif // w is bound straight into the model image, where fp32 tensors follow // byte-packed quantized ones and so need not be 4-byte aligned. x and out are // caller scratch and always are. static inline void rmsnorm(const float *x, const float *w, int n, float *out) { float ss = 0.f; for (int i = 0; i < n; i++) ss += x[i] * x[i]; float inv = 1.f / sqrtf(ss / n + RMS_EPS); if (((uintptr_t)w & (sizeof(float) - 1)) == 0) { for (int i = 0; i < n; i++) out[i] = w[i] * x[i] * inv; return; } const uint8_t *wb = (const uint8_t *)w; for (int i = 0; i < n; i++) { float wi; memcpy(&wi, wb + (size_t)i * sizeof(float), sizeof wi); out[i] = wi * x[i] * inv; } } static inline float gelu(float x) { return 0.5f * x * (1.f + erff(x * 0.70710678f)); } static inline float silu(float x) { return x / (1.f + expf(-x)); } // Parse header + bind all tensors. Returns 0 on ok, -1 on bad magic. static int llm_load(const uint8_t *base, Model *m) { const uint8_t *p = base; uint32_t h[4]; memcpy(h, p, 16); p += 16; /* magic, version, header_bytes, flags */ if (h[0] != LLM_MAGIC) return -1; /* Version 0 is not a version. A newer one may have changed tensor order, so * refuse rather than mis-bind. An unknown flag bit means the writer asked for * behaviour this reader does not implement, which is equally unsafe. */ if (h[1] == 0 || h[1] > LLM_FORMAT_VERSION) return -3; if (h[2] < LLM_HEADER_BYTES || h[2] > LLM_HEADER_MAX) return -2; if (h[3] & ~(uint32_t)LLM_FLAGS_KNOWN) return -3; uint32_t flags = h[3]; uint32_t vio[2]; memcpy(vio, p, 8); p += 8; /* input_vocab, output_vocab */ m->c.vocab = (int)vio[0]; m->out_vocab = (int)vio[1]; int32_t hv[7]; memcpy(hv, p, 28); p += 28; m->c.dim = hv[0]; m->c.n_layers = hv[1]; m->c.n_heads = hv[2]; m->c.ffn = hv[3]; m->c.ple_dim = hv[4]; m->c.seq_len = hv[5]; m->c.group = hv[6]; memcpy(&m->c.rope_theta, p, 4); p += 4; /* Skip any fields a later version appended. */ p = base + h[2]; /* Every dimension below indexes a fixed array or sizes an allocation, so a * malformed header must be rejected before any tensor is bound. */ if (m->c.vocab <= 0 || m->out_vocab <= 0 || m->c.dim <= 0 || m->c.ffn <= 0 || m->c.ple_dim <= 0 || m->c.seq_len <= 0 || m->c.group <= 0 || m->c.n_layers <= 0 || m->c.n_layers > LLM_MAX_LAYERS || m->c.n_heads <= 0 || m->c.dim % m->c.n_heads != 0 || (m->c.dim / m->c.n_heads) % 2 != 0) return -2; /* A tied head is a view of the first out_vocab embedding rows, so it cannot * be longer than the embedding. */ if ((flags & LLM_FLAG_TIED_HEAD) && m->out_vocab > m->c.vocab) return -2; #ifdef LLM_INT8_ACT /* quantize_act writes into a fixed LLM_Q8_MAX_INPUT buffer, sized by the * widest matvec input: dim, ffn or ple_dim. */ if (m->c.dim > LLM_Q8_MAX_INPUT || m->c.ffn > LLM_Q8_MAX_INPUT || m->c.ple_dim > LLM_Q8_MAX_INPUT) return -2; #endif m->head_matvec = NULL; m->layer_matvec = NULL; int D = m->c.dim, L = m->c.n_layers, P = m->c.ple_dim, F = m->c.ffn, V = m->c.vocab; p = bind_q(p, &m->tok_emb, V, D); p = bind_q(p, &m->ple_model_proj, L * P, D); p = bind_f(p, &m->ple_proj_norm, P); p = bind_q(p, &m->ple_table, V, L * P); for (int i = 0; i < L; i++) { p = bind_f(p, &m->attn_norm[i], D); p = bind_q(p, &m->qkv[i], 3 * D, D); p = bind_q(p, &m->attn_proj[i], D, D); p = bind_f(p, &m->ffn_norm[i], D); p = bind_q(p, &m->gate[i], F, D); p = bind_q(p, &m->up[i], F, D); p = bind_q(p, &m->down[i], D, F); p = bind_q(p, &m->ple_gate[i], P, D); p = bind_q(p, &m->ple_proj[i], D, P); p = bind_f(p, &m->ple_norm[i], D); } p = bind_f(p, &m->out_norm, D); /* Tied: the head is the FIRST out_vocab rows of the token embedding. The * embedding may store more rows than the model can ever emit (padding above * the tokenizer size), so tying does not imply equal row counts. * Untied: the head is appended as the final tensor. */ if (flags & LLM_FLAG_TIED_HEAD) { m->out_head = m->tok_emb; m->out_head.rows = m->out_vocab; } else { p = bind_q(p, &m->out_head, m->out_vocab, D); } m->image_bytes = (size_t)(p - base); return 0; } /* ---- int8 staging ------------------------------------------------------- * Costs 2x the bytes of the int4 form. The caller supplies the buffer, so * placement is the platform's choice. */ /* Byte offset of the float scale array in a staged buffer. rows*cols need not * be a multiple of 4, so the offset is rounded up to keep the float* aligned. */ static inline size_t llm_stage_scale_offset(const QT *t) { size_t w_bytes = (size_t)t->rows * t->cols * sizeof(int8_t); return (w_bytes + sizeof(float) - 1) & ~(size_t)(sizeof(float) - 1); } static inline size_t llm_stage_int8_bytes(const QT *t) { return llm_stage_scale_offset(t) + (size_t)t->rows * t->n_groups * sizeof(float); } static inline void llm_stage_int8(QT *t, void *buffer) { int8_t *w = (int8_t *)buffer; float *sc = (float *)((uint8_t *)buffer + llm_stage_scale_offset(t)); for (int r = 0; r < t->rows; r++) { const uint8_t *row = t->codes + (size_t)r * t->row_bytes; int8_t *dst = w + (size_t)r * t->cols; for (int j = 0; j < t->cols; j++) { uint8_t byte = row[j >> 1]; int code = (j & 1) ? (byte >> 4) : (byte & 0xF); dst[j] = (int8_t)(code - 8); } for (int gi = 0; gi < t->n_groups; gi++) sc[(size_t)r * t->n_groups + gi] = half2float(t->scales[(size_t)r * t->n_groups + gi]); } t->w8 = w; t->scale8 = sc; } /* Stage every per-position tensor, one allocation per tensor: a multi-megabyte * contiguous request fails on a fragmented heap even when total free is larger. * Returns the number staged; a tensor whose allocation fails keeps the int4 * path. Callers requiring a guaranteed placement compare this against * llm_core_stage_count(). */ static inline int llm_stage_core_int8_alloc(Model *m, void *(*alloc)(size_t)) { int staged = 0; QT *tensors[7]; void *buf = alloc(llm_stage_int8_bytes(&m->ple_model_proj)); if (buf) { llm_stage_int8(&m->ple_model_proj, buf); ++staged; } for (int l = 0; l < m->c.n_layers; l++) { tensors[0] = &m->qkv[l]; tensors[1] = &m->attn_proj[l]; tensors[2] = &m->gate[l]; tensors[3] = &m->up[l]; tensors[4] = &m->down[l]; tensors[5] = &m->ple_gate[l]; tensors[6] = &m->ple_proj[l]; for (int i = 0; i < 7; i++) { void *b = alloc(llm_stage_int8_bytes(tensors[i])); if (b) { llm_stage_int8(tensors[i], b); ++staged; } } } return staged; } /* Number of tensors llm_stage_core_int8_alloc attempts. */ static inline int llm_core_stage_count(const Model *m) { return 1 + 7 * m->c.n_layers; } // Scratch buffers, caller-allocated (host: malloc; device: PSRAM). typedef struct { float *x, *h, *qkv, *att, *g1, *g2, *ple, *tmpP, *trow, *logits; float *scores; // [seq_len], reused by each attention head float *kcache, *vcache; // [L * seq_len * D] #ifdef LLM_PROFILE struct { uint64_t input_us, attn_us, ffn_us, ple_us, head_us; uint32_t calls; } profile; #endif } Scratch; #ifdef LLM_PROFILE static void llm_profile_reset(Scratch *s) { memset(&s->profile, 0, sizeof(s->profile)); } #endif // One decode step: token at position pos -> logits[V]. KV cache persists across calls. /* Per-layer matvec dispatch: platform hook if set, otherwise MATVEC. */ #define LLM_LMV(m, t, x, y) \ do { if ((m)->layer_matvec) (m)->layer_matvec((t), (x), (y)); \ else MATVEC((t), (x), (y)); } while (0) static void llm_forward(Model *m, int token, int pos, Scratch *s) { int D = m->c.dim, L = m->c.n_layers, P = m->c.ple_dim, F = m->c.ffn; int H = m->c.n_heads, Dh = D / H, S = m->c.seq_len; #ifdef LLM_PROFILE uint64_t profile_t0 = (uint64_t)LLM_PROFILE_NOW(); #endif deq_row(&m->tok_emb, token, s->x); // embedding // ---- per-layer input: (RMSNorm(proj(x)/sqrt(D)) + table[tok]*sqrt(P)) / sqrt(2) LLM_LMV(m, &m->ple_model_proj, s->x, s->tmpP); // [L*P] float dscale = 1.f / sqrtf((float)D); for (int i = 0; i < L * P; i++) s->tmpP[i] *= dscale; for (int l = 0; l < L; l++) rmsnorm(s->tmpP + l * P, m->ple_proj_norm, P, s->tmpP + l * P); deq_row(&m->ple_table, token, s->trow); // [L*P] float sp = sqrtf((float)P), inv2 = 0.70710678f; for (int i = 0; i < L * P; i++) s->ple[i] = (s->tmpP[i] + s->trow[i] * sp) * inv2; // RoPE frequencies are identical across every head and layer at a position. // Reuse trow (dead after constructing s->ple) instead of recomputing the same // pow/cos/sin values L*H times. float *rope_c = s->trow, *rope_s = s->trow + Dh / 2; for (int i = 0; i < Dh / 2; i++) { float freq = powf(m->c.rope_theta, -2.f * i / Dh); rope_c[i] = cosf(pos * freq); rope_s[i] = sinf(pos * freq); } #ifdef LLM_PROFILE uint64_t profile_t1 = (uint64_t)LLM_PROFILE_NOW(); s->profile.input_us += profile_t1 - profile_t0; #endif for (int l = 0; l < L; l++) { // ---- attention rmsnorm(s->x, m->attn_norm[l], D, s->h); LLM_LMV(m, &m->qkv[l], s->h, s->qkv); // [3D] float *q = s->qkv, *k = s->qkv + D, *v = s->qkv + 2 * D; // split-half RoPE at position pos, per head for (int hh = 0; hh < H; hh++) { float *qh = q + hh * Dh, *kh = k + hh * Dh; for (int i = 0; i < Dh / 2; i++) { float c = rope_c[i], sn = rope_s[i]; float q1 = qh[i], q2 = qh[i + Dh / 2]; qh[i] = q1 * c - q2 * sn; qh[i + Dh / 2] = q2 * c + q1 * sn; float k1 = kh[i], k2 = kh[i + Dh / 2]; kh[i] = k1 * c - k2 * sn; kh[i + Dh / 2] = k2 * c + k1 * sn; } } float *kc = s->kcache + (size_t)l * S * D, *vc = s->vcache + (size_t)l * S * D; memcpy(kc + (size_t)pos * D, k, D * sizeof(float)); memcpy(vc + (size_t)pos * D, v, D * sizeof(float)); // causal attention over 0..pos float scale = 1.f / sqrtf((float)Dh); for (int hh = 0; hh < H; hh++) { float *qh = q + hh * Dh; float *ao = s->att + hh * Dh; for (int i = 0; i < Dh; i++) ao[i] = 0.f; float maxs = -1e30f; // pass 1: max for stable softmax for (int t = 0; t <= pos; t++) { float *kt = kc + (size_t)t * D + hh * Dh, dot = 0.f; for (int i = 0; i < Dh; i++) dot += qh[i] * kt[i]; dot *= scale; s->scores[t] = dot; if (dot > maxs) maxs = dot; } float denom = 0.f; for (int t = 0; t <= pos; t++) { float w = expf(s->scores[t] - maxs); denom += w; float *vt = vc + (size_t)t * D + hh * Dh; for (int i = 0; i < Dh; i++) ao[i] += w * vt[i]; } for (int i = 0; i < Dh; i++) ao[i] /= denom; } LLM_LMV(m, &m->attn_proj[l], s->att, s->h); for (int i = 0; i < D; i++) s->x[i] += s->h[i]; #ifdef LLM_PROFILE uint64_t profile_t2 = (uint64_t)LLM_PROFILE_NOW(); s->profile.attn_us += profile_t2 - profile_t1; #endif // ---- SwiGLU FFN rmsnorm(s->x, m->ffn_norm[l], D, s->h); LLM_LMV(m, &m->gate[l], s->h, s->g1); LLM_LMV(m, &m->up[l], s->h, s->g2); for (int i = 0; i < F; i++) s->g1[i] = silu(s->g1[i]) * s->g2[i]; LLM_LMV(m, &m->down[l], s->g1, s->h); for (int i = 0; i < D; i++) s->x[i] += s->h[i]; #ifdef LLM_PROFILE uint64_t profile_t3 = (uint64_t)LLM_PROFILE_NOW(); s->profile.ffn_us += profile_t3 - profile_t2; #endif // ---- PLE gate: x += RMSNorm(ple_proj(gelu(ple_gate(x)) * ple_l)) LLM_LMV(m, &m->ple_gate[l], s->x, s->g2); // [P] for (int i = 0; i < P; i++) s->g2[i] = gelu(s->g2[i]) * s->ple[l * P + i]; LLM_LMV(m, &m->ple_proj[l], s->g2, s->h); // [D] rmsnorm(s->h, m->ple_norm[l], D, s->h); for (int i = 0; i < D; i++) s->x[i] += s->h[i]; #ifdef LLM_PROFILE profile_t1 = (uint64_t)LLM_PROFILE_NOW(); s->profile.ple_us += profile_t1 - profile_t3; #endif } rmsnorm(s->x, m->out_norm, D, s->x); // output head: logits[v] = dot(out_head_row[v], x). out_head is tok_emb when // the model ties them, and a separate tensor when it does not. if (m->head_matvec) m->head_matvec(&m->out_head, s->x, s->logits); else MATVEC(&m->out_head, s->x, s->logits); #ifdef LLM_PROFILE s->profile.head_us += (uint64_t)LLM_PROFILE_NOW() - profile_t1; s->profile.calls++; #endif } #endif
main/gen_core.h
/* * gen_core.h — the platform-independent half of the PLE TinyLM port: asset * binding, scratch allocation, int8 staging, and the greedy generation loop. * * This ONE header is compiled into both the ESP32-S3 firmware (ple_main.c) * and the host oracle (host/ple_story.c). The token stream the two produce * must be byte-identical; sharing the loop makes that true by construction, * so anything that could diverge (allocators, matvec hooks, timing, console) * stays outside, injected by the caller. * * The inference engine itself is upstream's runtime/llm.h (slvDev/esp32-ai, * MIT, (c) 2026 Viacheslav Sierbov), vendored unmodified; this file mirrors * the setup sequence of upstream's firmware/esp32_tinystories/esp32_tinystories.ino * (allocation placement, norm copies, staging order) minus the display and * the non-blocking serial writes (determinism requires emitting every byte). * * Decoding runs in TWO modes, split by what each must prove: * * - `sampled == 0` (the BOOT story, and everything the host oracle runs): * temperature-0 argmax, exactly as upstream. This is the cross-platform * model-faithfulness probe: the two platforms' logits differ by a few * ulps (libm inside the vendored inference), and argmax is the decision * most robust to that — the byte-identical oracle diff rides on it. * - `sampled == 1` (interactive prompts on the device): Gumbel-max top-40 * at temperature 0.8 — argmax(l/T + Gumbel), distributionally identical * to softmax sampling — plus a 1.2x repetition penalty over a 64-token * window, so stories stop looping. Both were MEASURED before shipping * (20 stories per setting, host-native): repeated 4-word phrases fell * 3.13% -> 0.40%, and reading quality held or improved; the grammar * damage a penalty can cause only appeared at 1.5x ("Tom looked at each * other"), far above 1.2. On-device cost is 1.1% of generation time. The xorshift32 seeds * from the prompt ids plus a session story counter: the stream is still * a pure function of the session's typed inputs, so replay and the * EMULATOR goldens pin it bit-exactly. It is NOT host-comparable — at * ~460 sampled tokens per session the platform ulp noise flips a * near-tie more often than not (measured), which is exactly why the * oracle only ever diffs greedy regions. The Gumbel math uses a local * log2 polynomial (plain IEEE f32, -ffp-contract=off) so the rng * perturbations themselves are platform-exact. */ #ifndef GEN_CORE_H #define GEN_CORE_H #include "llm.h" #include "bpe_tokenizer.h" #define PLE_N_GENERATE 200 /* upstream N_GENERATE; seq_len (256) still caps */ #define PLE_MAX_PROMPT_IDS 128 #define PLE_TOP_K 40 /* 1 / temperature (T = 0.8): the Gumbel-max score is l/T + G. */ #define PLE_INV_T 1.25f /* VOC1 decode table (built by tools/gen_assets.py): id -> raw UTF-8 bytes. */ typedef struct { uint32_t n; /* token count */ const uint32_t *off; /* n+1 byte offsets into blob */ const uint8_t *blob; } PleVocab; typedef struct { Model m; Scratch s; PleVocab vocab; BpeTokenizer btk; /* Every emitted story byte goes through this sink (UART0 on device, stdout * on the host). Must accept every byte: dropping on backpressure, as the * upstream sketch does for a detached USB host, would break determinism. */ void (*emit_bytes)(const void *bytes, int len); /* Sampler state: stories completed this session (the seed mixer), the * current story's xorshift32 state, and the decode mode (0 = greedy — * the boot/oracle mode; 1 = Gumbel-max sampling for interactive * prompts). Zero-initialized on both platforms. */ uint32_t stories_run; uint32_t rng; int sampled; } PleRuntime; /* xorshift32 (Marsaglia): tiny, deterministic, never sticks at 0. */ static uint32_t ple_rng_next(uint32_t *state) { uint32_t x = *state; x ^= x << 13; x ^= x >> 17; x ^= x << 5; *state = x ? x : 0x9e3779b9u; return *state; } /* log2(x) for finite positive normal x, plain IEEE f32 ops only (no libm * — see header): split off the exponent through the bits, then a degree-5 * polynomial for log2(m) on m in [1, 2). ~1e-5 accuracy, which only * shades sampling probabilities imperceptibly — and, crucially, the SAME * bits on every platform. */ static float ple_log2f(float x) { union { float f; uint32_t u; } bx; bx.f = x; int e = (int)((bx.u >> 23) & 0xff) - 127; bx.u = (bx.u & 0x007fffffu) | 0x3f800000u; /* mantissa in [1, 2) */ float m = bx.f; /* minimax-ish fit of log2(m) over [1, 2), Horner */ float t = m - 1.0f; float p = -3.5780402e-2f; p = p * t + 1.7059870e-1f; p = p * t - 3.5556380e-1f; p = p * t + 4.8089329e-1f; p = p * t - 7.2055387e-1f; p = p * t + 1.4426951e0f; return (float)e + p * t; } #define PLE_LN2 0.69314718f /* Draw one token: top-PLE_TOP_K selection over the logits, then a * GUMBEL-MAX draw among those — argmax(l/T + Gumbel) is distributionally * identical to temperature softmax sampling, but the decision is an * ARGMAX over values with O(1) typical gaps, not a walk along a * cumulative boundary. That matters because the two platforms' logits * differ by a few ulps (libm inside the vendored inference; the greedy * era absorbed that silently, 400 pinned tokens without a flip): a * cumsum draw crosses a boundary somewhere every few hundred tokens, * while an argmax only flips when the top two scores land within ulps of * each other. The oracle diff still pins the exact stories, so a * once-in-many-repins near-tie would fail loudly there, never silently * (remedy: bump the seed mixer and re-pin). */ /* Repetition penalty over a 64-token window (sampled mode only). The flag * table is byte-per-token over the output vocab (25,353) — 32 KB of the * ~346 KB free internal SRAM, so the lookup stays O(1) on the hot path. */ #define PLE_REP_PENALTY 1.2f #define PLE_REP_WINDOW 64 static uint8_t ple_recent_flag[32768]; static int ple_recent_ring[PLE_REP_WINDOW]; static int ple_recent_n = 0, ple_recent_head = 0; static void ple_recent_reset(void) { for (int i = 0; i < ple_recent_n; i++) ple_recent_flag[ple_recent_ring[i]] = 0; ple_recent_n = 0; ple_recent_head = 0; } static void ple_recent_push(int tok) { if (tok < 0 || tok >= (int)sizeof ple_recent_flag) return; if (ple_recent_n == PLE_REP_WINDOW) { ple_recent_flag[ple_recent_ring[ple_recent_head]] = 0; ple_recent_ring[ple_recent_head] = tok; ple_recent_head = (ple_recent_head + 1) % PLE_REP_WINDOW; } else { ple_recent_ring[ple_recent_n++] = tok; } ple_recent_flag[tok] = 1; } static int ple_sample(PleRuntime *rt, const float *logits, int v) { int idx[PLE_TOP_K] = {0}; float val[PLE_TOP_K] = {0}; int k = 0; for (int i = 0; i < v; i++) { float l = logits[i]; if (ple_recent_flag[i]) l = l > 0.0f ? l / PLE_REP_PENALTY : l * PLE_REP_PENALTY; if (k == PLE_TOP_K && l <= val[PLE_TOP_K - 1]) continue; int j = k < PLE_TOP_K ? k : PLE_TOP_K - 1; while (j > 0 && val[j - 1] < l) { val[j] = val[j - 1]; idx[j] = idx[j - 1]; j--; } val[j] = l; idx[j] = i; if (k < PLE_TOP_K) k++; } int chosen = idx[0]; float best = -1e30f; for (int j = 0; j < k; j++) { /* u in (0, 1]: 24-bit draw + 1 ulp floor keeps log finite. */ float u = ((float)(ple_rng_next(&rt->rng) >> 8) + 1.0f) * (1.0f / 16777216.0f); /* Standard Gumbel: -ln(-ln(u)), via the local log2. */ float g = -(PLE_LN2 * ple_log2f(-(PLE_LN2 * ple_log2f(u)))); float score = val[j] * PLE_INV_T + g; if (score > best) { best = score; chosen = idx[j]; } } return chosen; } /* Parse the VOC1 blob. Returns NULL on success, else a static error string. */ static const char *ple_bind_vocab(PleVocab *v, const uint8_t *base, size_t size) { if (size < 12 || memcmp(base, "VOC1", 4) != 0) return "vocab.bin: bad magic"; uint32_t n, blob_len; memcpy(&n, base + 4, 4); memcpy(&blob_len, base + 8, 4); if (n == 0 || n > 0x10000) return "vocab.bin: bad count"; if (size != 12 + (size_t)(n + 1) * 4 + blob_len) return "vocab.bin: bad size"; const uint32_t *off = (const uint32_t *)(base + 12); if (off[0] != 0 || off[n] != blob_len) return "vocab.bin: bad offsets"; for (uint32_t i = 0; i < n; i++) if (off[i] > off[i + 1]) return "vocab.bin: offsets not monotonic"; v->n = n; v->off = off; v->blob = base + 12 + (size_t)(n + 1) * 4; return NULL; } /* llm_load + vocab + encoder, with the cross-checks the .ino does at boot. */ static const char *ple_bind_assets(PleRuntime *rt, const uint8_t *model_base, size_t model_size, const uint8_t *vocab_base, size_t vocab_size, const uint8_t *btk_base, size_t btk_size) { if (((uintptr_t)model_base & 15) != 0) return "model blob not 16-byte aligned"; int rc = llm_load(model_base, &rt->m); if (rc != 0) return "model header rejected"; if (rt->m.image_bytes != model_size) return "model image size mismatch"; const char *err = ple_bind_vocab(&rt->vocab, vocab_base, vocab_size); if (err) return err; /* The model header states how many logits it produces; the decode table * must agree or every emitted token decodes against the wrong entry. */ if ((int)rt->vocab.n != rt->m.out_vocab) return "vocab/model size mismatch"; if (bpe_tokenizer_load(btk_base, btk_size, &rt->btk) != 0) return "tokenizer.btk rejected"; if (rt->btk.active_vocab != rt->vocab.n) return "encoder/decoder size mismatch"; return NULL; } /* Scratch allocation, mirroring the .ino's placement: hot per-token buffers * from `fast` (internal SRAM on device), the 99 KiB logits array and the KV * cache from `big` (PSRAM on device). The host passes malloc for both. * Returns NULL on success, else the name of the allocation that failed. */ static const char *ple_alloc_scratch(PleRuntime *rt, void *(*fast)(size_t), void *(*big)(size_t)) { Cfg *c = &rt->m.c; int D = c->dim, L = c->n_layers, P = c->ple_dim, F = c->ffn, S = c->seq_len; Scratch *s = &rt->s; struct { float **p; size_t n; const char *what; void *(*a)(size_t); } plan[] = { { &s->x, (size_t)D, "x", fast }, { &s->h, (size_t)(F > D ? F : D), "h", fast }, { &s->qkv, (size_t)3 * D, "qkv", fast }, { &s->att, (size_t)D, "att", fast }, { &s->g1, (size_t)F, "g1", fast }, { &s->g2, (size_t)(P > F ? P : F), "g2", fast }, { &s->ple, (size_t)L * P, "ple", fast }, { &s->tmpP, (size_t)L * P, "tmpP", fast }, { &s->trow, (size_t)L * P, "trow", fast }, { &s->scores, (size_t)S, "scores", fast }, { &s->logits, (size_t)rt->m.out_vocab, "logits", big }, { &s->kcache, (size_t)L * S * D, "kcache", big }, { &s->vcache, (size_t)L * S * D, "vcache", big }, }; for (size_t i = 0; i < sizeof(plan) / sizeof(plan[0]); i++) { void *p = plan[i].a(plan[i].n * sizeof(float)); if (!p) return plan[i].what; *plan[i].p = (float *)p; } return NULL; } /* Stage the per-position core + the tied head to int8, in the .ino's order. * Returns the number staged; the caller checks it equals ple_stage_want(). */ static int ple_stage(PleRuntime *rt, void *(*alloc)(size_t)) { int staged = llm_stage_core_int8_alloc(&rt->m, alloc); void *b = alloc(llm_stage_int8_bytes(&rt->m.out_head)); if (b) { llm_stage_int8(&rt->m.out_head, b); staged++; } return staged; } static int ple_stage_want(const PleRuntime *rt) { return llm_core_stage_count(&rt->m) + 1; /* + the tied head */ } /* FNV-1a over the model image — the fingerprint upstream's deploy.sh prints * and the firmware echoes at boot; goldens pin it. */ static uint32_t ple_fingerprint(const PleRuntime *rt, const uint8_t *model_base) { uint32_t fp = 2166136261u; for (size_t i = 0; i < rt->m.image_bytes; i++) { fp ^= model_base[i]; fp *= 16777619u; } return fp; } static void ple_emit_token(PleRuntime *rt, int tok) { if (tok < 0 || (uint32_t)tok >= rt->vocab.n) return; const uint8_t *bytes = rt->vocab.blob + rt->vocab.off[tok]; int len = (int)(rt->vocab.off[tok + 1] - rt->vocab.off[tok]); if (len > 0) rt->emit_bytes(bytes, len); } /* Encode an ASCII prompt line. Returns id count, or a BTK_ERR_* (< 0). */ static int ple_encode(PleRuntime *rt, const char *text, int *ids, int cap) { uint16_t raw[PLE_MAX_PROMPT_IDS]; int n = bpe_encode_ascii(&rt->btk, text, raw, cap < PLE_MAX_PROMPT_IDS ? cap : PLE_MAX_PROMPT_IDS); if (n < 0) return n; for (int i = 0; i < n; i++) ids[i] = raw[i]; return n; } /* The generation loop, upstream's setup()/loop() order: emit each prompt * token as it primes, then sample up to PLE_N_GENERATE more (top-k, * temperature — see the header), capped by the trained seq_len. The story * seed derives from the prompt ids and the session story counter HERE, so * the firmware and the host oracle agree without either main doing * anything. Emits story bytes only — no timing, no status lines — so the * device and the host print an identical region. Returns the number of * GENERATED (post-prompt) tokens. */ static int ple_generate(PleRuntime *rt, const int *prompt_ids, int n_prompt) { Model *m = &rt->m; Scratch *s = &rt->s; uint32_t seed = 2166136261u; for (int i = 0; i < n_prompt; i++) { seed ^= (uint32_t)prompt_ids[i]; seed *= 16777619u; } seed ^= rt->stories_run++ * 0x9e3779b9u; rt->rng = seed ? seed : 0x9e3779b9u; ple_recent_reset(); int pos = 0, tok = 0; for (int i = 0; i < n_prompt; i++) { tok = prompt_ids[i]; ple_emit_token(rt, tok); llm_forward(m, tok, pos++, s); } int generated = 0; for (int step = 0; step < PLE_N_GENERATE && pos < m->c.seq_len; step++) { if (rt->sampled) { tok = ple_sample(rt, s->logits, m->out_vocab); ple_recent_push(tok); } else { int best = 0; float bv = -1e30f; for (int v = 0; v < m->out_vocab; v++) if (s->logits[v] > bv) { bv = s->logits[v]; best = v; } tok = best; } ple_emit_token(rt, tok); llm_forward(m, tok, pos++, s); generated++; } return generated; } #endif
main/bpe_tokenizer.h
// Portable compact ByteLevel-BPE encoder. The same code runs on the host and on // the ESP32; it reads the BTK1 asset that // firmware/esp32_barista/tools/generate_tokenizer_header.py packs. // // ASCII input only. Bytes at or above 0x80 are rejected rather than encoded, // because tokenizing them differently from Hugging Face would be silent: the // model would simply receive ids it was never trained on. Unicode support is a // later explicit extension, not an accident of permissiveness. // // BTK1 layout, all little endian: // // offset size field // 0 4 magic "BTK1" // 4 4 format version, must be 2 // 8 4 active vocabulary size // 12 4 merge count // 16 4 reserved, ignored // 20 4 merge base, the token id of merge rank 0 // 24 28 truncated sha256 of the source tokenizer, provenance only // 52 512 256 uint16 byte token ids, indexed by raw byte // 564 6*n merge entries: uint32 key (left << 16 | right), uint16 rank, // sorted ascending by key // // Offset 16 is reserved and carries zero. It is not an end-of-sequence id: the // end of a Barista answer is an output class in the word tables, not a token id // here, so this encoder does not expose it. // // The merge table must be sorted, because btk_merge_rank binary searches it. // The generator guarantees that; this loader does not re-verify it, since the // check belongs to the conformance gate rather than to every boot. It does // verify that the table is present in full, which is a bounds question rather // than a content one. #ifndef BPE_TOKENIZER_H #define BPE_TOKENIZER_H #include <stdint.h> #include <stddef.h> #include <string.h> #define BTK_FORMAT_VERSION 2 #define BTK_HEADER_BYTES 52 #define BTK_BYTE_TABLE_BYTES 512 #define BTK_MERGE_ENTRY_BYTES 6 #define BTK_MAX_INPUT_BYTES 512 /* Token ids and merge ranks are uint16 in the asset, so neither count can * exceed this. A corrupt header would otherwise send the binary search walking * past the end of the image. */ #define BTK_MAX_TABLE_ENTRIES 0x10000 /* bpe_encode_ascii failures. Negative so a caller can test for < 0 in bulk. */ #define BTK_ERR_TOO_LONG (-2) #define BTK_ERR_NOT_ASCII (-3) #define BTK_ERR_CAPACITY (-4) typedef struct { const uint8_t *asset; uint32_t active_vocab; uint32_t merge_count; /* Token id of merge rank 0, so rank r yields merge_base + r. Stored rather * than assumed, because it follows from where the vocabulary happens to place * its 256 single-byte symbols. */ uint32_t merge_base; const uint8_t *byte_ids; const uint8_t *merges; } BpeTokenizer; static uint16_t btk_u16(const uint8_t *p) { return (uint16_t)p[0] | ((uint16_t)p[1] << 8); } static uint32_t btk_u32(const uint8_t *p) { return (uint32_t)p[0] | ((uint32_t)p[1] << 8) | ((uint32_t)p[2] << 16) | ((uint32_t)p[3] << 24); } // Returns 0 on success, -1 on an asset this encoder cannot read. // // asset_size is required, not advisory: the merge count is a field of the asset // itself, so without a length there is nothing to check it against and a // truncated table sends btk_merge_rank reading past the end of the image. // The generated header supplies TOKENIZER_ENCODER_ASSET_SIZE for this argument. // // Nothing is written to tok until every check has passed, so a rejected asset // cannot leave a half-populated tokenizer behind. static int bpe_tokenizer_load( const uint8_t *asset, size_t asset_size, BpeTokenizer *tok) { if (asset == NULL || tok == NULL) return -1; /* Enough bytes to read the header and byte table before trusting any field. */ if (asset_size < BTK_HEADER_BYTES + BTK_BYTE_TABLE_BYTES) return -1; if (memcmp(asset, "BTK1", 4) != 0) return -1; if (btk_u32(asset + 4) != BTK_FORMAT_VERSION) return -1; uint32_t active_vocab = btk_u32(asset + 8); uint32_t merge_count = btk_u32(asset + 12); /* asset + 16 is reserved and deliberately not read. */ uint32_t merge_base = btk_u32(asset + 20); /* The 256 single-byte symbols are always present, so the vocabulary cannot be * smaller than that. The upper bound is what a uint16 token id can address. */ if (active_vocab < 256 || active_vocab > BTK_MAX_TABLE_ENTRIES) return -1; if (merge_count == 0 || merge_count > BTK_MAX_TABLE_ENTRIES) return -1; /* Rank 0 must name a real token, and rank merge_count-1 must too. Both bounds * matter: btk_encode_piece emits merge_base + rank as a uint16, so an id at or * past active_vocab is either outside the vocabulary or a wrapped value. * active_vocab is already capped above, so this also keeps the largest emitted * id, merge_base + merge_count - 1, inside uint16. */ if (merge_base == 0 || merge_base >= active_vocab) return -1; if (merge_base + merge_count > active_vocab) return -1; /* Only now is merge_count trustworthy enough to size the table with. */ size_t required = (size_t)BTK_HEADER_BYTES + BTK_BYTE_TABLE_BYTES + (size_t)merge_count * BTK_MERGE_ENTRY_BYTES; if (asset_size < required) return -1; tok->asset = asset; tok->active_vocab = active_vocab; tok->merge_count = merge_count; tok->merge_base = merge_base; tok->byte_ids = asset + BTK_HEADER_BYTES; tok->merges = tok->byte_ids + BTK_BYTE_TABLE_BYTES; return 0; } static uint16_t btk_byte_id(const BpeTokenizer *tok, uint8_t byte) { return btk_u16(tok->byte_ids + (size_t)byte * 2); } // Rank of the pair, or -1 if it is not a merge. Binary search: the table is // sorted by key, and at 7000-odd entries a linear scan per adjacent pair per // round would dominate encoding. static int btk_merge_rank( const BpeTokenizer *tok, uint16_t left, uint16_t right) { uint32_t key = ((uint32_t)left << 16) | right; uint32_t lo = 0, hi = tok->merge_count; while (lo < hi) { uint32_t mid = lo + (hi - lo) / 2; const uint8_t *entry = tok->merges + (size_t)mid * BTK_MERGE_ENTRY_BYTES; uint32_t candidate = btk_u32(entry); if (candidate < key) lo = mid + 1; else hi = mid; } if (lo == tok->merge_count) return -1; const uint8_t *entry = tok->merges + (size_t)lo * BTK_MERGE_ENTRY_BYTES; return btk_u32(entry) == key ? (int)btk_u16(entry + 4) : -1; } // Ranked BPE over one pre-token: repeatedly apply the lowest-ranked adjacent // merge until none applies. static int btk_encode_piece( const BpeTokenizer *tok, const uint8_t *piece, int piece_len, uint16_t *out, int out_cap) { if (piece_len > BTK_MAX_INPUT_BYTES || piece_len > out_cap) return BTK_ERR_TOO_LONG; int n = piece_len; for (int i = 0; i < n; i++) out[i] = btk_byte_id(tok, piece[i]); for (;;) { int best_rank = -1; uint16_t best_left = 0, best_right = 0; for (int i = 0; i + 1 < n; i++) { int rank = btk_merge_rank(tok, out[i], out[i + 1]); if (rank >= 0 && (best_rank < 0 || rank < best_rank)) { best_rank = rank; best_left = out[i]; best_right = out[i + 1]; } } if (best_rank < 0) break; int write = 0; for (int read = 0; read < n;) { if (read + 1 < n && out[read] == best_left && out[read + 1] == best_right) { out[write++] = (uint16_t)(tok->merge_base + best_rank); read += 2; } else { out[write++] = out[read++]; } } n = write; } return n; } static int btk_ascii_letter(uint8_t c) { return (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z'); } static int btk_ascii_digit(uint8_t c) { return c >= '0' && c <= '9'; } static int btk_ascii_space(uint8_t c) { return c == ' ' || c == '\t' || c == '\n' || c == '\r' || c == '\f' || c == '\v'; } static int btk_contraction_len(const uint8_t *s, int n) { static const char *suffixes[] = {"'s", "'t", "'re", "'ve", "'m", "'ll", "'d"}; if (n < 2 || s[0] != '\'') return 0; for (unsigned i = 0; i < sizeof(suffixes) / sizeof(suffixes[0]); i++) { int len = (int)strlen(suffixes[i]); if (len <= n && memcmp(s, suffixes[i], (size_t)len) == 0) return len; } return 0; } // Mirrors the Hugging Face ByteLevel pre-tokenizer over the supported ASCII // domain, then applies ranked BPE independently to each pre-token. Returns the // number of ids written, or one of the BTK_ERR_* codes. static int bpe_encode_ascii( const BpeTokenizer *tok, const char *text, uint16_t *out, int out_cap) { int len = (int)strlen(text); if (len > BTK_MAX_INPUT_BYTES) return BTK_ERR_TOO_LONG; for (int i = 0; i < len; i++) if ((uint8_t)text[i] >= 0x80) return BTK_ERR_NOT_ASCII; uint16_t piece_ids[BTK_MAX_INPUT_BYTES]; int pos = 0, written = 0; while (pos < len) { const uint8_t *s = (const uint8_t *)text; int start = pos, contraction = btk_contraction_len(s + pos, len - pos); if (contraction) { pos += contraction; } else if ( s[pos] == ' ' && pos + 1 < len && btk_ascii_letter(s[pos + 1])) { pos += 2; while (pos < len && btk_ascii_letter(s[pos])) pos++; } else if (btk_ascii_letter(s[pos])) { while (pos < len && btk_ascii_letter(s[pos])) pos++; } else if ( s[pos] == ' ' && pos + 1 < len && btk_ascii_digit(s[pos + 1])) { pos += 2; while (pos < len && btk_ascii_digit(s[pos])) pos++; } else if (btk_ascii_digit(s[pos])) { while (pos < len && btk_ascii_digit(s[pos])) pos++; } else if ( s[pos] == ' ' && pos + 1 < len && !btk_ascii_space(s[pos + 1]) && !btk_ascii_letter(s[pos + 1]) && !btk_ascii_digit(s[pos + 1])) { pos += 2; while (pos < len && !btk_ascii_space(s[pos]) && !btk_ascii_letter(s[pos]) && !btk_ascii_digit(s[pos])) pos++; } else if (!btk_ascii_space(s[pos]) && !btk_ascii_letter(s[pos]) && !btk_ascii_digit(s[pos])) { while (pos < len && !btk_ascii_space(s[pos]) && !btk_ascii_letter(s[pos]) && !btk_ascii_digit(s[pos])) pos++; } else { while (pos < len && btk_ascii_space(s[pos])) pos++; // Before non-whitespace, the regex leaves the last whitespace byte to be // consumed as the optional prefix of the next token class. if (pos < len && pos - start > 1) pos--; } int n = btk_encode_piece(tok, s + start, pos - start, piece_ids, BTK_MAX_INPUT_BYTES); if (n < 0 || written + n > out_cap) return BTK_ERR_CAPACITY; memcpy(out + written, piece_ids, (size_t)n * sizeof(uint16_t)); written += n; } return written; } #endif
main/tft.h
/* * Minimal ST7789 2.8" 240x320 text console for the PLE storyteller — * first-party (MIT), written for the Breadboard bench wiring: * * din=GPIO7 (GPSPI2 MOSI) clk=GPIO6 (GPSPI2 SCK) [the S3 SPI seam] * cs=GPIO5 dc=GPIO3 rst=GPIO4 (the bench's TFT wiring convention, * as the microSD photo frame example wires it; CS driven manually) * * SPI goes through the IDF spi_master driver with DMA and ROW-SIZED * transactions (one 240x16 text row = one 7,680-byte transfer): the * emulator's SPI seam completes one parked transaction per host service, * so many tiny FIFO chunks would be latency-bound where a few large * transfers stream at wire speed — the same shape MicroPython's * machine.SPI big-buffer writes take, which the GPSPI2 model is validated * against. The emulator charges honest wire time (BRE-91): every pixel * painted here costs the sim real bus nanoseconds, so the on-screen tok/s * is the true effective rate WITH the display attached, not a free * overlay. * * Layout (portrait 240x320, 5x7 font at 2x = 10x16 cells, 24 cols): * rows 0-1 prompt (yellow on navy) * rows 2-18 story text, word-wrapped, scrolls (white) * row 19 live stats: tok/s + token count (cyan) * * The story text arrives through the same emit seam as the console; this * file only ever OBSERVES those bytes — the UART stream stays byte-exact * (the host-oracle contract). */ #pragma once #include <string.h> #include "driver/gpio.h" #include "esp_heap_caps.h" #include "driver/spi_master.h" #include "esp_timer.h" #include "font5x7.h" #include "freertos/FreeRTOS.h" #include "freertos/task.h" #define TFT_PIN_MOSI 7 #define TFT_PIN_SCK 6 #define TFT_PIN_CS 5 #define TFT_PIN_DC 3 #define TFT_PIN_RST 4 #define TFT_HZ 20000000 #define TFT_W 240 #define TFT_H 320 #define CELL_W 10 /* 5x7 font at 2x + no gap (glyph carries its own) */ #define CELL_H 16 #define TFT_COLS (TFT_W / CELL_W) /* 24 */ #define TFT_ROWS (TFT_H / CELL_H) /* 20 */ #define BODY_TOP 2 /* rows 0-1 = prompt */ #define BODY_ROWS (TFT_ROWS - BODY_TOP - 1) /* 17; last row = stats */ #define RGB(r, g, b) ((uint16_t)((((r) & 0xf8) << 8) | (((g) & 0xfc) << 3) | ((b) >> 3))) #define C_BG RGB(10, 14, 26) /* deep navy */ #define C_PROMPT RGB(255, 214, 64) #define C_STORY RGB(240, 240, 235) #define C_STATS RGB(64, 220, 255) #define C_TITLE RGB(255, 122, 26) /* the bench's industrial orange */ static spi_device_handle_t tft_spi; /* One text row of RGB565 pixels, DMA-capable (internal RAM). */ #define ROW_BYTES (TFT_W * CELL_H * 2) static uint8_t *tft_row; static void tft_cs(int level) { gpio_set_level(TFT_PIN_CS, level); } static void tft_dc(int level) { gpio_set_level(TFT_PIN_DC, level); } /* One transfer (DMA path; up to ROW_BYTES). */ static void tft_xfer(const uint8_t *bytes, int len) { spi_transaction_t t = {0}; t.length = (size_t)len * 8; t.tx_buffer = bytes; spi_device_polling_transmit(tft_spi, &t); } static void tft_cmd(uint8_t cmd, const uint8_t *data, int len) { tft_cs(0); tft_dc(0); tft_xfer(&cmd, 1); if (len > 0) { tft_dc(1); tft_xfer(data, len); } tft_cs(1); } /* Open an address window and leave CS low with DC=data for a RAMWR stream. */ static void tft_window(int x, int y, int w, int h) { uint8_t ca[4] = {(uint8_t)(x >> 8), (uint8_t)x, (uint8_t)((x + w - 1) >> 8), (uint8_t)(x + w - 1)}; uint8_t ra[4] = {(uint8_t)(y >> 8), (uint8_t)y, (uint8_t)((y + h - 1) >> 8), (uint8_t)(y + h - 1)}; tft_cmd(0x2a, ca, 4); /* CASET */ tft_cmd(0x2b, ra, 4); /* RASET */ tft_cs(0); tft_dc(0); uint8_t ramwr = 0x2c; tft_xfer(&ramwr, 1); tft_dc(1); } /* Render one full text row (24 cells) into tft_row and send it as ONE * transaction. All drawing goes through this: fewest, largest transfers. */ static void tft_draw_row(int row, const char *text, uint16_t fg) { for (int y = 0; y < CELL_H; y++) { int fy = (y - 1) / 2; /* 1px top leading, 7 font rows at 2x, 1px bottom */ uint8_t *line = tft_row + (size_t)y * TFT_W * 2; for (int col = 0; col < TFT_COLS; col++) { char ch = text ? text[col] : ' '; if (ch == '\0') text = NULL, ch = ' '; if (ch < FONT_FIRST || ch > FONT_LAST) ch = '?'; const uint8_t *g = FONT5X7[ch - FONT_FIRST]; for (int x = 0; x < CELL_W; x++) { int fx = x / 2; int on = y >= 1 && y <= 14 && fx < FONT_W && (g[fx] >> fy) & 1; uint16_t c = on ? fg : C_BG; int o = (col * CELL_W + x) * 2; line[o] = (uint8_t)(c >> 8); line[o + 1] = (uint8_t)c; } } } tft_window(0, row * CELL_H, TFT_W, CELL_H); tft_xfer(tft_row, ROW_BYTES); tft_cs(1); } static void tft_text(int col, int row, const char *s, uint16_t fg) { char line[TFT_COLS + 1]; memset(line, ' ', sizeof line - 1); line[TFT_COLS] = '\0'; int n = (int)strlen(s); if (n > TFT_COLS - col) n = TFT_COLS - col; if (n > 0) memcpy(line + col, s, (size_t)n); tft_draw_row(row, line, fg); } /* ---- story console state ---- */ static char body[BODY_ROWS][TFT_COLS + 1]; static int body_row, body_col; static char word[TFT_COLS + 1]; static int word_len; static int64_t rate_t0; static int rate_tokens; static char stats_prev[TFT_COLS + 1]; static void tft_redraw_body(void) { for (int r = 0; r < BODY_ROWS; r++) tft_text(0, BODY_TOP + r, body[r], C_STORY); } static void tft_newline(void) { if (body_row < BODY_ROWS - 1) { body_row++; } else { memmove(body[0], body[1], sizeof(body) - sizeof(body[0])); body[BODY_ROWS - 1][0] = '\0'; body_col = 0; tft_redraw_body(); body_col = 0; body_row = BODY_ROWS - 1; return; } body_col = 0; } static void tft_put_word(void) { if (word_len == 0) return; if (body_col + word_len > TFT_COLS) tft_newline(); for (int i = 0; i < word_len && body_col < TFT_COLS; i++) { body[body_row][body_col] = word[i]; body[body_row][body_col + 1] = '\0'; body_col++; } word_len = 0; tft_text(0, BODY_TOP + body_row, body[body_row], C_STORY); } static void tft_stats_update(int force) { int64_t now = esp_timer_get_time(); if (!force && now - rate_t0 < 500000) return; char line[48]; /* roomy scratch; hard-cut to the 24-cell row below */ if (rate_tokens > 0 && now > rate_t0) { double rate = rate_tokens * 1e6 / (double)(now - rate_t0); snprintf(line, sizeof line, " %d tok @ %.1f tok/s", rate_tokens, rate); } else { snprintf(line, sizeof line, " thinking..."); } line[TFT_COLS] = '\0'; int n = (int)strlen(line); memset(line + n, ' ', (size_t)(TFT_COLS - n)); line[TFT_COLS] = '\0'; if (strcmp(line, stats_prev) != 0) { strcpy(stats_prev, line); tft_text(0, TFT_ROWS - 1, line, C_STATS); } } /* Observe one emit_bytes call (= one generated token's text). */ static void tft_story_bytes(const void *bytes, int len) { const char *p = (const char *)bytes; rate_tokens++; for (int i = 0; i < len; i++) { char c = p[i]; if (c == '\n') { tft_put_word(); tft_newline(); } else if (c == ' ') { tft_put_word(); if (body_col > 0) { if (body_col >= TFT_COLS) { tft_newline(); } else { body[body_row][body_col] = ' '; body[body_row][body_col + 1] = '\0'; body_col++; } } } else if (c >= 0x20 && c <= 0x7e) { if (word_len < TFT_COLS) word[word_len++] = c; if (word_len == TFT_COLS) tft_put_word(); /* unbreakable overlong run */ } } tft_stats_update(0); } /* New story: show the prompt, clear the body, restart the rate clock. */ static void tft_story_begin(const char *prompt) { tft_put_word(); memset(body, 0, sizeof body); body_row = 0; body_col = 0; word_len = 0; char line[TFT_COLS + 1]; snprintf(line, sizeof line, "> %.*s", TFT_COLS - 2, prompt); tft_text(0, 0, line, C_PROMPT); if (strlen(prompt) > (size_t)(TFT_COLS - 2)) { snprintf(line, sizeof line, " %.*s", TFT_COLS - 2, prompt + (TFT_COLS - 2)); tft_text(0, 1, line, C_PROMPT); } else { tft_draw_row(1, NULL, C_PROMPT); } tft_redraw_body(); rate_t0 = esp_timer_get_time(); rate_tokens = 0; stats_prev[0] = '\0'; tft_stats_update(1); } static void tft_story_end(void) { tft_put_word(); tft_stats_update(1); } static void tft_init(void) { gpio_config_t io = { .pin_bit_mask = (1ULL << TFT_PIN_CS) | (1ULL << TFT_PIN_DC) | (1ULL << TFT_PIN_RST), .mode = GPIO_MODE_OUTPUT, }; gpio_config(&io); tft_cs(1); tft_dc(1); tft_row = heap_caps_malloc(ROW_BYTES, MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL); if (!tft_row) return; /* no display worth crashing the model over */ spi_bus_config_t bus = { .mosi_io_num = TFT_PIN_MOSI, .miso_io_num = -1, .sclk_io_num = TFT_PIN_SCK, .quadwp_io_num = -1, .quadhd_io_num = -1, .max_transfer_sz = ROW_BYTES, }; spi_bus_initialize(SPI2_HOST, &bus, SPI_DMA_CH_AUTO); spi_device_interface_config_t dev = { .clock_speed_hz = TFT_HZ, .mode = 0, .spics_io_num = -1, /* CS is driven manually, like the seam contract */ .queue_size = 1, }; spi_bus_add_device(SPI2_HOST, &dev, &tft_spi); gpio_set_level(TFT_PIN_RST, 0); vTaskDelay(pdMS_TO_TICKS(10)); gpio_set_level(TFT_PIN_RST, 1); vTaskDelay(pdMS_TO_TICKS(120)); tft_cmd(0x01, NULL, 0); /* SWRESET */ vTaskDelay(pdMS_TO_TICKS(120)); tft_cmd(0x11, NULL, 0); /* SLPOUT */ vTaskDelay(pdMS_TO_TICKS(10)); uint8_t colmod = 0x05; /* RGB565 */ tft_cmd(0x3a, &colmod, 1); uint8_t madctl = 0x00; /* portrait, canvas-upright on the bench */ tft_cmd(0x36, &madctl, 1); tft_cmd(0x29, NULL, 0); /* DISPON */ for (int r = 0; r < TFT_ROWS; r++) tft_draw_row(r, NULL, C_BG); tft_text(0, 0, " PLE TinyLM 28.9M", C_TITLE); tft_text(0, 1, " staging model...", C_STORY); }
main/font5x7.h
/* * Classic 5x7 dot-matrix font, printable ASCII 0x20..0x7E: the ubiquitous * public-domain glyph set every character LCD/OLED library ships (column- * major, 5 bytes per glyph, bit 0 = top row). Rendered at 2x by tft.h. * Verified glyph-by-glyph against a host-side ASCII render * (tools/font_check.c) before committing — not trusted from memory. */ #pragma once #include <stdint.h> #define FONT_W 5 #define FONT_H 7 #define FONT_FIRST 0x20 #define FONT_LAST 0x7e static const uint8_t FONT5X7[95][5] = { {0x00, 0x00, 0x00, 0x00, 0x00}, /* ' ' */ {0x00, 0x00, 0x5f, 0x00, 0x00}, /* ! */ {0x00, 0x07, 0x00, 0x07, 0x00}, /* " */ {0x14, 0x7f, 0x14, 0x7f, 0x14}, /* # */ {0x24, 0x2a, 0x7f, 0x2a, 0x12}, /* $ */ {0x23, 0x13, 0x08, 0x64, 0x62}, /* % */ {0x36, 0x49, 0x55, 0x22, 0x50}, /* & */ {0x00, 0x05, 0x03, 0x00, 0x00}, /* ' */ {0x00, 0x1c, 0x22, 0x41, 0x00}, /* ( */ {0x00, 0x41, 0x22, 0x1c, 0x00}, /* ) */ {0x14, 0x08, 0x3e, 0x08, 0x14}, /* * */ {0x08, 0x08, 0x3e, 0x08, 0x08}, /* + */ {0x00, 0x50, 0x30, 0x00, 0x00}, /* , */ {0x08, 0x08, 0x08, 0x08, 0x08}, /* - */ {0x00, 0x60, 0x60, 0x00, 0x00}, /* . */ {0x20, 0x10, 0x08, 0x04, 0x02}, /* / */ {0x3e, 0x51, 0x49, 0x45, 0x3e}, /* 0 */ {0x00, 0x42, 0x7f, 0x40, 0x00}, /* 1 */ {0x42, 0x61, 0x51, 0x49, 0x46}, /* 2 */ {0x21, 0x41, 0x45, 0x4b, 0x31}, /* 3 */ {0x18, 0x14, 0x12, 0x7f, 0x10}, /* 4 */ {0x27, 0x45, 0x45, 0x45, 0x39}, /* 5 */ {0x3c, 0x4a, 0x49, 0x49, 0x30}, /* 6 */ {0x01, 0x71, 0x09, 0x05, 0x03}, /* 7 */ {0x36, 0x49, 0x49, 0x49, 0x36}, /* 8 */ {0x06, 0x49, 0x49, 0x29, 0x1e}, /* 9 */ {0x00, 0x36, 0x36, 0x00, 0x00}, /* : */ {0x00, 0x56, 0x36, 0x00, 0x00}, /* ; */ {0x08, 0x14, 0x22, 0x41, 0x00}, /* < */ {0x14, 0x14, 0x14, 0x14, 0x14}, /* = */ {0x00, 0x41, 0x22, 0x14, 0x08}, /* > */ {0x02, 0x01, 0x51, 0x09, 0x06}, /* ? */ {0x32, 0x49, 0x79, 0x41, 0x3e}, /* @ */ {0x7e, 0x11, 0x11, 0x11, 0x7e}, /* A */ {0x7f, 0x49, 0x49, 0x49, 0x36}, /* B */ {0x3e, 0x41, 0x41, 0x41, 0x22}, /* C */ {0x7f, 0x41, 0x41, 0x22, 0x1c}, /* D */ {0x7f, 0x49, 0x49, 0x49, 0x41}, /* E */ {0x7f, 0x09, 0x09, 0x09, 0x01}, /* F */ {0x3e, 0x41, 0x49, 0x49, 0x7a}, /* G */ {0x7f, 0x08, 0x08, 0x08, 0x7f}, /* H */ {0x00, 0x41, 0x7f, 0x41, 0x00}, /* I */ {0x20, 0x40, 0x41, 0x3f, 0x01}, /* J */ {0x7f, 0x08, 0x14, 0x22, 0x41}, /* K */ {0x7f, 0x40, 0x40, 0x40, 0x40}, /* L */ {0x7f, 0x02, 0x0c, 0x02, 0x7f}, /* M */ {0x7f, 0x04, 0x08, 0x10, 0x7f}, /* N */ {0x3e, 0x41, 0x41, 0x41, 0x3e}, /* O */ {0x7f, 0x09, 0x09, 0x09, 0x06}, /* P */ {0x3e, 0x41, 0x51, 0x21, 0x5e}, /* Q */ {0x7f, 0x09, 0x19, 0x29, 0x46}, /* R */ {0x46, 0x49, 0x49, 0x49, 0x31}, /* S */ {0x01, 0x01, 0x7f, 0x01, 0x01}, /* T */ {0x3f, 0x40, 0x40, 0x40, 0x3f}, /* U */ {0x1f, 0x20, 0x40, 0x20, 0x1f}, /* V */ {0x3f, 0x40, 0x38, 0x40, 0x3f}, /* W */ {0x63, 0x14, 0x08, 0x14, 0x63}, /* X */ {0x07, 0x08, 0x70, 0x08, 0x07}, /* Y */ {0x61, 0x51, 0x49, 0x45, 0x43}, /* Z */ {0x00, 0x7f, 0x41, 0x41, 0x00}, /* [ */ {0x02, 0x04, 0x08, 0x10, 0x20}, /* \ */ {0x00, 0x41, 0x41, 0x7f, 0x00}, /* ] */ {0x04, 0x02, 0x01, 0x02, 0x04}, /* ^ */ {0x40, 0x40, 0x40, 0x40, 0x40}, /* _ */ {0x00, 0x01, 0x02, 0x04, 0x00}, /* ` */ {0x20, 0x54, 0x54, 0x54, 0x78}, /* a */ {0x7f, 0x48, 0x44, 0x44, 0x38}, /* b */ {0x38, 0x44, 0x44, 0x44, 0x20}, /* c */ {0x38, 0x44, 0x44, 0x48, 0x7f}, /* d */ {0x38, 0x54, 0x54, 0x54, 0x18}, /* e */ {0x08, 0x7e, 0x09, 0x01, 0x02}, /* f */ {0x0c, 0x52, 0x52, 0x52, 0x3e}, /* g */ {0x7f, 0x08, 0x04, 0x04, 0x78}, /* h */ {0x00, 0x44, 0x7d, 0x40, 0x00}, /* i */ {0x20, 0x40, 0x44, 0x3d, 0x00}, /* j */ {0x7f, 0x10, 0x28, 0x44, 0x00}, /* k */ {0x00, 0x41, 0x7f, 0x40, 0x00}, /* l */ {0x7c, 0x04, 0x18, 0x04, 0x78}, /* m */ {0x7c, 0x08, 0x04, 0x04, 0x78}, /* n */ {0x38, 0x44, 0x44, 0x44, 0x38}, /* o */ {0x7c, 0x14, 0x14, 0x14, 0x08}, /* p */ {0x08, 0x14, 0x14, 0x18, 0x7c}, /* q */ {0x7c, 0x08, 0x04, 0x04, 0x08}, /* r */ {0x48, 0x54, 0x54, 0x54, 0x20}, /* s */ {0x04, 0x3f, 0x44, 0x40, 0x20}, /* t */ {0x3c, 0x40, 0x40, 0x20, 0x7c}, /* u */ {0x1c, 0x20, 0x40, 0x20, 0x1c}, /* v */ {0x3c, 0x40, 0x30, 0x40, 0x3c}, /* w */ {0x44, 0x28, 0x10, 0x28, 0x44}, /* x */ {0x0c, 0x50, 0x50, 0x50, 0x3c}, /* y */ {0x44, 0x64, 0x54, 0x4c, 0x44}, /* z */ {0x00, 0x08, 0x36, 0x41, 0x00}, /* { */ {0x00, 0x00, 0x7f, 0x00, 0x00}, /* | */ {0x00, 0x41, 0x36, 0x08, 0x00}, /* } */ {0x08, 0x08, 0x2a, 0x1c, 0x08}, /* ~ */ };
