MODEL FILES · EVERY BYTE ACCOUNTED FOR
Weight Layout on Disk (GGUF / SafeTensors)
Lay out a five-tensor model in GGUF v3 or safetensors and read the exact byte offsets of its header, metadata, tensor table, padding and data.
Your experiment
Start with a GGUF file of Q8_0 weights, hidden size 256 and 1,000 tokens. Read where the data section starts, then turn off padding, change the alignment, try Q4_K with hidden size 320, and switch to safetensors with Q8_0 and then F16.
Every input recomputes the result immediately; there is no animation because nothing here unfolds over time. An input outside its allowed range is rejected with a message and the previous valid result stays on screen.
Hex dump and header
Computed data
Metrics
GGUF follows the v3 specification in ggml's docs/gguf.md: a 24-byte header, metadata keys as length-prefixed strings with a uint32 value type, tensor infos with uint64 dimensions in ggml order, a uint32 ggml type and a uint64 offset relative to the data section, which must be a multiple of the alignment (32 unless general.alignment says otherwise). safetensors follows its README and reference writer: a u64 header length, a JSON header space-padded to a multiple of 8, tensors ordered by dtype size and then name, and a contiguous byte buffer. The five tensors are a minimal illustration, not a complete model.
A GGUF file from the first byte
With File format set to GGUF v3, the file opens with a 24-byte header: the magic bytes 47 47 55 46, version 3, the tensor count and the metadata count. Metadata follows as key strings with an 8-byte length, a 4-byte value type and the value. Then comes one tensor info per tensor: name, number of dimensions, each dimension as 8 bytes, the ggml type and an offset. In the opening file general.architecture takes 45 bytes and general.name 41, and the five tensor infos end at byte 391.
Alignment and padding
Tensor data must start at a multiple of the alignment, 32 by default, so the writer adds 25 zero bytes and the data section starts at byte 416. Each tensor offset is relative to that section and is itself rounded up, so token_embd, 272,000 bytes of Q8_0, is followed by the norm at offset 272,000. Turn off padding and the data starts at 391: INVALID, because a reader that maps the file expects aligned tensors. Turning off Writer pads to alignment drops those zero bytes. Setting GGUF alignment to 64, written as general.alignment, moves the data to byte 448.
Block types and odd shapes
The Weight type sets how the matrices are stored. Q8_0 stores 32 weights in 34 bytes, so the 256 x 1,000 embedding is 1,000 rows of 8 blocks; the Vocabulary size sets that row count, so raising it grows the embedding and the data section. One-dimensional norms stay F32. Q4_K needs rows that are a multiple of 256; with hidden size 320 the embedding and the query weight cannot be split into 256-weight blocks and are stored as Q5_0, which is what llama.cpp falls back to, while ffn_down, with rows of 1,280, stays Q4_K. The file is 523,776 bytes against 330,784 for Q4_K at hidden size 256.
The safetensors layout
A safetensors file starts with an 8-byte little-endian header length, then a JSON header naming every tensor with its dtype, its shape in PyTorch order and its byte range, then one contiguous buffer. Choosing Q8_0 gives INVALID: safetensors has no block-quantized dtype. With F16 the header is 480 bytes and the data starts at byte 488. The reference writer puts the two F32 norms first because it orders tensors by dtype size, then by name, and pads the header with spaces to a multiple of 8; without that padding the buffer would start at byte 485.
Reading the tool and its limits
The stacked bar splits the file into header, padding and tensor data, the lanes report the alignment checks and fallbacks, the table gives every region's start and length, and the text panel prints the first 96 bytes in hex plus the safetensors header. Only the header bytes are real; tensor data is sized, not filled. A complete model would also carry tokenizer metadata, many more tensors and possibly several shard files.