Why publish this
Law firms evaluating private AI need more than model names. They need to know which hardware class, model format, backend, context setting, and quantization path actually generated legal output. These profiles publish the successful parts of PLI Labs' owned-hardware record while leaving out hostnames, network addresses, local paths, service manager files, and failed runbooks.
Public boundary: these are measured success profiles, not deployment instructions for a specific private machine. Replace model paths, endpoints, and accelerator grouping with your own validated environment.
B70-class successful lane
The B70 profile used a four-card Intel Arc Pro B70 workstation with 32 GB per card, Intel Level Zero / SYCL, and a GGUF-compatible llama.cpp server path. The promoted lane was a layer-split large-MoE GGUF stack for long-context legal drafting and review.
llama-server \
--model <local-gguf-path> \
-ngl 99 \
-sm layer \
-c 12288 \
-b 16384 \
-ub 16384 \
-np 1 \
-cb \
-ctk f16 \
-ctv f16 \
-fa auto \
--kv-offload \
--defrag-thold 0.1 \
--no-mmap| Tier | Prompt | Output | Prompt tok/s | Decode tok/s | Client tok/s |
|---|---|---|---|---|---|
| L0 legal canary | 9,274 | 1,024 | 609.79 | 35.17 | 23.06 |
| L0 service window | 9,274 | 1,024 | 613.64 | 35.24 | 23.15 |
| L1 medium legal context | 17,646 | 2,048 | 584.44 | 34.79 | 22.97 |
| L2 long legal context | 35,034 | 2,048 | 482.60 | 30.80 | 14.71 |
The practical finding is simple: the B70 layer-split lane was usable for real legal prompts, with decode near 35 tokens per second at shorter and medium legal tiers and near 31 tokens per second at the longer tier.
V100-class successful lanes
The V100 profile used an anonymous V100-SXM2-32GB multi-GPU Linux workstation. The public benchmark profile used nine V100 32GB accelerators, 288 GB aggregate VRAM, and local model storage. The useful lesson is not aggregate memory by itself; it is validated model/backend/context fit.
Ollama / GGUF local runner
The GGUF lane used Ollama-style local model management, model-specific GGUF quantization, and both 8K and max-context tests where available.
| Model class | VRAM | 8K decode | Max decode | 8K prompt | Max context |
|---|---|---|---|---|---|
| Qwen 30B-A3B MoE | 18 GB | 97 | 94 | 325 | 262K |
| Gemma 26B MoE | 17 GB | 88 | 86 | 372 | 262K |
| Qwen 35B-A3B MoE | 24 GB | 62 | 63 | 257 | 262K |
| Command R 32B | 18 GB | 39 | 38 | 501 | 262K |
| Qwen 122B-A10B MoE | 81 GB | 36 | 37 | 112 | 262K |
| Vision 32B | 20 GB | 23 | n/a | 192 | 256K |
| Dense 31B | 19 GB | 24 | 28 | 72 | 262K |
| Command R+ 104B | 59 GB | 12 | 12 | 145 | 65K |
| Dense 70B reasoning | 75 GB | 10 | 10 | 90 | 262K |
V100-focused vLLM API serving
The API lane used AWQ4 model artifacts, FP8 E5M2 KV cache storage, and an explicitly validated context tier.
python -m vllm.entrypoints.openai.api_server \
--model <local-awq-model> \
--quantization awq \
--kv-cache-dtype fp8_e5m2 \
--max-model-len <context-tier> \
--tensor-parallel-size <validated-v100-group-size>| Model class | Context tier | Wall time | Completion tok/s |
|---|---|---|---|
| 172B-class MoE AWQ | 92,704 | 19.54 s | 13.10 |
| 139B-class MoE AWQ | 131,072 | 17.42 s | 14.70 |
| 162B-class MoE AWQ | 131,072 | 17.65 s | 14.50 |
Hugging Face releases
- MiniMax-M2.7 legal REAP 0.35 checkpoint
- MiniMax-M2.7 legal REAP 0.45 checkpoint
- MiniMax-M2.7 legal REAP 0.35 GGUF
- MiniMax-M2.7 legal REAP 0.45 GGUF
- MiniMax-M2.7 REAP 172B-A10B Q4_K_M GGUF