资讯AI 辅助草稿来源原文 ↗

热点Show HN: TrnRT – An open-source LLM inference engine for AWS Trainium

github.com · 发布于 2026-11-17 16:01

来源节选(非本站原创长文,完整内容请见来源)

Hi HN, Sanchit here from RunAnywhere (YC W26). We just open-sourced TrnRT: Trainium RunTime, our inference engine for AWS Trainium. We started 3 weeks back, and currently we’re able to serve and run GLM 5.3 flash (FP8) on AWS trn2.48xlarge, and will bring more models support soon.

It’s open to contributions :) What it does today: - Serves 2 instances of GLM-5.3 flash on a single box, 8 trn2 with a cache aware router. - Can process 3.57M TPM (tokens per minute) on SemiAnalysis's AgentX benchmarks.

来源:Hacker News 讨论