Triton Inference Server Feature Requests

NBX Windows Requirements:

• Platform: Windows 10 64-bit

• Version: 1.0.0.30

• GPU: GeForce RTX 2060, Quadro RTX 3000, TITAN RTX or higher

• CPU: recommended: Intel Core i5 8600, AMD Ryzen r5 2600 or higher

• Driver: NVIDIA Display Driver version 456.38 or higher

Clara team has received requests from medical instrument companies on Windows (no data center customers so far however).

Framework backend support: Tensorflow, Pytorch, TRT using ONNX

Hardware support: Quadro RTX typically 4000 - 8000 class. Many customers are now looking at embedded

Versions of Windows:

Medical instruments use Windows 10 IoT Enterprise. Existing designs used 2016 LTSB, new designs would use the updated version 2019 LTSC. This would primarily be using the TensorRT backend.

Prior customer engagements:

RBX/NBX requesting support on Windows for C++ library interface, not gRPC or REST

MSFT Bing requests Windows support for TRTIS.

Healthcare: GE refuses to use TRTIS unless it builds on Windows

Phillips trying to use for image recon, but need quick time to first inference so using VMs is not an option

Need FWs to have Windows build, then more core TRTIS work for windows support