Welcome to ventu’s documentation!

pypi versions Python Test Python document

Serving the deep learning models easily.

Install

pip install ventu

Features

  • Only need to implement Model(preprocess, postprocess, inference or batch_inference)

  • request & response data validation using pydantic

  • API document using SpecTree (when run with run_http)

  • backend service using falcon supports both JSON and msgpack

  • dynamic batching with batching using Unix domain socket or TCP

    • errors in one request won’t affect others in the same batch

  • support all the runtime

  • health check

  • inference warm-up

Indices and tables