Welcome to ventu’s documentation!

Serving the deep learning models easily.

Install

pip install ventu

Features

• Only need to implement Model(preprocess, postprocess, inference or batch_inference)

• request & response data validation using pydantic

• API document using SpecTree (when run with run_http)

• backend service using falcon supports both JSON and msgpack

• dynamic batching with batching using Unix domain socket or TCP

  • errors in one request won’t affect others in the same batch

• support all the runtime

• health check

• inference warm-up

Contents:

• Ventu API
  • Config
  • Protocol
  • HTTP service

Indices and tables

• Index

• Module Index

• Search Page