ONNX Runtime Documentation

Cross-platform, high-performance ML inferencing and training accelerator for deep learning models from PyTorch, TensorFlow, and more

Get Started API Reference

Quick start

Get up and running with ONNX Runtime in minutes

Install the package for your platform and language. Python users can install via pip: ```bash theme={null} pip install onnxruntime ``` For GPU acceleration with CUDA: ```bash theme={null} pip install onnxruntime-gpu ``` Create an inference session with your ONNX model: ```python theme={null} import onnxruntime as ort # Create inference session session = ort.InferenceSession("model.onnx") ``` Prepare your input and run inference: ```python theme={null} import numpy as np # Prepare input input_name = session.get_inputs()[0].name input_data = np.random.randn(1, 3, 224, 224).astype(np.float32) # Run inference outputs = session.run(None, {input_name: input_data}) ```

Key features

Everything you need for production ML deployment

Deploy on Windows, Linux, macOS, iOS, Android, and web browsers with consistent APIs Leverage CUDA, TensorRT, DirectML, CoreML, OpenVINO, and more for optimal performance Use Python, C/C++, C#, Java, and JavaScript with idiomatic APIs for each language Automatic graph optimizations and quantization for faster inference Speed up PyTorch model training with ORTModule integration Convert models from PyTorch, TensorFlow, scikit-learn, and more

Explore by topic

Deep dive into specific areas of ONNX Runtime

Understand ONNX format, execution providers, and sessions Run models for predictions across platforms and languages Accelerate model training with ORTModule Hardware acceleration options for CPUs, GPUs, and specialized chips Optimize inference speed and memory usage Convert models from PyTorch, TensorFlow, and scikit-learn

API reference

Complete API documentation for all supported languages

InferenceSession, SessionOptions, quantization, and transformers OrtApi, sessions, tensors, and execution providers InferenceSession, SessionOptions, and tensor operations OrtSession, OrtEnvironment, and inference APIs Web and Node.js inference with WebAssembly backend

Join the community

Connect with other ONNX Runtime developers and get support

GitHub

Star the repo and contribute

Discussions

Ask questions and share ideas

Ready to accelerate your ML models?

Start deploying high-performance models across platforms with ONNX Runtime

Install ONNX Runtime