paper

Orca the origin of continous batching

A Distributed Serving System for Transformer-Based Generative Models

Posted by Kylin on September 8, 2023

[TOC]

osdi 22

这是continuous batching的思想来源：

截屏2023-09-14 09.18.38

实验做了一个latency和throughput的图：

截屏2023-09-14 09.19.52

CATALOG
    FEATURED TAGS
    From Bohemia Linux Git Cloud Computer Architecture Database machine learning paper software coding quant Object Detection job programming Competition lecture nlp competition llmsys
    FRIENDS