ray.serve.start#

ray.serve.start(detached: bool = True, proxy_location: Union[None, str, ray.serve.config.ProxyLocation] = None, http_options: Union[None, dict, ray.serve.config.HTTPOptions] = None, dedicated_cpu: bool = False, grpc_options: Optional[ray.serve.config.gRPCOptions] = None, **kwargs)[source]#

Start Serve on the cluster.

Used to set cluster-scoped configurations such as HTTP options. In most cases, this does not need to be called manually and Serve will be started when an application is first deployed to the cluster.

These cluster-scoped options cannot be updated dynamically. To update them, start a new cluster or shut down Serve on the cluster and start it again.

These options can also be set in the config file deployed via REST API.

Parameters

  • detached – [DEPRECATED: in the future, this will always be True] Whether or not the instance should be detached from this script. If set, the instance will live on the Ray cluster until it is explicitly stopped with serve.shutdown().

  • proxy_location – Where to run proxies that handle ingress traffic to the cluster (defaults to every node in the cluster with at least one replica on it). See ProxyLocation for supported options.

  • http_options – HTTP config options for the proxies. These can be passed as an unstructured dictionary or the structured HTTPOptions class. See HTTPOptions for supported options.

  • dedicated_cpu – [DEPRECATED] Whether to reserve a CPU core for the Serve controller actor.

  • grpc_options – [EXPERIMENTAL] gRPC config options for the proxies. See gRPCOptions for supported options.

PublicAPI: This API is stable across Ray releases.