gRPCModelRunner.h File Reference

gRPCModelRunner class supporting communication via gRPC. More...

#include "MLModelRunner/MLModelRunner.h"
#include <future>
#include <google/protobuf/text_format.h>
#include <grpcpp/grpcpp.h>
#include <grpcpp/health_check_service_interface.h>
#include <memory>
#include <thread>

Include dependency graph for gRPCModelRunner.h:

Go to the source code of this file.

Classes
class	MLBridge::gRPCModelRunner< Client, Stub, Request, Response >
	This class is used to create the grpc model runner object. More...

Namespaces
namespace	MLBridge

Detailed Description

gRPCModelRunner class supporting communication via gRPC.

This class is used to communicate with the gRPC server and send/receive data to/from the model. Supports interleaved communication with the model.

There are two ways to use this class:

1. Training mode - gRPC Server: In this mode, the gRPCModelRunner object is created with the server address and the service object. The service object is used to create the server and the server waits for the client to connect. Once the client connects, the server waits for the client to send the request and then responds with the result. In Training mode, GrpcCompilerInterface class from GrpcCompilerInterface.py acts as the client and sends the request to the server.
2. Inference mode - gRPC Client: In this mode, the gRPCModelRunner object is created with the server address, request and response objects. The request object is used to send the features to the model and the response object is used to receive the result from the model.

In Inference mode, the compiler pass using this class acts as the client and sends the request to the server implemented by using GrpcCompilerInterface.py.

Interfacing with the model using protobuf (.proto) files: Users should define the service, stub, request and response by writing a .proto file. The service and stub objects are generated using the protoc compiler. The request and response objects are generated using the protoc compiler or created by the user. The build process will automatically compile .proto files and generates the relevant stubs and request/response objects in both CPP and Python which will be used by gRPCModelRunner and GrpcCompilerInterface respectively..

In Inference mode, users should override getAdvice() RPC method in the Python model code to process the request and send the response back to the client. This method is called by the gRPC server in the evaluate_untyped() method of gRPCModelRunner class to get the result from the model after populating the features in the request object.

In Training mode, users should override the RPC function/service that they declare in the .proto file in the Compiler pass which is using the gRPCModelRunner. This function is called by the gRPC client (Python model) that is using the GrpcCompilerInterface class to send the request to the server.

Usage:

1. Create a .proto file with the service and message definitions
2. Generate the stubs using protoc
3. Create a gRPCModelRunner object with the server address, stub, request and response
4. Populate the features to be sent to the model
5. Call evaluate() to get the result back from the model

Definition in file gRPCModelRunner.h.