MLCompilerBridge
Tools for streamlining communication with ML models for compiler optimizations.
Loading...
Searching...
No Matches
Public Types | Public Member Functions | Protected Member Functions | Protected Attributes | Private Member Functions | List of all members
MLBridge::MLModelRunner Class Referenceabstract

MLModelRunner - The main interface for interacting with the ML models. More...

#include <MLModelRunner.h>

Inheritance diagram for MLBridge::MLModelRunner:
Inheritance graph
[legend]

Public Types

enum class  Kind : int {
  Unknown , Pipe , gRPC , ONNX ,
  TFAOT
}
 Type of the MLModelRunner. More...
 

Public Member Functions

 MLModelRunner (const MLModelRunner &)=delete
 
MLModelRunner & operator= (const MLModelRunner &)=delete
 
virtual ~MLModelRunner ()=default
 
template<typename T >
std::enable_if< std::is_fundamental< T >::value, T >::type evaluate ()
 Main user-facing method for interacting with the ML models.
 
template<typename T >
std::enable_if< std::is_fundamental< typenamestd::remove_pointer< T >::type >::value, void >::type evaluate (T &data, size_t &dataSize)
 Main user-facing method for interacting with the ML models.
 
Kind getKind () const
 
SerDesKind getSerDesKind () const
 
virtual void requestExit ()=0
 
template<typename U , typename T , typename... Types>
void populateFeatures (const std::pair< U, T > &var1, const std::pair< U, Types > &...var2)
 User-facing interface for setting the features to be sent to the model.
 
template<typename U , typename T , typename... Types>
void populateFeatures (const std::pair< U, T > &&var1, const std::pair< U, Types > &&...var2)
 
void populateFeatures ()
 
void setRequest (void *request)
 Mainly used in the case of gRPC where the request object is not known explicitly.
 
void setResponse (void *response)
 Mainly used in the case of gRPC where the response object is not known explicitly.
 

Protected Member Functions

 MLModelRunner (Kind Type, SerDesKind SerDesType, llvm::LLVMContext *Ctx=nullptr)
 
 MLModelRunner (Kind Type, llvm::LLVMContext *Ctx=nullptr)
 
virtual void * evaluateUntyped ()=0
 Should be implemented by the derived class to call the model and get the result.
 

Protected Attributes

llvm::LLVMContext * Ctx
 
const Kind Type
 
const SerDesKind SerDesType
 
std::unique_ptr< BaseSerDes > SerDes
 

Private Member Functions

void initSerDes ()
 

Detailed Description

MLModelRunner - The main interface for interacting with the ML models.

Definition at line 56 of file MLModelRunner.h.

Member Enumeration Documentation

◆ Kind

enum class MLBridge::MLModelRunner::Kind : int
strong

Type of the MLModelRunner.

Enumerator
Unknown 
Pipe 
gRPC 
ONNX 
TFAOT 

Definition at line 84 of file MLModelRunner.h.

Constructor & Destructor Documentation

◆ MLModelRunner() [1/3]

MLBridge::MLModelRunner::MLModelRunner ( const MLModelRunner & )
delete

◆ ~MLModelRunner()

virtual MLBridge::MLModelRunner::~MLModelRunner ( )
virtualdefault

◆ MLModelRunner() [2/3]

MLBridge::MLModelRunner::MLModelRunner ( Kind Type,
SerDesKind SerDesType,
llvm::LLVMContext * Ctx = nullptr )
inlineprotected

Definition at line 121 of file MLModelRunner.h.

◆ MLModelRunner() [3/3]

MLBridge::MLModelRunner::MLModelRunner ( Kind Type,
llvm::LLVMContext * Ctx = nullptr )
inlineprotected

Definition at line 128 of file MLModelRunner.h.

Member Function Documentation

◆ evaluate() [1/2]

template<typename T >
std::enable_if< std::is_fundamental< T >::value, T >::type MLBridge::MLModelRunner::evaluate ( )
inline

Main user-facing method for interacting with the ML models.

Definition at line 65 of file MLModelRunner.h.

◆ evaluate() [2/2]

template<typename T >
std::enable_if< std::is_fundamental< typenamestd::remove_pointer< T >::type >::value, void >::type MLBridge::MLModelRunner::evaluate ( T & data,
size_t & dataSize )
inline

Main user-facing method for interacting with the ML models.

Definition at line 74 of file MLModelRunner.h.

◆ evaluateUntyped()

virtual void * MLBridge::MLModelRunner::evaluateUntyped ( )
protectedpure virtual

Should be implemented by the derived class to call the model and get the result.

Implemented in MLBridge::gRPCModelRunner< Client, Stub, Request, Response >, MLBridge::ONNXModelRunner, MLBridge::PipeModelRunner, and MLBridge::TFModelRunner< TGen >.

◆ getKind()

Kind MLBridge::MLModelRunner::getKind ( ) const
inline

Definition at line 86 of file MLModelRunner.h.

◆ getSerDesKind()

SerDesKind MLBridge::MLModelRunner::getSerDesKind ( ) const
inline

Definition at line 87 of file MLModelRunner.h.

◆ initSerDes()

void MLBridge::MLModelRunner::initSerDes ( )
inlineprivate

Definition at line 145 of file MLModelRunner.h.

◆ operator=()

MLModelRunner & MLBridge::MLModelRunner::operator= ( const MLModelRunner & )
delete

◆ populateFeatures() [1/3]

void MLBridge::MLModelRunner::populateFeatures ( )
inline

Definition at line 110 of file MLModelRunner.h.

◆ populateFeatures() [2/3]

template<typename U , typename T , typename... Types>
void MLBridge::MLModelRunner::populateFeatures ( const std::pair< U, T > && var1,
const std::pair< U, Types > &&... var2 )
inline

Definition at line 104 of file MLModelRunner.h.

◆ populateFeatures() [3/3]

template<typename U , typename T , typename... Types>
void MLBridge::MLModelRunner::populateFeatures ( const std::pair< U, T > & var1,
const std::pair< U, Types > &... var2 )
inline

User-facing interface for setting the features to be sent to the model.

The features are passed as a list of key-value pairs. The key is the name of the feature and the value is the value of the feature. The value can be a scalar or a vector.

Definition at line 97 of file MLModelRunner.h.

◆ requestExit()

virtual void MLBridge::MLModelRunner::requestExit ( )
pure virtual

Implemented in MLBridge::gRPCModelRunner< Client, Stub, Request, Response >, MLBridge::ONNXModelRunner, MLBridge::PipeModelRunner, and MLBridge::TFModelRunner< TGen >.

◆ setRequest()

void MLBridge::MLModelRunner::setRequest ( void * request)
inline

Mainly used in the case of gRPC where the request object is not known explicitly.

Definition at line 114 of file MLModelRunner.h.

◆ setResponse()

void MLBridge::MLModelRunner::setResponse ( void * response)
inline

Mainly used in the case of gRPC where the response object is not known explicitly.

Definition at line 118 of file MLModelRunner.h.

Member Data Documentation

◆ Ctx

llvm::LLVMContext* MLBridge::MLModelRunner::Ctx
protected

Definition at line 137 of file MLModelRunner.h.

◆ SerDes

std::unique_ptr<BaseSerDes> MLBridge::MLModelRunner::SerDes
protected

Definition at line 142 of file MLModelRunner.h.

◆ SerDesType

const SerDesKind MLBridge::MLModelRunner::SerDesType
protected

Definition at line 139 of file MLModelRunner.h.

◆ Type

const Kind MLBridge::MLModelRunner::Type
protected

Definition at line 138 of file MLModelRunner.h.

The documentation for this class was generated from the following file:
Generated by doxygen 1.10.0