MLBridge::PipeModelRunner Class Reference

A MLModelRunner that asks for advice from an external agent, or host. More...

#include <PipeModelRunner.h>

Inheritance diagram for MLBridge::PipeModelRunner:

Collaboration diagram for MLBridge::PipeModelRunner:

Public Member Functions
	PipeModelRunner (llvm::StringRef OutboundName, llvm::StringRef InboundName, SerDesKind Kind, llvm::LLVMContext *Ctx=nullptr)
void	requestExit () override
virtual	~PipeModelRunner ()
Public Member Functions inherited from MLBridge::MLModelRunner
	MLModelRunner (const MLModelRunner &)=delete
MLModelRunner &	operator= (const MLModelRunner &)=delete
virtual	~MLModelRunner ()=default
template<typename T >
std::enable_if< std::is_fundamental< T >::value, T >::type	evaluate ()
	Main user-facing method for interacting with the ML models.
template<typename T >
std::enable_if< std::is_fundamental< typenamestd::remove_pointer< T >::type >::value, void >::type	evaluate (T &data, size_t &dataSize)
	Main user-facing method for interacting with the ML models.
Kind	getKind () const
SerDesKind	getSerDesKind () const
template<typename U , typename T , typename... Types>
void	populateFeatures (const std::pair< U, T > &var1, const std::pair< U, Types > &...var2)
	User-facing interface for setting the features to be sent to the model.
template<typename U , typename T , typename... Types>
void	populateFeatures (const std::pair< U, T > &&var1, const std::pair< U, Types > &&...var2)
void	populateFeatures ()
void	setRequest (void *request)
	Mainly used in the case of gRPC where the request object is not known explicitly.
void	setResponse (void *response)
	Mainly used in the case of gRPC where the response object is not known explicitly.

Static Public Member Functions
static bool	classof (const MLModelRunner *R)

Private Member Functions
void	send (void *data)
void *	receive ()
void *	evaluateUntyped () override
	Should be implemented by the derived class to call the model and get the result.
std::string	readNBytes (size_t N)

Private Attributes
std::string	InboundName
int	Inbound = -1
std::error_code	OutEC
std::error_code	InEC
std::unique_ptr< llvm::raw_fd_ostream >	OutStream

Additional Inherited Members
Public Types inherited from MLBridge::MLModelRunner
enum class	Kind : int {   Unknown , Pipe , gRPC , ONNX ,   TFAOT }
	Type of the MLModelRunner. More...
Protected Member Functions inherited from MLBridge::MLModelRunner
	MLModelRunner (Kind Type, SerDesKind SerDesType, llvm::LLVMContext *Ctx=nullptr)
	MLModelRunner (Kind Type, llvm::LLVMContext *Ctx=nullptr)
Protected Attributes inherited from MLBridge::MLModelRunner
llvm::LLVMContext *	Ctx
const Kind	Type
const SerDesKind	SerDesType
std::unique_ptr< BaseSerDes >	SerDes

Detailed Description

A MLModelRunner that asks for advice from an external agent, or host.

It uses 2 files - ideally named pipes - one to send data to that agent, and one to receive advice. The compiler will send observations; the host is expected to reply with a tensor value after each observation as a binary buffer that's conforming to the shape of the advice. Interleaved, the data closely resembles the training log for a log where we don't capture the reward signal.

Note that the correctness of the received data is the responsibility of the host. In particular, if insufficient data were sent, the compiler will block when waiting for an advice.

Note that the host can either open the pipes RW, or open first the pipe to the compiler - i.e. the "Inbound" - and then the "Outbound", to avoid deadlock. This is because the compiler first tries to open the inbound (which will hang until there's a writer on the other end).

Definition at line 55 of file PipeModelRunner.h.

Constructor & Destructor Documentation

PipeModelRunner()

MLBridge::PipeModelRunner::PipeModelRunner	(	llvm::StringRef	OutboundName,
		llvm::StringRef	InboundName,
		SerDesKind	Kind,
		llvm::LLVMContext *	Ctx = nullptr )

Definition at line 33 of file PipeModelRunner.cpp.

~PipeModelRunner()

MLBridge::PipeModelRunner::~PipeModelRunner ( )

virtual

Definition at line 77 of file PipeModelRunner.cpp.

Member Function Documentation

classof()

static bool MLBridge::PipeModelRunner::classof ( const MLModelRunner * R )

inlinestatic

Definition at line 60 of file PipeModelRunner.h.

evaluateUntyped()

void * MLBridge::PipeModelRunner::evaluateUntyped ( )

overrideprivatevirtual

Should be implemented by the derived class to call the model and get the result.

Implements MLBridge::MLModelRunner.

Definition at line 134 of file PipeModelRunner.cpp.

readNBytes()

std::string MLBridge::PipeModelRunner::readNBytes ( size_t N )

private

Definition at line 85 of file PipeModelRunner.cpp.

receive()

void * MLBridge::PipeModelRunner::receive ( )

private

Definition at line 120 of file PipeModelRunner.cpp.

requestExit()

void MLBridge::PipeModelRunner::requestExit ( )

inlineoverridevirtual

Implements MLBridge::MLModelRunner.

Definition at line 64 of file PipeModelRunner.h.

send()

void MLBridge::PipeModelRunner::send ( void * data )

private

Definition at line 106 of file PipeModelRunner.cpp.

Member Data Documentation

Inbound

int MLBridge::PipeModelRunner::Inbound = -1

private

Definition at line 75 of file PipeModelRunner.h.

InboundName

std::string MLBridge::PipeModelRunner::InboundName

private

Definition at line 74 of file PipeModelRunner.h.

InEC

std::error_code MLBridge::PipeModelRunner::InEC

private

Definition at line 77 of file PipeModelRunner.h.

OutEC

std::error_code MLBridge::PipeModelRunner::OutEC

private

Definition at line 76 of file PipeModelRunner.h.

OutStream

std::unique_ptr<llvm::raw_fd_ostream> MLBridge::PipeModelRunner::OutStream

private

Definition at line 78 of file PipeModelRunner.h.

---

The documentation for this class was generated from the following files:

• include/MLModelRunner/PipeModelRunner.h
• MLModelRunner/PipeModelRunner.cpp