This class implements SparseTensor. This class holds sparse non-zero data (values) and sparse format specific indices. There are two main uses for the class (similar to that of Tensor) More...

#include <sparse_tensor.h>

Classes
class	BlockSparseMutator
	Mutable data access More...

class	BlockSparseView
	Read only access to BlockSparse index More...

class	CooMutator
	Gives mutable access to Coo buffers so they can be populated More...

class	CooView
	Read only access to Coo indices More...

class	CsrMutator
	Give writable access to Csr values and indices More...

class	CsrView
	Read only access to Csr indices More...

Public Member Functions
	SparseTensor (MLDataType elt_type, const TensorShape &dense_shape, const TensorShape &values_shape, void *values_data, const OrtMemoryInfo &location)
	This constructs an instance that points to user defined buffers. Make use of Use* functions to supply format specific indices that reside in the user supplied buffers. The instance constructed this way will not copy data. The lifespan of supplied buffers is expected to eclipse the lifespan of the sparse tensor instance. More...

	SparseTensor (MLDataType elt_type, const TensorShape &dense_shape, std::shared_ptr< IAllocator > allocator)
	Use this constructor to hold sparse data in the buffer allocated with the specified allocator. Use Make*() methods to populate the instance with data which will be copied into the allocated buffer. More...

	SparseTensor () noexcept

	~SparseTensor ()

	ORT_DISALLOW_COPY_AND_ASSIGNMENT (SparseTensor)

size_t	NumValues () const

const Tensor &	Values () const noexcept
	Read only accessor to non-zero values More...

	SparseTensor (SparseTensor &&o) noexcept

SparseTensor &	operator= (SparseTensor &&o) noexcept

SparseFormat	Format () const noexcept
	Returns SparseFormat that the instance currently holds if the value returned in kUndefined, the instance is not populated More...

const TensorShape &	DenseShape () const noexcept
	Returns a would be dense_shape More...

int64_t	RequiredAllocationSize () const noexcept
	Calculates and returns how much this fully initialized SparseTensor data (would) occupy in a contiguous allocation block, or, in fact, occupies if it owns the buffer. More...

int32_t	GetElementType () const
	Returns Tensor element type enum. Useful for type dispatching More...

MLDataType	DataType () const noexcept
	Return Element MLDataType More...

bool	IsDataTypeString () const
	Test for string type More...

template<class T >
bool	IsDataType () const
	Checks if the Tensor contains data type T More...

const OrtMemoryInfo &	Location () const noexcept

CooView	AsCoo () const
	Returns Coo index view More...

Status	UseCooIndices (gsl::span< int64_t > indices)
	Uses COO index contained in the user allocated buffer along with the values buffer passed on to the constructor. The buffer is used as is and its lifespan must eclipse the lifespan of the sparse tensor instance. The OrtMemoryInfo (location) of the index is assumed to be the same as values. More...

Status	MakeCooData (const IDataTransfer &data_transfer, const OrtMemoryInfo &data_location, size_t values_count, const void *values_data, gsl::span< const int64_t > indices)
	The method allocates a single contiguous buffer and copies specified values and indices into it using supplied IDataTransfer. More...

Status	MakeCooStrings (size_t string_count, const char const strings, gsl::span< const int64_t > indices)
	The method allocates a single contiguous buffer and creates instances of std::strings in it, with copies of the supplied zero-terminated strings followed by COO indices. All data is assumed to be on CPU and the allocator supplied must be a CPU based allocator. More...

CooMutator	MakeCooData (size_t values_count, size_t index_count)
	Allocates memory for values and index and returns a mutator so data can be copied into the buffer. More...

CsrView	AsCsr () const
	Returns Csr indices read only view More...

Status	UseCsrIndices (gsl::span< int64_t > inner_index, gsl::span< int64_t > outer_index)
	This function will use Csr indices contained within the user allocated buffers. The lifespan of the buffers must eclipse the lifespan of sparse tensor instance. More...

Status	MakeCsrData (const IDataTransfer &data_transfer, const OrtMemoryInfo &data_location, size_t values_count, const void *values_data, gsl::span< const int64_t > inner_index, gsl::span< const int64_t > outer_index)
	The function will allocate a single contiguous buffer and will copy values and indices into it. More...

Status	MakeCsrStrings (size_t string_count, const char const strings, gsl::span< const int64_t > inner_index, gsl::span< const int64_t > outer_index)
	The method allocates a single contiguous buffer and creates instances of std::strings in it, with copies of the supplied zero-terminated strings followed by COO indices. All data is assumed to be on CPU and the allocator supplied must be a CPU based allocator More...

CsrMutator	MakeCsrData (size_t values_count, size_t inner_index_count, size_t outer_index_count)
	Allocates memory for values and index and returns mutator so data can be populated. More...

BlockSparseView	AsBlockSparse () const
	Return BlockSparseIndex view More...

Status	UseBlockSparseIndices (const TensorShape &indices_shape, int32_t *indices_data)
	Use blocksparse indices contained in the user allocated buffer. The shape of the index must be 2-D and must contain one tuple per each of the value blocks that were supplied to the constructor. The supplied buffer lifespan must eclipse the life of sparse tensor instance. More...

Status	MakeBlockSparseData (const IDataTransfer &data_transfer, const OrtMemoryInfo &data_location, const TensorShape &values_shape, const void values_data, const TensorShape &indices_shape, const int32_t indices_data)
	The function allocates a single contiguous buffer and copies values and index into it. The shape of the values is expected to be at least 3-D but may contain more dimensions. At the very minimum it should be (num_blocks, block_size, block_size). More...

Status	MakeBlockSparseStrings (const TensorShape &values_shape, const char const strings, const TensorShape &indices_shape, const int32_t *indices_data)
	The method allocates a single contiguous buffer and creates instances of std::strings in it, with copies of the supplied zero-terminated strings followed by COO indices. All data is assumed to be on CPU and the allocator supplied must be a CPU based allocator. More...

BlockSparseMutator	MakeBlockSparseData (const TensorShape &values_shape, const TensorShape &indices_shape)
	Allocates memory for values and index and returns mutator so data can be populated More...

Status	Copy (const DataTransferManager &data_transfer_manager, SparseTensor &dst_tensor) const
	X-device copy. Destination tensor must have allocator set. More...

Status	Copy (const IDataTransfer &data_transfer, SparseTensor &dst_tensor) const
	X-device copy. Destination tensor must have allocator set. More...

Static Public Member Functions
static void	InitOrtValue (MLDataType elt_type, const TensorShape &dense_shape, const TensorShape &values_shape, void *values_data, const OrtMemoryInfo &location, OrtValue &ort_value)
	The factory function creates an instance of SparseTensor on the heap using appropriate constructor and initializes OrtValue instance wit it. More...

static void	InitOrtValue (MLDataType elt_type, const TensorShape &dense_shape, std::shared_ptr< IAllocator > allocator, OrtValue &ort_value)
	The factory function creates an instance of SparseTensor on the heap using appropriate constructor and initializes OrtValue instance wit it. More...

static const SparseTensor &	GetSparseTensorFromOrtValue (const OrtValue &v)
	The function will check if the OrtValue is allocated fetch the containing SparseTensor instance or throw if it does not contain one. It will check that the SparseTensor has sparse format set (i.e. fully constructed). More...

static SparseTensor &	GetSparseTensorFromOrtValue (OrtValue &v)
	/// The function will check if the OrtValue is allocated fetch the containing SparseTensor instance or throw if it does not contain one. It will check that the SparseTensor does not have sparse format set and will return non-const ref to so indices can be added to it. More...

Detailed Description

This class implements SparseTensor. This class holds sparse non-zero data (values) and sparse format specific indices. There are two main uses for the class (similar to that of Tensor)

one is to re-present model sparse inputs. Such inputs typically reside in user allocated buffers that are not owned by SparseTensor instance and the instance serves as a facade to expose user allocated buffers. Such buffers should already contain proper values and format specific indices. Use the first constructor to instantiate SparseTensor and supply values_data pointer. Use*() functions can be used to supply pointers to format specific indices. These buffers are used as is and will not be modified or deallocated by the instance. However, the lifespan of the buffers must eclipse the lifespan of the SparseTensor instance.
Represent sparse data that is a result of format conversion or a computation result. Use second constructor to supply a desired allocator. Use Make*() format specific interfaces to supply values and format specific indices. The specified data will be copied into an internally allocated buffer. Internally, we will represent a SparseTensor as a single contiguous buffer that contains values followed by format specific indices. We use Tensors to project values and indices into various parts of buffer.

Definition at line 55 of file sparse_tensor.h.

Constructor & Destructor Documentation

onnxruntime::SparseTensor::SparseTensor	(	MLDataType	elt_type,
		const TensorShape &	dense_shape,
		const TensorShape &	values_shape,
		void *	values_data,
		const OrtMemoryInfo &	location
	)

This constructs an instance that points to user defined buffers. Make use of Use* functions to supply format specific indices that reside in the user supplied buffers. The instance constructed this way will not copy data. The lifespan of supplied buffers is expected to eclipse the lifespan of the sparse tensor instance.

Parameters

elt_type	MlDataType
dense_shape	a shape of original tensor in dense form
values_shape	shape for user supplied values. Use {0} shape for fully sparse tensors.
values_data	a pointer to values. Use nullptr for fully sparse tensors.
location	description of the user allocated memory

onnxruntime::SparseTensor::SparseTensor	(	MLDataType	elt_type,
		const TensorShape &	dense_shape,
		std::shared_ptr< IAllocator >	allocator
	)

Use this constructor to hold sparse data in the buffer allocated with the specified allocator. Use Make*() methods to populate the instance with data which will be copied into the allocated buffer.

Parameters

elt_type
dense_shape
allocator

onnxruntime::SparseTensor::SparseTensor ( )

noexcept

onnxruntime::SparseTensor::~SparseTensor ( )

onnxruntime::SparseTensor::SparseTensor ( SparseTensor && o )

noexcept

Member Function Documentation

BlockSparseView onnxruntime::SparseTensor::AsBlockSparse ( ) const

Return BlockSparseIndex view

Returns: an instance of BlockSparseView

CooView onnxruntime::SparseTensor::AsCoo ( ) const

Returns Coo index view

Returns: CooView instance

CsrView onnxruntime::SparseTensor::AsCsr ( ) const

Returns Csr indices read only view

Returns

Status onnxruntime::SparseTensor::Copy	(	const DataTransferManager &	data_transfer_manager,
		SparseTensor &	dst_tensor
	)		const

X-device copy. Destination tensor must have allocator set.

Parameters

data_transfer_manager
exec_q_id
dst_tensor

Returns

Status onnxruntime::SparseTensor::Copy	(	const IDataTransfer &	data_transfer,
		SparseTensor &	dst_tensor
	)		const

X-device copy. Destination tensor must have allocator set.

Parameters

dst_tensor

Returns

MLDataType onnxruntime::SparseTensor::DataType ( ) const

inlinenoexcept

Return Element MLDataType

Returns

Definition at line 201 of file sparse_tensor.h.

const TensorShape& onnxruntime::SparseTensor::DenseShape ( ) const

inlinenoexcept

Returns a would be dense_shape

Returns: reference to dense_shape

Definition at line 177 of file sparse_tensor.h.

SparseFormat onnxruntime::SparseTensor::Format ( ) const

inlinenoexcept

Returns SparseFormat that the instance currently holds if the value returned in kUndefined, the instance is not populated

Returns: format enum

Definition at line 169 of file sparse_tensor.h.

int32_t onnxruntime::SparseTensor::GetElementType ( ) const

inline

Returns Tensor element type enum. Useful for type dispatching

Returns

Definition at line 193 of file sparse_tensor.h.

static const SparseTensor& onnxruntime::SparseTensor::GetSparseTensorFromOrtValue ( const OrtValue & v )

static

The function will check if the OrtValue is allocated fetch the containing SparseTensor instance or throw if it does not contain one. It will check that the SparseTensor has sparse format set (i.e. fully constructed).

Parameters

v	OrtValue instance

Returns: const SparseTensor Reference

static SparseTensor& onnxruntime::SparseTensor::GetSparseTensorFromOrtValue ( OrtValue & v )

static

/// The function will check if the OrtValue is allocated fetch the containing SparseTensor instance or throw if it does not contain one. It will check that the SparseTensor does not have sparse format set and will return non-const ref to so indices can be added to it.

Parameters

v OrtValue

Returns: non-const reference to SparseTensor

static void onnxruntime::SparseTensor::InitOrtValue	(	MLDataType	elt_type,
		const TensorShape &	dense_shape,
		const TensorShape &	values_shape,
		void *	values_data,
		const OrtMemoryInfo &	location,
		OrtValue &	ort_value
	)

static

The factory function creates an instance of SparseTensor on the heap using appropriate constructor and initializes OrtValue instance wit it.

Parameters

elt_type	element data type
dense_shape	dense shape of the sparse tensor
values_shape	values shape. Use {0} for fully sparse tensors.
values_data	pointer to a user allocated buffer. Use nullptr for fully sparse tensors.
location	description of the user allocated buffer
ort_value	default constructed input/output ort_value

static void onnxruntime::SparseTensor::InitOrtValue	(	MLDataType	elt_type,
		const TensorShape &	dense_shape,
		std::shared_ptr< IAllocator >	allocator,
		OrtValue &	ort_value
	)

static

The factory function creates an instance of SparseTensor on the heap using appropriate constructor and initializes OrtValue instance wit it.

Parameters

elt_type	element data type
dense_shape	dense shape of the sparse tensor
allocator	allocator to use
ort_value	default constructed input/output ort_value

template<class T >

bool onnxruntime::SparseTensor::IsDataType ( ) const

inline

Checks if the Tensor contains data type T

Template Parameters

T

Returns: true if tensor contains data of type T

Definition at line 219 of file sparse_tensor.h.

bool onnxruntime::SparseTensor::IsDataTypeString ( ) const

inline

Test for string type

Returns: true if tensor values are strings

Definition at line 209 of file sparse_tensor.h.

const OrtMemoryInfo& onnxruntime::SparseTensor::Location ( ) const

inlinenoexcept

Definition at line 223 of file sparse_tensor.h.

Status onnxruntime::SparseTensor::MakeBlockSparseData	(	const IDataTransfer &	data_transfer,
		const OrtMemoryInfo &	data_location,
		const TensorShape &	values_shape,
		const void *	values_data,
		const TensorShape &	indices_shape,
		const int32_t *	indices_data
	)

The function allocates a single contiguous buffer and copies values and index into it. The shape of the values is expected to be at least 3-D but may contain more dimensions. At the very minimum it should be (num_blocks, block_size, block_size).

Parameters

data_transfer
data_location
values_shape	The shape is expected to be at least 3-D. However, use {0} for fully sparse tensors.
values_data	Pointer to a data to be copied. Use nullptr for fully sparse tensors.
indices_shape	The shape is expected to be 2-D. However, you can use {0} for fully sparse tensors.
indices_data	Pointer to index data to be copied. Use nullptr for fully sparse tensors.

Returns

BlockSparseMutator onnxruntime::SparseTensor::MakeBlockSparseData	(	const TensorShape &	values_shape,
		const TensorShape &	indices_shape
	)

Allocates memory for values and index and returns mutator so data can be populated

Parameters

values_shape	Shape is expected to be 3-D, use {0} for fully sparse tensors
indices_shape	Shape is expected to be 2-D, use {0} for fully sparse tensors

Returns

Status onnxruntime::SparseTensor::MakeBlockSparseStrings	(	const TensorShape &	values_shape,
		const char const	strings,
		const TensorShape &	indices_shape,
		const int32_t *	indices_data
	)

The method allocates a single contiguous buffer and creates instances of std::strings in it, with copies of the supplied zero-terminated strings followed by COO indices. All data is assumed to be on CPU and the allocator supplied must be a CPU based allocator.

Parameters

values_shape	Use {0} shape for fully sparse tensors
strings	array of char* ptrs, use nullptr for fully sparse tensor
indices_shape	Use {0} for fully sparse tensors
indices_data	use nullptr for fully sparse tensors

Returns

Status onnxruntime::SparseTensor::MakeCooData	(	const IDataTransfer &	data_transfer,
		const OrtMemoryInfo &	data_location,
		size_t	values_count,
		const void *	values_data,
		gsl::span< const int64_t >	indices
	)

The method allocates a single contiguous buffer and copies specified values and indices into it using supplied IDataTransfer.

The indices size must either exactly match the number of values in which case indices shape would be 1-D (values_count) or it must be twice the number of values in which case its shape would be 2-D (values_count, 2).

Values shape is supplied at construction time and its Size() must match values_count.

Parameters

values_count	Use 0 for fully sparse tensors.
values_data	pointer to a buffer to be copied. Use nullptr for fully sparse tensors.
indices

Returns

CooMutator onnxruntime::SparseTensor::MakeCooData	(	size_t	values_count,
		size_t	index_count
	)

Allocates memory for values and index and returns a mutator so data can be copied into the buffer.

Parameters

values_count	use 0 for fully sparse tensors
index_count	use 0 for fully sparse tensors

Returns

Status onnxruntime::SparseTensor::MakeCooStrings	(	size_t	string_count,
		const char const	strings,
		gsl::span< const int64_t >	indices
	)

The method allocates a single contiguous buffer and creates instances of std::strings in it, with copies of the supplied zero-terminated strings followed by COO indices. All data is assumed to be on CPU and the allocator supplied must be a CPU based allocator.

Parameters

string_count	use 0 for fully sparse tensors
strings	array of char* pointers. use nullptr for fully sparse tensors
indices	span of indices. Use empty span for fully sparse tensors.

Returns: Status

Status onnxruntime::SparseTensor::MakeCsrData	(	const IDataTransfer &	data_transfer,
		const OrtMemoryInfo &	data_location,
		size_t	values_count,
		const void *	values_data,
		gsl::span< const int64_t >	inner_index,
		gsl::span< const int64_t >	outer_index
	)

The function will allocate a single contiguous buffer and will copy values and indices into it.

Parameters

data_transfer
data_location
values_count	use 0 for fully sparse tensors
values_data	pointer to data to be copied. Use nullptr for fully sparse tensors.
inner_index	inner index to be copied. Use empty span for fully sparse tensors.
outer_index	outer index to be copied. Use empty span for fully sparse tensors.

Returns

CsrMutator onnxruntime::SparseTensor::MakeCsrData	(	size_t	values_count,
		size_t	inner_index_count,
		size_t	outer_index_count
	)

Allocates memory for values and index and returns mutator so data can be populated.

Parameters

values_count	Use 0 for fully sparse tensors.
inner_index_count	Use 0 for fully sparse tensors.
outer_index_count	Use 0 for fully sparse tensors.

Returns

Status onnxruntime::SparseTensor::MakeCsrStrings	(	size_t	string_count,
		const char const	strings,
		gsl::span< const int64_t >	inner_index,
		gsl::span< const int64_t >	outer_index
	)

The method allocates a single contiguous buffer and creates instances of std::strings in it, with copies of the supplied zero-terminated strings followed by COO indices. All data is assumed to be on CPU and the allocator supplied must be a CPU based allocator

Parameters

string_count
strings	array of char* pointers
inner_index	inner index to be copied. Use empty span for fully sparse tensors.
outer_index	outer index to be copied. Use empty span for fully sparse tensors.

Returns

size_t onnxruntime::SparseTensor::NumValues ( ) const

inline

Returns: nnz

Definition at line 151 of file sparse_tensor.h.

SparseTensor& onnxruntime::SparseTensor::operator= ( SparseTensor && o )

noexcept

onnxruntime::SparseTensor::ORT_DISALLOW_COPY_AND_ASSIGNMENT ( SparseTensor )

int64_t onnxruntime::SparseTensor::RequiredAllocationSize ( ) const

noexcept

Calculates and returns how much this fully initialized SparseTensor data (would) occupy in a contiguous allocation block, or, in fact, occupies if it owns the buffer.

Returns: required allocation size

Status onnxruntime::SparseTensor::UseBlockSparseIndices	(	const TensorShape &	indices_shape,
		int32_t *	indices_data
	)

Use blocksparse indices contained in the user allocated buffer. The shape of the index must be 2-D and must contain one tuple per each of the value blocks that were supplied to the constructor. The supplied buffer lifespan must eclipse the life of sparse tensor instance.

Parameters

indices_shape	Use {0} for fully sparse tensors.
indices_data	Ptr to user allocated buffer. Use nullptr for fully spare tensors.

Returns

Status onnxruntime::SparseTensor::UseCooIndices ( gsl::span< int64_t > indices )

Uses COO index contained in the user allocated buffer along with the values buffer passed on to the constructor. The buffer is used as is and its lifespan must eclipse the lifespan of the sparse tensor instance. The OrtMemoryInfo (location) of the index is assumed to be the same as values.

The index size must either exactly match the number of values in which case index shape would be 1-D (values_count) or it must be twice the number of values in which case its shape would be 2-D (values_count, 2)

Parameters

indices user allocated buffer span. Use empty span for fully sparse tensors.

Returns: Status

Status onnxruntime::SparseTensor::UseCsrIndices	(	gsl::span< int64_t >	inner_index,
		gsl::span< int64_t >	outer_index
	)

This function will use Csr indices contained within the user allocated buffers. The lifespan of the buffers must eclipse the lifespan of sparse tensor instance.

Parameters

inner_index	User allocated buffer span. use empty span for fully sparse tensors
outer_index	User allocated buffer span. Use empty span for fully sparse tensors

Returns

const Tensor& onnxruntime::SparseTensor::Values ( ) const

inlinenoexcept

Read only accessor to non-zero values

Returns

Definition at line 157 of file sparse_tensor.h.

The documentation for this class was generated from the following file:

onnxruntime/core/framework/sparse_tensor.h

Classes

Public Member Functions

Static Public Member Functions

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation