nvtext::bpe_merge_pairs Struct Reference

The table of merge pairs for the BPE encoder. More...

#include <byte_pair_encoding.hpp>

Public Member Functions
	bpe_merge_pairs (std::unique_ptr< cudf::column > &&input, rmm::cuda_stream_view stream=cudf::get_default_stream(), rmm::mr::device_memory_resource *mr=rmm::mr::get_current_device_resource())
	Construct a new bpe merge pairs object. More...
	bpe_merge_pairs (cudf::strings_column_view const &input, rmm::cuda_stream_view stream=cudf::get_default_stream(), rmm::mr::device_memory_resource *mr=rmm::mr::get_current_device_resource())
	Construct a new bpe merge pairs object. More...

Public Attributes
bpe_merge_pairs_impl *	impl {}
	Implementation of the BPE merge pairs table.

Detailed Description

The table of merge pairs for the BPE encoder.

To create an instance, call nvtext::load_merge_pairs

Definition at line 37 of file byte_pair_encoding.hpp.

Constructor & Destructor Documentation

bpe_merge_pairs() [1/2]

nvtext::bpe_merge_pairs::bpe_merge_pairs	(	std::unique_ptr< cudf::column > &&	input,
		rmm::cuda_stream_view	stream = cudf::get_default_stream(),
		rmm::mr::device_memory_resource *	mr = rmm::mr::get_current_device_resource()
	)

Construct a new bpe merge pairs object.

Parameters

input	The input file containing the BPE merge pairs
stream	CUDA stream used for device memory operations and kernel launches.
mr	Device memory resource used to allocate the device memory

bpe_merge_pairs() [2/2]

nvtext::bpe_merge_pairs::bpe_merge_pairs	(	cudf::strings_column_view const &	input,
		rmm::cuda_stream_view	stream = cudf::get_default_stream(),
		rmm::mr::device_memory_resource *	mr = rmm::mr::get_current_device_resource()
	)

Construct a new bpe merge pairs object.

Parameters

input	The input column of strings
stream	CUDA stream used for device memory operations and kernel launches.
mr	Device memory resource used to allocate the device memory

---

The documentation for this struct was generated from the following file:

• byte_pair_encoding.hpp