Class uri_data_provider

Inheritance Relationships

Base Type

Class Documentation

class ral::io::uri_data_provider : public ral::io::data_provider

can generate a series of randomaccessfiles from uris that are provided when it goes out of scope it will close any files it opened this last point is debatable in terms of if this is the desired functionality

Public Functions

uri_data_provider(std::vector<Uri> uris, std::vector<std::map<std::string, std::string>> uri_values, bool ignore_missing_paths = false)
uri_data_provider(std::vector<Uri> uris, bool ignore_missing_paths = false)
virtual std::shared_ptr<data_provider> clone() override
virtual ~uri_data_provider()
virtual bool has_next()

tells us if there are more files in the list of uris to be provided

virtual void reset()

Resets current_file to 0

virtual data_handle get_next(bool open_file = true)

gets a randomaccessfile to the uri at file_uris[current_file] and advances current_file by 1 open_file = true will actually open the file and return a std::shared_ptr<arrow::io::RandomAccessFile>. If its false it will return a nullptr

virtual std::vector<data_handle> get_some(std::size_t num_files, bool open_file = true)

Tries to get up to num_files data_handles. We use this instead of a get_all() because if there are too many files, trying to get too many file handles will cause a crash. Using get_some() forces breaking up the process of getting file_handles. open_file = true will actually open the file and return a std::shared_ptr<arrow::io::RandomAccessFile>. If its false it will return a nullptr

virtual void close_file_handles()

Closes currently open set of file handles maintained by the provider

virtual size_t get_num_handles()

Get the number of data_handles that will be provided.