Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ntuple] Support all template parameters of STL containers #16754

Open
hahnjo opened this issue Oct 28, 2024 · 5 comments
Open

[ntuple] Support all template parameters of STL containers #16754

hahnjo opened this issue Oct 28, 2024 · 5 comments

Comments

@hahnjo
Copy link
Member

hahnjo commented Oct 28, 2024

As pointed out by ATLAS, it's currently not possible to use a std::vector with a non-default allocator in RNTuple:

#include <ROOT/RNTupleModel.hxx>

#include <scoped_allocator>
#include <vector>

void ntuple_vector_allocator() {
  auto model = ROOT::Experimental::RNTupleModel::CreateBare();
  model->MakeField<std::vector<int, std::scoped_allocator_adaptor<std::allocator<int>>>>("v");
}

leads to

Processing ntuple_vector_allocator.C...
terminate called after throwing an instance of 'ROOT::Experimental::RException'
  what():  vector<int,scoped_allocator_adaptor<allocator<int> > > is not supported
At:
  ROOT::Experimental::RClassField::RClassField(std::string_view, std::string_view, TClass *) [/home/jhahnfel/ROOT/src/tree/ntuple/v7/src/RField.cxx:1807]

Aborted (core dumped)

This is because RField is only partially specialized for std::vector<ItemT>, so a non-default Allocator falls back to the default RField declaration (which assumes it's a class and checks that it's not in std namespace).

Note that in principle, this affects all STL containers. For many of them, there are other template parameters, for example Hash for std::unordered_set:

#include <ROOT/RNTupleModel.hxx>

#include <unordered_set>

struct IntHash : public std::hash<int> {};

void ntuple_unordered_set_hash() {
  auto model = ROOT::Experimental::RNTupleModel::CreateBare();
  model->MakeField<std::unordered_set<int, IntHash>>("s");
}

leads to

Processing ntuple_unordered_set_hash.C...
terminate called after throwing an instance of 'ROOT::Experimental::RException'
  what():  unordered_set<int,IntHash,equal_to<int>,allocator<int> > is not supported
At:
  ROOT::Experimental::RClassField::RClassField(std::string_view, std::string_view, TClass *) [/home/jhahnfel/ROOT/src/tree/ntuple/v7/src/RField.cxx:1807]

Aborted (core dumped)
@hahnjo
Copy link
Member Author

hahnjo commented Oct 28, 2024

As discussed, for std::vector with a non-default allocator we should likely go through the collection proxy support. For maps and sets, we may already support (via the collection proxies) and it's just a matter of relaxing the partial specializations and maybe the logic in RFieldBase::Create.

@dpiparo
Copy link
Member

dpiparo commented Oct 28, 2024

Will the on-disk representation of vectors be the same for all possible allocators? Will it be possible to read back vectors with an allocator which was different than the one used when they were written?

@hahnjo
Copy link
Member Author

hahnjo commented Oct 28, 2024

Will the on-disk representation of vectors be the same for all possible allocators? Will it be possible to read back vectors with an allocator which was different than the one used when they were written?

Very relevant questions; IMHO additional template parameters should not change the on-disk representation of STL containers, and it should be possible to transparently read them back with different parameters than at write time. If nothing is specified and the model is reconstructed, the user gets the default STL container as before. I think this would be compatible with the current state of the specification, otherwise we may need to think this through before declaring 1.0.

@jblomer
Copy link
Contributor

jblomer commented Oct 28, 2024

To be more precise than just a thumbs up: I also think the on-disk representation should not depend on the allocator.

@pcanal
Copy link
Member

pcanal commented Oct 28, 2024

Will the on-disk representation of vectors be the same for all possible allocators? Will it be possible to read back vectors with an allocator which was different than the one used when they were written?

For reference, this is already the case (for both questions, the answer is yes) for TTree (and thus must be the case for RNTuple)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants