Tomasz Kapela via llvm-dev
2017-Jan-16 14:25 UTC
[llvm-dev] [RFC 0/2] Propose a new pointer trait.
Hi, I'm part of an engineering team doing research on persistent memory support and we have stumbled upon an interesting problem. The issue is, we would like to be able to use the standard library containers in a persistent memory context (think NVDIMM-N). What I mean is that you allocate a container from said memory, use it like you normally would. After the application terminates, expectedly or otherwise, you can place the container back into the newly created process's address space - it will be in the last consistent state in which it was before termination (interruption/crash/exit). The obvious way to achieve this is to substitute the allocator. The programming model is based on memory mapped files and because of that the pointer type used by the allocator is a fancy pointer, so that it can do all the offset calculations. User data consistency is provided via a transaction mechanism provided by the library. We snapshot data before modifying them to be able to roll the transaction back in case of an error. The usage model we are proposing is this (pseudocode, not the actual API): int main() { auto handle = pool::open("path/to/file", ...); using pvector = std::vector<foo, persistent_allocator<foo>>; { auto tx = transaction::start(handle); // transactions are per pool file auto vec_ptr = allocate_from_persistent_memory<pvector>(); vec_ptr->emplace_back(foo, parameters); } } The problem occurs when standard library containers have metadata. For example rb tree nodes carry their color, which will not be included in the transaction's undo log. To address this we propose to define a new type in the std::pointer_traits, which could be used to wrap the containers' metadata in a user-defined type. This should be fully backward compatible and as such not mandatory. This is a standard level change and we would like to gather feedback for this idea before we start the whole process. This feature is not persistent memory specific, even we use it as part of providing transactional data consistency. This is in fact a working proof of concept available at (https://github.com/pmem/libcxx) combined with the allocator from (https://github.com/pmem/nvml). I will be happy to answer any questions you might have and await your feedback. Thanks, Tom Tomasz Kapela (2): pm: change deque typedef pm: add persistency_type typedef to pointer_traits include/__hash_table | 20 ++++++++------ include/__tree | 8 +++--- include/deque | 8 +++--- include/list | 7 +++-- include/map | 7 +++-- include/memory | 31 ++++++++++++++++++++++ include/set | 9 +++++-- .../pointer.traits.types/persistency_type.pass.cpp | 25 +++++++++++++++++ 8 files changed, 94 insertions(+), 21 deletions(-) create mode 100644 test/std/utilities/memory/pointer.traits/pointer.traits.types/persistency_type.pass.cpp -- 2.9.3
Tomasz Kapela via llvm-dev
2017-Jan-16 14:25 UTC
[llvm-dev] [RFC 1/2] pm: change deque typedef
Necessary to make the deque understand fancy pointers. Signed-off-by: Tomasz Kapela <tomasz.kapela at intel.com> --- include/deque | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/include/deque b/include/deque index 0454162..431a50d 100644 --- a/include/deque +++ b/include/deque @@ -935,8 +935,7 @@ protected: typedef typename __rebind_alloc_helper<__alloc_traits, pointer>::type __pointer_allocator; typedef allocator_traits<__pointer_allocator> __map_traits; typedef typename __map_traits::pointer __map_pointer; - typedef typename __rebind_alloc_helper<__alloc_traits, const_pointer>::type __const_pointer_allocator; - typedef typename allocator_traits<__const_pointer_allocator>::const_pointer __map_const_pointer; + typedef typename allocator_traits<__pointer_allocator>::const_pointer __map_const_pointer; typedef __split_buffer<pointer, __pointer_allocator> __map; typedef __deque_iterator<value_type, pointer, reference, __map_pointer, -- 2.9.3
Tomasz Kapela via llvm-dev
2017-Jan-16 14:25 UTC
[llvm-dev] [RFC 2/2] pm: add persistency_type typedef to pointer_traits
The typedef by default binds to value_type, and to pointer_traits::persistency_type if available. Signed-off-by: Tomasz Kapela <tomasz.kapela at intel.com> --- include/__hash_table | 20 ++++++++------ include/__tree | 8 +++--- include/deque | 5 ++-- include/list | 7 +++-- include/map | 7 +++-- include/memory | 31 ++++++++++++++++++++++ include/set | 9 +++++-- .../pointer.traits.types/persistency_type.pass.cpp | 25 +++++++++++++++++ 8 files changed, 93 insertions(+), 19 deletions(-) create mode 100644 test/std/utilities/memory/pointer.traits/pointer.traits.types/persistency_type.pass.cpp diff --git a/include/__hash_table b/include/__hash_table index 6d6a6a1..5025191 100644 --- a/include/__hash_table +++ b/include/__hash_table @@ -112,8 +112,9 @@ struct __hash_node > { typedef _Tp __node_value_type; + typedef typename __rebind_persistency_type<_VoidPtr, size_t>::type size_type; - size_t __hash_; + size_type __hash_; __node_value_type __value_; }; @@ -782,12 +783,12 @@ class __bucket_list_deallocator { typedef _Alloc allocator_type; typedef allocator_traits<allocator_type> __alloc_traits; - typedef typename __alloc_traits::size_type size_type; - - __compressed_pair<size_type, allocator_type> __data_; public: typedef typename __alloc_traits::pointer pointer; - +private: + typedef typename __rebind_persistency_type<pointer, typename __alloc_traits::size_type>::type size_type; + __compressed_pair<size_type, allocator_type> __data_; +public: _LIBCPP_INLINE_VISIBILITY __bucket_list_deallocator() _NOEXCEPT_(is_nothrow_default_constructible<allocator_type>::value) @@ -913,11 +914,14 @@ public: typedef typename __alloc_traits::pointer pointer; typedef typename __alloc_traits::const_pointer const_pointer; #ifndef _LIBCPP_ABI_FIX_UNORDERED_CONTAINER_SIZE_TYPE - typedef typename __alloc_traits::size_type size_type; + typedef typename __alloc_traits::size_type raw_size_type; #else - typedef typename _NodeTypes::size_type size_type; + typedef typename _NodeTypes::size_type raw_size_type; #endif - typedef typename _NodeTypes::difference_type difference_type; + typedef typename _NodeTypes::difference_type raw_difference_type; + + typedef typename __rebind_persistency_type<pointer, raw_difference_type>::type difference_type; + typedef typename __rebind_persistency_type<pointer, raw_size_type>::type size_type; public: // Create __node diff --git a/include/__tree b/include/__tree index 170d4c0..e97dc4d 100644 --- a/include/__tree +++ b/include/__tree @@ -730,9 +730,11 @@ public: typedef typename _NodeBaseTypes::__node_base_pointer pointer; typedef typename _NodeBaseTypes::__parent_pointer __parent_pointer; + typedef typename __rebind_persistency_type<pointer, bool>::type bool_type; + pointer __right_; __parent_pointer __parent_; - bool __is_black_; + bool_type __is_black_; _LIBCPP_INLINE_VISIBILITY pointer __parent_unsafe() const { return static_cast<pointer>(__parent_);} @@ -998,8 +1000,8 @@ public: typedef typename __alloc_traits::pointer pointer; typedef typename __alloc_traits::const_pointer const_pointer; - typedef typename __alloc_traits::size_type size_type; - typedef typename __alloc_traits::difference_type difference_type; + typedef typename __rebind_persistency_type<pointer, typename __alloc_traits::difference_type>::type difference_type; + typedef typename __rebind_persistency_type<pointer, typename __alloc_traits::size_type>::type size_type; public: typedef typename _NodeTypes::__void_pointer __void_pointer; diff --git a/include/deque b/include/deque index 431a50d..1354ca8 100644 --- a/include/deque +++ b/include/deque @@ -925,10 +925,11 @@ protected: typedef allocator_traits<allocator_type> __alloc_traits; typedef value_type& reference; typedef const value_type& const_reference; - typedef typename __alloc_traits::size_type size_type; - typedef typename __alloc_traits::difference_type difference_type; typedef typename __alloc_traits::pointer pointer; typedef typename __alloc_traits::const_pointer const_pointer; + typedef typename __alloc_traits::size_type raw_size_type; + typedef typename __rebind_persistency_type<pointer, raw_size_type>::type size_type; + typedef typename __alloc_traits::difference_type difference_type; static const difference_type __block_size; diff --git a/include/list b/include/list index 76c50a1..0c46987 100644 --- a/include/list +++ b/include/list @@ -529,7 +529,6 @@ protected: typedef _Tp value_type; typedef _Alloc allocator_type; typedef allocator_traits<allocator_type> __alloc_traits; - typedef typename __alloc_traits::size_type size_type; typedef typename __alloc_traits::void_pointer __void_pointer; typedef __list_iterator<value_type, __void_pointer> iterator; typedef __list_const_iterator<value_type, __void_pointer> const_iterator; @@ -544,11 +543,15 @@ protected: typedef __link_pointer __link_const_pointer; typedef typename __alloc_traits::pointer pointer; typedef typename __alloc_traits::const_pointer const_pointer; - typedef typename __alloc_traits::difference_type difference_type; typedef typename __rebind_alloc_helper<__alloc_traits, __node_base>::type __node_base_allocator; typedef typename allocator_traits<__node_base_allocator>::pointer __node_base_pointer; + typedef typename __alloc_traits::size_type raw_size_type; + typedef typename __alloc_traits::difference_type raw_difference_type; + typedef typename __rebind_persistency_type<pointer, raw_size_type>::type size_type; + typedef typename __rebind_persistency_type<pointer, raw_difference_type>::type difference_type; + __node_base __end_; __compressed_pair<size_type, __node_allocator> __size_alloc_; diff --git a/include/map b/include/map index ecd9d92..ed6eb92 100644 --- a/include/map +++ b/include/map @@ -847,13 +847,16 @@ private: public: typedef typename __alloc_traits::pointer pointer; typedef typename __alloc_traits::const_pointer const_pointer; - typedef typename __alloc_traits::size_type size_type; - typedef typename __alloc_traits::difference_type difference_type; typedef __map_iterator<typename __base::iterator> iterator; typedef __map_const_iterator<typename __base::const_iterator> const_iterator; typedef _VSTD::reverse_iterator<iterator> reverse_iterator; typedef _VSTD::reverse_iterator<const_iterator> const_reverse_iterator; + typedef typename __alloc_traits::size_type raw_size_type; + typedef typename __alloc_traits::difference_type raw_difference_type; + typedef typename __rebind_persistency_type<pointer, raw_size_type>::type size_type; + typedef typename __rebind_persistency_type<pointer, raw_difference_type>::type difference_type; + _LIBCPP_INLINE_VISIBILITY map() _NOEXCEPT_( diff --git a/include/memory b/include/memory index 638999e..a06439d 100644 --- a/include/memory +++ b/include/memory @@ -814,6 +814,29 @@ struct __pointer_traits_difference_type<_Ptr, true> typedef typename _Ptr::difference_type type; }; +template <class _Ptr> +struct __has_persistency_type +{ +private: + struct __two {char __lx; char __lxx;}; + template <class _Up> static __two __test(...); + template <class _Up> static char __test(typename _Up::persistency_type* = 0); +public: + static const bool value = sizeof(__test<_Ptr>(0)) == 1; +}; + +template <class _Tp, class _Ptr, bool = __has_persistency_type<_Ptr>::value> +struct __pointer_traits_persistency_type +{ + typedef _Tp type; +}; + +template <class _Tp, class _Ptr> +struct __pointer_traits_persistency_type<_Tp, _Ptr, true> +{ + typedef typename _Ptr::persistency_type type; +}; + template <class _Tp, class _Up> struct __has_rebind { @@ -931,6 +954,7 @@ struct _LIBCPP_TEMPLATE_VIS pointer_traits typedef _Ptr pointer; typedef typename __pointer_traits_element_type<pointer>::type element_type; typedef typename __pointer_traits_difference_type<pointer>::type difference_type; + typedef typename __pointer_traits_persistency_type<element_type, pointer>::type persistency_type; #ifndef _LIBCPP_CXX03_LANG template <class _Up> using rebind = typename __pointer_traits_rebind<pointer, _Up>::type; @@ -954,6 +978,7 @@ struct _LIBCPP_TEMPLATE_VIS pointer_traits<_Tp*> typedef _Tp* pointer; typedef _Tp element_type; typedef ptrdiff_t difference_type; + typedef _Tp persistency_type; #ifndef _LIBCPP_CXX03_LANG template <class _Up> using rebind = _Up*; @@ -979,6 +1004,11 @@ struct __rebind_pointer { #endif }; +template<class _Ptr, class _Tp> +struct __rebind_persistency_type { + typedef typename pointer_traits<typename __rebind_pointer<_Ptr, _Tp>::type>::persistency_type type; +}; + // allocator_traits namespace __has_pointer_type_imp @@ -1844,6 +1874,7 @@ class _LIBCPP_TEMPLATE_VIS allocator<const _Tp> public: typedef size_t size_type; typedef ptrdiff_t difference_type; + typedef const _Tp persistency_type; typedef const _Tp* pointer; typedef const _Tp* const_pointer; typedef const _Tp& reference; diff --git a/include/set b/include/set index be5c952..62b42cd 100644 --- a/include/set +++ b/include/set @@ -422,8 +422,13 @@ private: public: typedef typename __base::pointer pointer; typedef typename __base::const_pointer const_pointer; - typedef typename __base::size_type size_type; - typedef typename __base::difference_type difference_type; + + typedef typename __alloc_traits::size_type raw_size_type; + typedef typename __alloc_traits::difference_type raw_difference_type; + + typedef typename __rebind_persistency_type<pointer, raw_size_type>::type size_type; + typedef typename __rebind_persistency_type<pointer, raw_difference_type>::type difference_type; + typedef typename __base::const_iterator iterator; typedef typename __base::const_iterator const_iterator; typedef _VSTD::reverse_iterator<iterator> reverse_iterator; diff --git a/test/std/utilities/memory/pointer.traits/pointer.traits.types/persistency_type.pass.cpp b/test/std/utilities/memory/pointer.traits/pointer.traits.types/persistency_type.pass.cpp new file mode 100644 index 0000000..67756f7 --- /dev/null +++ b/test/std/utilities/memory/pointer.traits/pointer.traits.types/persistency_type.pass.cpp @@ -0,0 +1,25 @@ +//===----------------------------------------------------------------------===// +// +// The LLVM Compiler Infrastructure +// +// This file is dual licensed under the MIT and the University of Illinois Open +// Source Licenses. See LICENSE.TXT for details. +// + +#include <memory> +#include <type_traits> + +template<typename T> +class wrapper { +}; + +template<typename T> +class custom_ptr { +public: + using persistency_type = wrapper<T>; +}; + +int main() { + static_assert((std::is_same<std::pointer_traits<custom_ptr<char>>::persistency_type, wrapper<char>>::value), ""); + static_assert((std::is_same<std::pointer_traits<char*>::persistency_type, char>::value), ""); +} -- 2.9.3
Eric Fiselier via llvm-dev
2017-Jan-17 02:45 UTC
[llvm-dev] [RFC 0/2] Propose a new pointer trait.
A couple of comments/questions: 1) I don't see why `persistent_type` is tied to pointer_traits at all? Wouldn't it work the same as a separate trait? 2) Given an arbitrary `T` what are the requirements on the "persistent type for T" (e.g. `pointer_traits<T*>::persistent_type`)? Is it intended to be usable exactly like `T` itself? If so how do you plan to achieve this and do you have an example? If not how are implementations supposed to access the actual underlying object? 3) Changing the types of internal node data members is somewhat problematic, at least in terms of implementation. __tree and __hash_table don't actually invoke the constructors for the node types, and instead only construct the "value" member directly using `allocator_traits::construct` as required by [container.requirements.general]. If the "persistent" types have non-trivial constructors or destructors they won't be called leading to UB. This isn't an impossible problem to solve, but it would take a lot of work. /Eric PS. Email regarding libc++ should be sent to cfe-dev as well. On Mon, Jan 16, 2017 at 7:25 AM, Tomasz Kapela via llvm-dev < llvm-dev at lists.llvm.org> wrote:> Hi, > I'm part of an engineering team doing research on persistent memory > support and > we have stumbled upon an interesting problem. The issue is, we would like > to be > able to use the standard library containers in a persistent memory context > (think NVDIMM-N). What I mean is that you allocate a container from said > memory, use it like you normally would. After the application terminates, > expectedly or otherwise, you can place the container back into the newly > created process's address space - it will be in the last consistent state > in > which it was before termination (interruption/crash/exit). The obvious way > to > achieve this is to substitute the allocator. The programming model is > based on > memory mapped files and because of that the pointer type used by the > allocator > is a fancy pointer, so that it can do all the offset calculations. > > User data consistency is provided via a transaction mechanism provided by > the > library. We snapshot data before modifying them to be able to roll the > transaction back in case of an error. The usage model we are proposing is > this > (pseudocode, not the actual API): > > int main() { > auto handle = pool::open("path/to/file", ...); > using pvector = std::vector<foo, persistent_allocator<foo>>; > { > auto tx = transaction::start(handle); // transactions are per pool > file > auto vec_ptr = allocate_from_persistent_memory<pvector>(); > vec_ptr->emplace_back(foo, parameters); > } > } > > The problem occurs when standard library containers have metadata. For > example > rb tree nodes carry their color, which will not be included in the > transaction's undo log. To address this we propose to define a new type in > the > std::pointer_traits, which could be used to wrap the containers' metadata > in > a user-defined type. This should be fully backward compatible and as such > not > mandatory. > > This is a standard level change and we would like to gather feedback for > this > idea before we start the whole process. This feature is not persistent > memory > specific, even we use it as part of providing transactional data > consistency. > > This is in fact a working proof of concept available at > (https://github.com/pmem/libcxx) combined with the allocator from > (https://github.com/pmem/nvml). > > I will be happy to answer any questions you might have and await your > feedback. > > Thanks, > Tom > > > Tomasz Kapela (2): > pm: change deque typedef > pm: add persistency_type typedef to pointer_traits > > include/__hash_table | 20 ++++++++------ > include/__tree | 8 +++--- > include/deque | 8 +++--- > include/list | 7 +++-- > include/map | 7 +++-- > include/memory | 31 > ++++++++++++++++++++++ > include/set | 9 +++++-- > .../pointer.traits.types/persistency_type.pass.cpp | 25 +++++++++++++++++ > 8 files changed, 94 insertions(+), 21 deletions(-) > create mode 100644 test/std/utilities/memory/ > pointer.traits/pointer.traits.types/persistency_type.pass.cpp > > -- > 2.9.3 > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170116/107e1f52/attachment.html>
Tomasz Kapela via llvm-dev
2017-Jan-17 18:49 UTC
[llvm-dev] [RFC 0/2] Propose a new pointer trait.
Hi Eric, thanks for the swift reply. As for your questions: ad.1) In fact it doesn't have to be tied to pointer_traits. Initially i had it in the allocator_traits, because that made more sense. The implementation of __tree in the __tree_node_base doesn't know anything about the allocator. The only thing it has is the _VoidPtr template parameter, which is a rebound allocator_traits::pointer. I wanted to have a working implementation without rewriting the whole library. I believe the most suitable solution would be to add something like memory_traits from which you could retrieve the persistency_type (which probably isn't the best name), pointer and maybe even the allocator. But that's a massively invasive change. ad.2) I believe that externally it should behave like T. In my implementation it falls back to a T for a T* pointer. Additionally it should be default constructible, have an implicit converting constructor, should be castable to T and assignable from anything convertible to T. In fact in our github organization we have a working implementation of both libcxx (https://github.com/pmem/libcxx), a complementary allocator with a custom fancy pointer and a template class p<> to be used with the persistency_type (https://github.com/pmem/nvml/tree/master/src/include/libpmemobj%2B%2B). ad.3) Unfortunately this whole approach does have caveats. This is one of them. All of the member data that I have seen in the standard library's containers are fundamental types. Therefore I think any persistency_type wrapper should be default constructible and trivially destructible. There is one other issue, since the member access operator cannot be overloaded, they cannot be class types. There are probably other issues I'm not seeing right now. That is why I'm consulting you as standard library experts. I hope this answers you questions and comments. Looking forward to more feedback. Tom On 01/16, Eric Fiselier wrote:> A couple of comments/questions: > > 1) I don't see why `persistent_type` is tied to pointer_traits at all? > Wouldn't it work the same as a separate trait? > > 2) Given an arbitrary `T` what are the requirements on the "persistent type > for T" (e.g. `pointer_traits<T*>::persistent_type`)? > Is it intended to be usable exactly like `T` itself? If so how do you plan > to achieve this and do you have an example? > If not how are implementations supposed to access the actual underlying > object? > > 3) Changing the types of internal node data members is somewhat > problematic, at least in terms of implementation. > __tree and __hash_table don't actually invoke the constructors for the node > types, and instead only construct the > "value" member directly using `allocator_traits::construct` as required by > [container.requirements.general]. > If the "persistent" types have non-trivial constructors or destructors they > won't be called leading to UB. This isn't > an impossible problem to solve, but it would take a lot of work. > > /Eric > > PS. Email regarding libc++ should be sent to cfe-dev as well. > > On Mon, Jan 16, 2017 at 7:25 AM, Tomasz Kapela via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > > > Hi, > > I'm part of an engineering team doing research on persistent memory > > support and > > we have stumbled upon an interesting problem. The issue is, we would like > > to be > > able to use the standard library containers in a persistent memory context > > (think NVDIMM-N). What I mean is that you allocate a container from said > > memory, use it like you normally would. After the application terminates, > > expectedly or otherwise, you can place the container back into the newly > > created process's address space - it will be in the last consistent state > > in > > which it was before termination (interruption/crash/exit). The obvious way > > to > > achieve this is to substitute the allocator. The programming model is > > based on > > memory mapped files and because of that the pointer type used by the > > allocator > > is a fancy pointer, so that it can do all the offset calculations. > > > > User data consistency is provided via a transaction mechanism provided by > > the > > library. We snapshot data before modifying them to be able to roll the > > transaction back in case of an error. The usage model we are proposing is > > this > > (pseudocode, not the actual API): > > > > int main() { > > auto handle = pool::open("path/to/file", ...); > > using pvector = std::vector<foo, persistent_allocator<foo>>; > > { > > auto tx = transaction::start(handle); // transactions are per pool > > file > > auto vec_ptr = allocate_from_persistent_memory<pvector>(); > > vec_ptr->emplace_back(foo, parameters); > > } > > } > > > > The problem occurs when standard library containers have metadata. For > > example > > rb tree nodes carry their color, which will not be included in the > > transaction's undo log. To address this we propose to define a new type in > > the > > std::pointer_traits, which could be used to wrap the containers' metadata > > in > > a user-defined type. This should be fully backward compatible and as such > > not > > mandatory. > > > > This is a standard level change and we would like to gather feedback for > > this > > idea before we start the whole process. This feature is not persistent > > memory > > specific, even we use it as part of providing transactional data > > consistency. > > > > This is in fact a working proof of concept available at > > (https://github.com/pmem/libcxx) combined with the allocator from > > (https://github.com/pmem/nvml). > > > > I will be happy to answer any questions you might have and await your > > feedback. > > > > Thanks, > > Tom > > > > > > Tomasz Kapela (2): > > pm: change deque typedef > > pm: add persistency_type typedef to pointer_traits > > > > include/__hash_table | 20 ++++++++------ > > include/__tree | 8 +++--- > > include/deque | 8 +++--- > > include/list | 7 +++-- > > include/map | 7 +++-- > > include/memory | 31 > > ++++++++++++++++++++++ > > include/set | 9 +++++-- > > .../pointer.traits.types/persistency_type.pass.cpp | 25 +++++++++++++++++ > > 8 files changed, 94 insertions(+), 21 deletions(-) > > create mode 100644 test/std/utilities/memory/ > > pointer.traits/pointer.traits.types/persistency_type.pass.cpp > > > > -- > > 2.9.3 > > > > _______________________________________________ > > LLVM Developers mailing list > > llvm-dev at lists.llvm.org > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >