On Tue, Jan 16, 2018 at 1:38 PM, Kostya Serebryany <kcc at google.com> wrote:>> I see that master has the same value for kMaxNumChunks, is there >> anything in particular that leads you to think i wouldn't run into the >> same limit? > > > No. It's just that I haven't heard anyone else complain recently. > If you have a reproducer that works on trunk, I'll be happy to look at it. >FWIW, I upgraded to 5.0.1, and we hit the issue again. We do not really have a repro available, in the sense that the box needs to receive traffic for a couple of hours in order for the build to hit the issue, it is however fairly consistent. Is there any instrumentation that we could add in order to understand the issue better, and in the meantime, is it safe to just increase the value for kMaxNumChunks ? Thanks, Frederik
+Aleksey, who has been dealing with the allocator recently. If you have a "((idx)) < ((kMaxNumChunks))" (0x40000, 0x40000) check failure, it means that you've allocated (and did not deallocate) 2^18 large heap regions, each *at least* (2^17+1) bytes. This means, that you have live large heap chunks of 2^35 bytes (or more) in total, which is 32Gb. Does this sound correct? If yes, yea, I guess we need to bump kMaxNumChunks On Wed, Jan 24, 2018 at 11:03 AM, Frederik Deweerdt < frederik.deweerdt at gmail.com> wrote:> On Tue, Jan 16, 2018 at 1:38 PM, Kostya Serebryany <kcc at google.com> wrote: > >> I see that master has the same value for kMaxNumChunks, is there > >> anything in particular that leads you to think i wouldn't run into the > >> same limit? > > > > > > No. It's just that I haven't heard anyone else complain recently. > > If you have a reproducer that works on trunk, I'll be happy to look at > it. > > > FWIW, I upgraded to 5.0.1, and we hit the issue again. We do not > really have a repro available, in the sense that the box needs to > receive traffic for a couple of hours in order for the build to hit > the issue, it is however fairly consistent. Is there any > instrumentation that we could add in order to understand the issue > better, and in the meantime, is it safe to just increase the value for > kMaxNumChunks ? >> > Thanks, > Frederik >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180124/9c3a564b/attachment.html>
On Wed, Jan 24, 2018 at 11:30 AM, Kostya Serebryany <kcc at google.com> wrote:> +Aleksey, who has been dealing with the allocator recently. > > If you have a "((idx)) < ((kMaxNumChunks))" (0x40000, 0x40000) > check failure, it means that you've allocated (and did not deallocate) 2^18 > large heap regions, each *at least* (2^17+1) bytes. > This means, that you have live large heap chunks of 2^35 bytes (or more) in > total, which is 32Gb. > Does this sound correct?Yes, it does. Our software typically allocates several hundred Gb, and the size of the objects depend on the traffic. They would realistically reach 2^17+1 bytes.> If yes, yea, I guess we need to bump kMaxNumChunks > >I'll increase the limit to 2^19 for our build, and I'll report the results here. Thanks, Frederik