If your system does not support fallocate(2), we use ftruncate(2) to create an output file. fallocate(2) succeeds even if your disk have less space than the requested size, because it creates a sparse file. If you mmap such sparse file, you'll receive a SIGBUS when the disk actually becomes full. So, lld can die suddenly with SIGBUS when your disk becomes full, and currently we are not doing anything about it. It's sometimes hard to notice that that was caused by the lack of disk space. I wonder if we should print out a hint (e.g. "Bus error -- disk full?") when we receive a SIGBUS. Any opinions? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171023/e50ba4d0/attachment.html>
---- On Mon, 23 Oct 2017 15:21:25 -0700 Rui Ueyama via llvm-dev <llvm-dev at lists.llvm.org> wrote ---- I wonder if we should print out a hint (e.g. "Bus error -- disk full?") when we receive a SIGBUS. Any opinions? Sounds like a good idea to me. Please also include "SIGBUS" in the message. The SIGBUS signal can have various causes, but the disk full cause is both non-obvious and probably common, so the hint seems useful. -- Bob -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171023/14d0f99d/attachment.html>
On Mon, Oct 23, 2017 at 5:21 PM, Rui Ueyama via llvm-dev < llvm-dev at lists.llvm.org> wrote:> If your system does not support fallocate(2), we use ftruncate(2) to > create an output file. fallocate(2) succeeds even if your disk have less > space than the requested size, because it creates a sparse file. If you > mmap such sparse file, you'll receive a SIGBUS when the disk actually > becomes full. > > So, lld can die suddenly with SIGBUS when your disk becomes full, and > currently we are not doing anything about it. It's sometimes hard to notice > that that was caused by the lack of disk space. > > I wonder if we should print out a hint (e.g. "Bus error -- disk full?") > when we receive a SIGBUS. Any opinions? > >What about reading back the file's size with stat() before mapping? .st_blocks should give the "real" size. BTW, posix_fallocate() might provide better portability and decrease the likelihood of falling back on ftruncate().> _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >-- -Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171023/ee6554d5/attachment.html>
On Mon, Oct 23, 2017 at 3:40 PM, Brian Cain <brian.cain at gmail.com> wrote:> > > On Mon, Oct 23, 2017 at 5:21 PM, Rui Ueyama via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> If your system does not support fallocate(2), we use ftruncate(2) to >> create an output file. fallocate(2) succeeds even if your disk have less >> space than the requested size, because it creates a sparse file. If you >> mmap such sparse file, you'll receive a SIGBUS when the disk actually >> becomes full. >> >> So, lld can die suddenly with SIGBUS when your disk becomes full, and >> currently we are not doing anything about it. It's sometimes hard to notice >> that that was caused by the lack of disk space. >> >> I wonder if we should print out a hint (e.g. "Bus error -- disk full?") >> when we receive a SIGBUS. Any opinions? >> >> > What about reading back the file's size with stat() before mapping? > .st_blocks should give the "real" size. >Creating a sparse file is not an error, and a sparse file works fine as long as your disk has enough space (which is almost always true.) So I don't know how stat helps.> BTW, posix_fallocate() might provide better portability and decrease the > likelihood of falling back on ftruncate(). >Yes, but we want to avoid that function because when it falls back, it is very slow. What it does when fallocate(2) is not available is to actually write 0 to every block to make sure that all disk blocks are allocated. Doing it every time the linker creates a new file is a bit overkill to guard against disk full situation which isn't common.> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> > > > -- > -Brian >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171023/ea65dfca/attachment.html>
For Zig we use LLD as a library. So for us it would be better to avoid global state such as SIGBUS (or any other signal handlers), instead returning an error from the link function when linking fails. If lld can encapsulate this signal handling and prevent the application using lld from getting the signal directly, instead carefully handling the signal in LLD itself and translating it into a proper error code or message, this would be reasonable. On Mon, Oct 23, 2017 at 6:21 PM, Rui Ueyama via llvm-dev < llvm-dev at lists.llvm.org> wrote:> If your system does not support fallocate(2), we use ftruncate(2) to > create an output file. fallocate(2) succeeds even if your disk have less > space than the requested size, because it creates a sparse file. If you > mmap such sparse file, you'll receive a SIGBUS when the disk actually > becomes full. > > So, lld can die suddenly with SIGBUS when your disk becomes full, and > currently we are not doing anything about it. It's sometimes hard to notice > that that was caused by the lack of disk space. > > I wonder if we should print out a hint (e.g. "Bus error -- disk full?") > when we receive a SIGBUS. Any opinions? > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171023/26dc43a6/attachment.html>
On Mon, Oct 23, 2017 at 5:28 PM, Andrew Kelley <superjoe30 at gmail.com> wrote:> For Zig we use LLD as a library. So for us it would be better to avoid > global state such as SIGBUS (or any other signal handlers), instead > returning an error from the link function when linking fails. If lld can > encapsulate this signal handling and prevent the application using lld from > getting the signal directly, instead carefully handling the signal in LLD > itself and translating it into a proper error code or message, this would > be reasonable. >Signal handlers and signal masks are inherently process-wide, so there's no way to encapsulate them to lld functions. So my plan is to change the name of lld::{coff,elf}::link's `ExitEarly` parameter to `IsStandalone`, and we (not only call exit(2) but also) set a signal handler only when the argument is true. Since library users pass false to the parameter, it shouldn't change the behavior of lld for the library use case. On Mon, Oct 23, 2017 at 6:21 PM, Rui Ueyama via llvm-dev <> llvm-dev at lists.llvm.org> wrote: > >> If your system does not support fallocate(2), we use ftruncate(2) to >> create an output file. fallocate(2) succeeds even if your disk have less >> space than the requested size, because it creates a sparse file. If you >> mmap such sparse file, you'll receive a SIGBUS when the disk actually >> becomes full. >> >> So, lld can die suddenly with SIGBUS when your disk becomes full, and >> currently we are not doing anything about it. It's sometimes hard to notice >> that that was caused by the lack of disk space. >> >> I wonder if we should print out a hint (e.g. "Bus error -- disk full?") >> when we receive a SIGBUS. Any opinions? >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171023/516c1428/attachment.html>
> Date: Mon, 23 Oct 2017 15:21:25 -0700 > From: Rui Ueyama via llvm-dev <llvm-dev at lists.llvm.org> > > If your system does not support fallocate(2), we use ftruncate(2) to create > an output file. fallocate(2) succeeds even if your disk have less space > than the requested size, because it creates a sparse file. If you mmap such > sparse file, you'll receive a SIGBUS when the disk actually becomes full. > > So, lld can die suddenly with SIGBUS when your disk becomes full, and > currently we are not doing anything about it. It's sometimes hard to notice > that that was caused by the lack of disk space. > > I wonder if we should print out a hint (e.g. "Bus error -- disk full?") > when we receive a SIGBUS. Any opinions?I'm not a huge fan of catching "fatal" signals like this. It tends to make debugging more difficult as you don't get a core dump anymore. And since SIGBUS is also generated for unaligned access that is somewhat annoying. If you go this route, please realize that: * Some systems actually generate SIGSEGV in this scenario. * You can only call functions that are async-signal safe. Hight-level output functions (stdio, iostream) are almost certainly not async-signal safe. * You may be able to distinguish a SIGBUS caused by unaligned access from the "disk-full" case by looking at siginfo, but beware of portability issues there. * You probably only want to install the handler on systems that lack fallocate(2). I think you present a compelling reason to implement fallocate(2) on OpenBSD. Cheers, Mark
Rafael Avila de Espindola via llvm-dev
2017-Oct-30 21:03 UTC
[llvm-dev] lld: sigbus error handling
Rui Ueyama via llvm-dev <llvm-dev at lists.llvm.org> writes:> If your system does not support fallocate(2), we use ftruncate(2) to create > an output file. fallocate(2) succeeds even if your disk have less space > than the requested size, because it creates a sparse file. If you mmap such > sparse file, you'll receive a SIGBUS when the disk actually becomes full. > > So, lld can die suddenly with SIGBUS when your disk becomes full, and > currently we are not doing anything about it. It's sometimes hard to notice > that that was caused by the lack of disk space. > > I wonder if we should print out a hint (e.g. "Bus error -- disk full?") > when we receive a SIGBUS. Any opinions?I think we should change the llvm implementation of resize_file to fail if it cannot allocate the space. That is, it should only use ftruncate on OS X where apparently HFS allocates space with it. If resize_file fails than lld can fail gracefully or use annonymous memory and a plain write instead of mmap for producing the output. Cheers, Rafael
But that would disable mmap IO on systems that don't support fallocate. I'm not sure if OpenBSD people are for example happy about that. On Mon, Oct 30, 2017 at 2:03 PM, Rafael Avila de Espindola < rafael.espindola at gmail.com> wrote:> Rui Ueyama via llvm-dev <llvm-dev at lists.llvm.org> writes: > > > If your system does not support fallocate(2), we use ftruncate(2) to > create > > an output file. fallocate(2) succeeds even if your disk have less space > > than the requested size, because it creates a sparse file. If you mmap > such > > sparse file, you'll receive a SIGBUS when the disk actually becomes full. > > > > So, lld can die suddenly with SIGBUS when your disk becomes full, and > > currently we are not doing anything about it. It's sometimes hard to > notice > > that that was caused by the lack of disk space. > > > > I wonder if we should print out a hint (e.g. "Bus error -- disk full?") > > when we receive a SIGBUS. Any opinions? > > I think we should change the llvm implementation of resize_file to fail > if it cannot allocate the space. That is, it should only use ftruncate > on OS X where apparently HFS allocates space with it. > > If resize_file fails than lld can fail gracefully or use annonymous > memory and a plain write instead of mmap for producing the output. > > Cheers, > Rafael >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171030/4c21b6e2/attachment.html>