Hi, We have a whole bunch of readers(we would have some more too), and was thinking if we should have a vector of Readers, and have a function isMyFormat in each of them. Any reader that knows to handle, goes ahead and parses the file. On a side note, we currently use .objtxt as an figure out if the file is a YAML file or not. I have added FIXME's in the code, if we could some kind of magic (or) a better way to figure out if the file is YAML ? Thanks Shankar Easwaran -- Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by the Linux Foundation
I guess in each isMyFormat(), you would check the given file's magic using llvm::sys::fs::identify_magic(), and then check if it's a known value for that reader. That would be repeated in each isMyFormat(), which is not very good. I'd do that using a mapping from file magic to reader. I mean, we could call identify_magic() at some central place, look up the mapping, and then dispatch. On Wed, Oct 9, 2013 at 11:23 AM, Shankar Easwaran <shankare at codeaurora.org>wrote:> Hi, > > We have a whole bunch of readers(we would have some more too), and was > thinking if we should have a vector of Readers, and have a function > isMyFormat in each of them. > > Any reader that knows to handle, goes ahead and parses the file. > > On a side note, we currently use .objtxt as an figure out if the file is a > YAML file or not. I have added FIXME's in the code, if we could some kind > of magic (or) a better way to figure out if the file is YAML ? > > Thanks > > Shankar Easwaran > > -- > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted > by the Linux Foundation > > ______________________________**_________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/**mailman/listinfo/llvmdev<http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131009/cf3a7565/attachment.html>
All files cannot be identified with a magic though. For example Linker scripts, and currently YAML files. On 10/9/2013 1:36 PM, Rui Ueyama wrote:> I guess in each isMyFormat(), you would check the given file's magic using > llvm::sys::fs::identify_magic(), and then check if it's a known value for > that reader. That would be repeated in each isMyFormat(), which is not very > good. > > I'd do that using a mapping from file magic to reader. I mean, we could > call identify_magic() at some central place, look up the mapping, and then > dispatch. > > On Wed, Oct 9, 2013 at 11:23 AM, Shankar Easwaran > <shankare at codeaurora.org>wrote: > >> Hi, >> >> We have a whole bunch of readers(we would have some more too), and was >> thinking if we should have a vector of Readers, and have a function >> isMyFormat in each of them. >> >> Any reader that knows to handle, goes ahead and parses the file. >> >> On a side note, we currently use .objtxt as an figure out if the file is a >> YAML file or not. I have added FIXME's in the code, if we could some kind >> of magic (or) a better way to figure out if the file is YAML ? >> >> Thanks >> >> Shankar Easwaran >> >> -- >> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted >> by the Linux Foundation >> >> ______________________________**_________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/**mailman/listinfo/llvmdev<http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev> >>-- Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by the Linux Foundation
On Oct 9, 2013, at 11:23 AM, Shankar Easwaran <shankare at codeaurora.org> wrote:> We have a whole bunch of readers(we would have some more too), and was thinking if we should have a vector of Readers, and have a function isMyFormat in each of them. > > Any reader that knows to handle, goes ahead and parses the file. > > On a side note, we currently use .objtxt as an figure out if the file is a YAML file or not. I have added FIXME's in the code, if we could some kind of magic (or) a better way to figure out if the file is YAML ?On this topic, we should come up with standard file extension names. I made up .objtxt for atoms-in-yaml when writing the first test cases. We will soon need extensions for other kinds of yaml files (such as mach-o in yaml). With linker scripts we are stuck with there being no magic at the start and no standard file extension. For new yaml files that we are inventing we should define a standard file extension. I agree with Rui that we should not be calling identify_magic() in every reader. Another approach is to just call each reader to try to parse until one succeeds. The first reader should be the native reader (e.g. ELF or mach-o), so that in the real world there is no time wasted looking at test case formats. -Nick
On 10/9/2013 3:09 PM, Nick Kledzik wrote:> On Oct 9, 2013, at 11:23 AM, Shankar Easwaran <shankare at codeaurora.org> wrote: >> We have a whole bunch of readers(we would have some more too), and was thinking if we should have a vector of Readers, and have a function isMyFormat in each of them. >> >> Any reader that knows to handle, goes ahead and parses the file. >> >> On a side note, we currently use .objtxt as an figure out if the file is a YAML file or not. I have added FIXME's in the code, if we could some kind of magic (or) a better way to figure out if the file is YAML ? > On this topic, we should come up with standard file extension names. I made up .objtxt for atoms-in-yaml when writing the first test cases. We will soon need extensions for other kinds of yaml files (such as mach-o in yaml). With linker scripts we are stuck with there being no magic at the start and no standard file extension. For new yaml files that we are inventing we should define a standard file extension.Isnt having a YAML file starting with the below better, so that you dont need to go through file extensions. magic : arch: You would also be able to figure out if the yaml file is a valid input for the flavor/target too.> I agree with Rui that we should not be calling identify_magic() in every reader. Another approach is to just call each reader to try to parse until one succeeds. The first reader should be the native reader (e.g. ELF or mach-o), so that in the real world there is no time wasted looking at test case formats.Agree. This is better. Thanks Shankar Easwaran
On Wed, Oct 9, 2013 at 4:09 PM, Nick Kledzik <kledzik at apple.com> wrote:> On Oct 9, 2013, at 11:23 AM, Shankar Easwaran <shankare at codeaurora.org> > wrote: > > We have a whole bunch of readers(we would have some more too), and was > thinking if we should have a vector of Readers, and have a function > isMyFormat in each of them. > > > > Any reader that knows to handle, goes ahead and parses the file. > > > > On a side note, we currently use .objtxt as an figure out if the file is > a YAML file or not. I have added FIXME's in the code, if we could some kind > of magic (or) a better way to figure out if the file is YAML ? > > On this topic, we should come up with standard file extension names. I > made up .objtxt for atoms-in-yaml when writing the first test cases. We > will soon need extensions for other kinds of yaml files (such as mach-o in > yaml). With linker scripts we are stuck with there being no magic at the > start and no standard file extension. For new yaml files that we are > inventing we should define a standard file extension. >There are a handful of common ones. Checking if the name contains `.lds` or `.ldscript` would probably cover most of the use cases. On the other hand, GNU ld's behavior is to assume that anything that isn't recognized as a binary object file (which are presumably identified by magic) is a linker script, and we probably want to emulate that behavior (certainly we don't want to assume that something passed to a GNU ld driver is YAML, unless explicitly told so via a flag not present in GNU ld). -- Sean Silva> > I agree with Rui that we should not be calling identify_magic() in every > reader. Another approach is to just call each reader to try to parse until > one succeeds. The first reader should be the native reader (e.g. ELF or > mach-o), so that in the real world there is no time wasted looking at test > case formats. > > -Nick > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131011/e071bb3c/attachment.html>
On Wed, Oct 9, 2013 at 11:23 AM, Shankar Easwaran <shankare at codeaurora.org>wrote:> Hi, > > We have a whole bunch of readers(we would have some more too), and was > thinking if we should have a vector of Readers, and have a function > isMyFormat in each of them. > > Any reader that knows to handle, goes ahead and parses the file. > > On a side note, we currently use .objtxt as an figure out if the file is a > YAML file or not. I have added FIXME's in the code, if we could some kind > of magic (or) a better way to figure out if the file is YAML ? > > Thanks > > Shankar Easwaran > > -- > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted > by the Linux Foundation > >So apparently I didn't reply all when I suggested this. For determining which YAML reader to use, we should use YAML tags. This allows multiple different types of input files in a single YAML stream. !archive <blah> --- !ELF <blah> --- !atoms <blah> For differentiating between linker scripts and YAML, I agree that some form of comment magic is best. - Michael Spencer -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131014/a9434db3/attachment.html>
On Mon, Oct 14, 2013 at 8:41 PM, Michael Spencer <bigcheesegs at gmail.com>wrote:> On Wed, Oct 9, 2013 at 11:23 AM, Shankar Easwaran <shankare at codeaurora.org > > wrote: > >> Hi, >> >> We have a whole bunch of readers(we would have some more too), and was >> thinking if we should have a vector of Readers, and have a function >> isMyFormat in each of them. >> >> Any reader that knows to handle, goes ahead and parses the file. >> >> On a side note, we currently use .objtxt as an figure out if the file is >> a YAML file or not. I have added FIXME's in the code, if we could some kind >> of magic (or) a better way to figure out if the file is YAML ? >> >> Thanks >> >> Shankar Easwaran >> >> -- >> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted >> by the Linux Foundation >> >> > So apparently I didn't reply all when I suggested this. > > For determining which YAML reader to use, we should use YAML tags. This > allows multiple different types of input files in a single YAML stream. > > !archive > <blah> > --- > !ELF > <blah> > --- > !atoms > <blah> > > For differentiating between linker scripts and YAML, I agree that some > form of comment magic is best. >Since our YAML stuff is all internal anyway, wouldn't it be simpler to just hardcode the limited set of extensions we use for YAML files, and only do that with non-emulated drivers unless explicitly asked to do so? -- Sean Silva> > - Michael Spencer > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131014/53c2ae21/attachment.html>