> On Sat, 20 Dec, at 12:48:18AM, Ady wrote:
> >
> > So, my first (still incomplete and still inaccurate) attempt to write
> > some rules about the PATH directive, and ask about remaining
doubts!...
> >
> >
> > The search (for c32 files) is supposed to respect the following rules:
> >
> > 1_ The search for c32 files is performed according to the following
> > rules. The search for each required c32 file is stopped once the first
> > matching file name is found; remnant potential paths are not searched
> > (even if the execution of the firstly-found c32 file fails in some
way;
> > e.g. unmatching version, unmatching firmware architecture).
>
> Correct.
>
> > 2_ Every required c32 file starts a new search (for itself); i.e. once
> > a c32 file is found, if additional c32 (dependency) files are also
> > required then each c32 file triggers its own new search, according to
> > all these rules.
>
> Correct.
>
> > 3_ When the c32 file is explicitly referenced by an absolute path, the
> > explicitly-referenced c32 file is search-for in such location first.
>
> Correct.
>
> This question forced me to go stare at the code ;-) Specifically
> findpath() and open_file().
>
> Every time we attempt to open a .c32 module we call fopen() and the call
> stack ultimately looks like this,
>
> fopen()
> open()
> open_file()
> searchdir()
>
> So this is where the "search for absolute path first" semantics
come
> from. This part is the same for every file that gets opened, not just
> .c32 files.
>
> I've noticed a bug in the PATH code because lookups of an absolute path
> happen relative to a directory in PATH.
>
> That sentence sounds pretty confusing, so an example might help explain
> this bug...
>
> If you try to open "/absolute/path/to/my/file.c32" and that file
doesn't
> exist, the PATH lookup code will append that path to every entry in the
> PATH list.
>
> Assuming your PATH contains "/boot:/foobar", the PATH lookup code
will
> try and open the following files,
Hmm, I thought it was decided to consider the colon (":") character as
deprecated, and instead we should be using a space character as
separator for the PATH directive (or multiple lines). Well, at least as
for the current MASTER HEAD, Syslinux 6.03.
>
> /boot/absolute/path/to/my/file.c32
> /foobar/absolute/path/to/my/file.c32
>
> Which is totally bogus, and contrary to what "absolute path"
means.
>
> I've created the following bug report to track this issue,
>
> http://bugzilla.syslinux.org/show_bug.cgi?id=58
Was/is this behavior really undesired?
And, what _should_ had been the correct behavior?
>
> > 4_ When the c32 file is explicitly referenced by a relative path, the
> > explicitly-referenced c32 file is search-for in such location first,
> > where the base for the relative path is the Current Working Directory.
> > (Note: this is not the same as searching in the CWD itself.)
>
> Yes, this is the same as for the regular file lookup (see answer to 3.)
>
> > 5_ If the CONFIG directive was used so to change the Working
Directory,
> > then the _prior_ Working Directory is no longer relevant; only the
> > *Current* Working Directory is considered as base for relative paths
> > where the c32 files are searched-for. The (new) CWD takes precedence
> > over any PATH directive.
>
> Wow, good question. Yes, the new current working directory should take
> precedence.
>
> > 6_ After the relevant aforementioned paths, the paths stated in all
the
> > parsed PATH directives are searched-for, one-by-one until the file is
> > found, in the same order as they are stated in the parsed
configuration
> > files. Note that all paths referenced in PATH directives are parsed as
> > *absolute* paths, even when they are not explicitly written with a
> > starting slash symbol ("/").
>
> I'm not sure the absolute part is true. It should make sense to require
> that, but from reading the code, I think any directories not prefixed
> with "/" are treated as being relative to the current working
directory.
>
> Do you have some data to suggest otherwise?
>
For paths (in a cfg or in CLI) other than in the PATH directive: any
path not prefixed with a slash character (nor with a colon character,
indicating a TFTP root) is to be parsed as relative to the CWD. But...
For the PATH directive (only), the following (alternative) lines:
_ PATH some/path
_ PATH /some/path
_ PATH some/path/
_ PATH /some/path/
tested separately / independently (not together / not simultaneously /
not in the same boot attempt), seem to be treated all as an absolute
path (if I remember my old tests correctly, but please don't trust my
memory).
In other words, the PATH directive disregards whether the argument (for
the PATH directive itself) was written in relative or in absolute
notation, and it always parses it as absolute path. Considering that
the PATH directive cannot make use if a relative path, this behavior
makes sense. Possible potential alternative behavior(s) could/would be
either to fail, or to accept relative paths based on the CWD.
> > 7_ If the CONFIG directive was used, the paths in prior PATH
directives
> > are still valid, and they take precedence over new PATH directives. In
> > other words, newly-parsed PATH directives are appended after
> > previously-parsed PATH directives (the new ones do not replace prior
> > ones). Once a configuration file was parsed:
> > 7.1_ there is no method to "reset" the PATH directive.
> > 7.2_ there is no method to eliminate a path already stated in the PATH
> > directive.
> > 7.3_ there is no method to alternate the order in which the paths
> > stated in PATH directives are searched-for.
> > 7.4_ all aforementioned "PATH rules" (starting from rule #1)
are
> > re-evaluated, according to the new CWD.
>
> Correct.
>
> > 8_ These "PATH rules" apply for almost every-and-any c32
file, except
> > for "ldlinux.c32". Other types of files (such as
configuration files,
> > kernels, initrd, memdisk,...) are only searched-for according to the
> > paths being used in their respective lines in the Syslinux
> > configuration file, or according to the paths used in the boot command
> > prompt; i.e. if non-c32 files are not found in the firstly-expected
> > location, no additional searching is performed for such files.
>
> I think this is also correct.
>
> > And now, the current remaining doubts!!!:
> >
> > D1_ Matt Fleming said:
> > > Yes, the PATH directive is used as a fallback if the file denoted
by
> > an absolute path doesn't exist.
> >
> > D1Q: When using an *absolute* path and the c32 file is not found in
> > such location, is the CWD expected to be searched as first fallback
for
> > the *explicitly-referenced* c32 file, before searching according to
the
> > PATH directive(s)? Note that I am not asking about the c32 dependency
> > files, but about the explicitly-referenced one(s) (e.g. not about
> > lib*.c32 files but about "/some/absolute/path/menu.c32").
>
> When using an absolute path, the current working directory is not
> searched.
>
Then the statement that "the CWD is *always* searched-for first" is
inaccurate.
The statement (as included in the official documentation) seems to be
directed to c32 dependencies, but then this is one of the reasons why
the PATH rules _seemed_ to be relevant for c32 dependencies only. Since
the PATH rules apply to all c32 files (not just c32 dependencies), then
the wording for the PATH rules should account for each-and-all cases,
not just for dependencies.
>From users' perspective...
The statement might be accurate for c32 *dependencies*, but not as much
for explicitly-referenced c32 files (because it seems to depend on the
way the c32 file is referenced; in absolute or relative notation).
Which means that the other statement, "all c32 files are treated
equally, whether they are dependencies or not", is also not 100%
accurate.
So I have to go back to a prior question: what _should_ be the adequate
behavior?
Do we want the same (or a different) behavior(s) for
explicitly-referenced c32 files comparing to c32 dependencies?
Do we want a different behavior for explicitly-referenced c32 files
depending on the type of path notation?
Do we want the same behavior for explicitly-referenced c32 files
independently of the type of path notation?
So my first attempt to write a complete set of PATH rules failed (plus,
we have a bug and I am not sure what the correct behavior should had
been).
If I understood correctly, the behavior of the c32 *dependencies* (WRT
to the PATH directive / rules) is not influenced by the path notation
of the explicitly-referenced file; but these questions are also about
the behavior of those explicitly-referenced c32 files (WRT to the PATH
directive / rules).
> > D2_ For this doubt I will use an example:
> > UI some/relative/path/menu.c32
> >
> > Whether typed in CLI or parsing a configuration file, note that this
is
> > a relative path *based* on the CWD, but not the CWD itself.
> >
> > The "PATH rules" as documented in the official Syslinux 6.03
> > distribution archives state that the CWD is "always"
searched-for
> > first, before the PATH directive(s). This statement (and the example
> > included in the official Syslinux v.6.03 documentation) clarifies the
> > expected behavior for paths written in absolute notation. But...
> >
> >
Apparently, I was wrong!? As per this discussion, the statement doesn't
seem to apply to c32 files referenced by absolute paths:
[quote=Matt Fleming]
When using an absolute path, the current working directory is not
searched.
[/quote=Matt Fleming]
I need to (re-)test this, although I am not sure I currently should,
considering my current doubts (as I am writing them here), plus, the
existence of a bug (#58).
> > D2Q1: In the case I am presenting here (c32 module explicitly
> > referenced in relative notation but not referenced to be located in
the
> > CWD itself), if menu.c32 is not found first in the referenced
> > directory:
> > <CWD>some/relative/path/
> > then, is "menu.c32" supposed to be searched in the CWD
*itself* too
> > (before the PATH directive paths)? Note that I am not asking here
about
> > the non-referenced c32 files such as lib*.c32.
>
> I think I understand this question, and the answer is "no". The
file
> lookup code will not strip "some/relative/path/" when performing
the
> lookup, which I think is the question you're asking.
>
Then again, the statement:
"the CWD is *always* searched-for first"
is not accurate.
Perhaps I need to present here several cases (with incremental
complexities), so the complete search-for priorities / list of
possibilities is presented as answers. Then perhaps a complete set of
rules could be derived from these cases. I fear it would take too long
to prepare, and the most probable reaction would be "tl;dr"(?).
> > D2Q2: In the case I am presenting here (c32 module explicitly
> > referenced in relative notation but not referenced to be located in
the
> > CWD itself), in which locations are the c32 *dependencies* of
> > "menu.c32" searched-for and in which order (before searching
according
> > to the PATH directive)? Are its *dependencies* searched-for in
> > "<CWD>some/relative/path/"? Are its *dependencies*
searched-for in the
> > CWD (too)?
>
> Loading of dependencies follows the same rules as loading any .c32 file,
> there's nothing special about the actual lookup process for
> dependencies.
>
> However, it's worth pointing out that dependencies are specified as
> simple filenames without any path component in the ELF sections for .c32
> files. You can see this for yourself using readelf(1) or some other ELF
> header parsing tool.
>
> Here's the output for chain.c32,
>
> Dynamic section at offset 0x63b8 contains 16 entries:
> Tag Type Name/Value
> 0x0000000000000001 (NEEDED) Shared library: [libutil.c32]
> 0x0000000000000001 (NEEDED) Shared library: [libcom32.c32]
>
> See how there's no path component for libutil.c32 and libcom32.c32?
This
> is the reason that dependencies are first searched for in the CWD.
>
> But the file lookup code never distinguishes between .c32 and their
> dependencies.
>
> In your example, dependencies will be searched for in CWD, but not
> "<CWD>some/relative/path", because lookup is performed
using the CWD of
> the process doing the loading.
>
> > D2Q3: Could the search for each of the c32 dependencies be different
in
> > each case, according to, or depending on, where the *prior* c32 file
> > was found (e.g. hdt.c32, or libutil.c32, before searching for
> > libcom32.c32, or libmenu.c32, or..., all so eventually hdt.c32 will
> > load and work)? Or, instead, is each search fully independent of the
> > result of the prior (library) c32 file that have been just
> > searched-for?
>
> Each search should be independent.
>
> > D3Q1: I know that in the CLI there is no equivalent to the PATH
> > directive, but the PATH *rules* are still valid in the command prompt,
> > right? Are still _all_ the rules valid? Are there any differences?
>
> No, there shouldn't be any differences.
>
> > D3Q2: Are there any *different* "PATH rules" for the boot
prompt,
> > whether using (typing-in) relative paths or absolute paths notation?
>
> Nope.
>
Now I'm confused. You just mentioned different behaviors depending on
the type of notation, relative or absolute; perhaps not regarding c32
dependencies but regarding the explicitly-referenced c32 files.
> > D4_ When typing in the boot prompt, we know that the ".c32"
file name
> > extension is used as fallback. For example, (IIRC) if I type in the
> > boot prompt:
> >
> > ls
> >
> > then Syslinux will try to find such file name (or label),
"as-is"
> > ("ls"). If such exact file name (or label) is not found in
the expected
> > location, then Syslinux tries to find "ls.c32" in the same
location.
> >
> >
> > D4Q1: Are any "PATH rules" applied to such
"fallback" file name,
> > "ls(.c32)"?
>
> Yes, the usual PATH rules. There are no different PATH rules.
>
> > D4Q2: Are any "PATH rules" applied to the dependencies of
such
> > "fallback" file name, "ls(.c32)"?
>
> Yes, the usual PATH rules.
>
> > Matt, my goal is to improve documentation and to be able to test
> > whether Syslinux actually behaves as expected regarding to the PATH
> > rules. I would appreciate if these doubts could be clarified (and if I
> > made/make a mistake or inaccuracy, please correct me), so the PATH
> > rules' documentation can be improved.
>
> Thanks, I appreciate that you want to provide better documentation.
>
> --
> Matt Fleming, Intel Open Source Technology Center
>
With your help, I hope I can eventually get to a complete set of rules
covering all cases.
I also hope that current bugs related to the PATH directive won't
generate user's misuse (because correcting such bugs in the future
could potentially mean "breaking behavior" from the user's
perspective). For now, there are not
many users actually using the PATH directive in so many ways, but this
status
might not be true so much longer.
TIA,
Ady.