Syslinux - Nov 2016 - [PATCH 0/2] improve Lua API for files and initramfs objects

"H. Peter Anvin" <hpa at zytor.com> writes:
> On 11/10/16 04:38, Paul Emmerich via Syslinux wrote:
> 
>> Ferenc W?gner <wferi at niif.hu>:
>>
>>> for reading configuration files from disk.  Does it not work with
HTTP/
>>> TFTP for you?  Using that, load_file could be implemented in Lua
as:
>> 
>> that does work, yes. It just looked like a file object should have a
some way
>> to get the contents. I'm actually not using it in our scripts.
>
> It does seem useful.  So what you are saying here is that it isn't
> actually broken, it is just suboptimal?
No, this interface is completely missing at the moment.  You can read a
file using standard Lua I/O calls and work with its content.  Or you can
read a file using loadfile() provided by Syslinux, but then you can't
look at its contents from Lua.  You can run it as a kernel or use it as
an initramfs, but the only available Lua methods are name and size now.
Paul adds the data method to export the contents to Lua, mainly for the
sake of completeness, if I understand him correctly.
-- 
Feri

On November 19, 2016 6:06:43 AM PST, wferi at niif.hu
wrote:
>"H. Peter Anvin" <hpa at zytor.com> writes:
>
>> On 11/10/16 04:38, Paul Emmerich via Syslinux wrote:
>> 
>>> Ferenc W?gner <wferi at niif.hu>:
>>>
>>>> for reading configuration files from disk.  Does it not work
with
>HTTP/
>>>> TFTP for you?  Using that, load_file could be implemented in
Lua
>as:
>>> 
>>> that does work, yes. It just looked like a file object should have
a
>some way
>>> to get the contents. I'm actually not using it in our scripts.
>>
>> It does seem useful.  So what you are saying here is that it isn't
>> actually broken, it is just suboptimal?
>
>No, this interface is completely missing at the moment.  You can read a
>file using standard Lua I/O calls and work with its content.  Or you
>can
>read a file using loadfile() provided by Syslinux, but then you can't
>look at its contents from Lua.  You can run it as a kernel or use it as
>an initramfs, but the only available Lua methods are name and size now.
>Paul adds the data method to export the contents to Lua, mainly for the
>sake of completeness, if I understand him correctly.
It was the "for completeness" part that I wanted to know.

Incidentally, why do we copy the data if it is in a Lua string (or are we?)
Seems pointless to me.
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

hpa at zytor.com writes:
> On November 19, 2016 6:06:43 AM PST, wferi at niif.hu wrote:
>
>>"H. Peter Anvin" <hpa at zytor.com> writes:
>>
>>> On 11/10/16 04:38, Paul Emmerich via Syslinux wrote:
>>> 
>>>> Ferenc W?gner <wferi at niif.hu>:
>>>>
>>>>> for reading configuration files from disk.  Does it not
work with
>>>>> HTTP/ TFTP for you?  Using that, load_file could be
implemented in
>>>>> Lua as:
>>>> 
>>>> that does work, yes. It just looked like a file object should
have
>>>> a some way to get the contents. I'm actually not using it
in our
>>>> scripts.
>>>
>>> It does seem useful.  So what you are saying here is that it
isn't
>>> actually broken, it is just suboptimal?
>>
>> No, this interface is completely missing at the moment.  You can read
>> a file using standard Lua I/O calls and work with its content.  Or
>> you can read a file using loadfile() provided by Syslinux, but then
>> you can't look at its contents from Lua.  You can run it as a
kernel
>> or use it as an initramfs, but the only available Lua methods are
>> name and size now.  Paul adds the data method to export the contents
>> to Lua, mainly for the sake of completeness, if I understand him
>> correctly.
>
> It was the "for completeness" part that I wanted to know.
>
> Incidentally, why do we copy the data if it is in a Lua string (or are
> we?) Seems pointless to me.
Are you asking about the following hunk?  (No, you probably don't, skip!)

+static int sl_filedata(lua_State * L)
+{
+    const syslinux_file *file = luaL_checkudata(L, 1, SYSLINUX_FILE);
+
+    lua_pushlstring(L, file->data, file->size);
+
+    return 1;
+}

This indeed copies the data from the syslinux_file structure (which is
an opaque SYSLINUX_FILE object on the Lua side) into an (interned) Lua
string to make it available to the Lua string functions.

You probably ask why we copy the data when we create those SYSLINUX_FILE
userdata objects.  We don't, the data is read directly into the
syslinux_file structure by the loadfile() Syslinux function.  This was
already the case when I first saw the Lua binding in Syslinux.  Thinking
about it now, the SYSLINUX_FILE userdata objects and their entire
interface could be replaced by Lua tables with two string fields for
much the same effect, unless the native Lua string I/O is much less
efficient than loadfile().  If we can drop the name method, even simple
Lua strings would suffice.  The complication arises when you assemble
initramfs objects incrementally: either you'll have to copy the data, or
keep a reference to each constituent Lua string and delay creating the
actual initramfs structure until its point of usage (and destroy it if
you somehow fail back to Lua).  This might be worth exploring.
-- 
Feri