Syslinux - Nov 2016 - [PATCH 0/2] improve Lua API for files and initramfs objects

Hi,
> Ferenc W?gner <wferi at niif.hu>:
> for reading configuration files from disk.  Does it not work with HTTP/
> TFTP for you?  Using that, load_file could be implemented in Lua as:
that does work, yes. It just looked like a file object should have a some way
to get the contents. I'm actually not using it in our scripts.

> All in all, I can't see the need for new bindings for this task, but
> maybe I miss something.  At the same time, file:data() might
> occasionally come useful.
initramfs:load_file() is basically a memory optimization over your solution.
It calls initramfs_load_file() directly, this loads the file exactly once into
memory.

Going through a Lua string loads the file, then copies it into a (interned) Lua
string,
then copies this string again in initramfs:add_file(). This doubles the peak
memory usage
(might even triple it if you are not freeing the file before passing it into
add_file)

This is a concern for our setup: we load relatively large wimboot images and
support relatively small VMs at the same time.


 Paul

Paul Emmerich <P.Emmerich at first-colo.net> writes:
> initramfs:load_file() is basically a memory optimization over your
> solution.  It calls initramfs_load_file() directly, this loads the
> file exactly once into memory.
True.
> Going through a Lua string loads the file, then copies it into a
> (interned) Lua string, then copies this string again in
> initramfs:add_file(). This doubles the peak memory usage (might even
> triple it if you are not freeing the file before passing it into
> add_file)
>
> This is a concern for our setup: we load relatively large wimboot
> images and support relatively small VMs at the same time.
Then calling initramfs_load_file() directly is a useful optimization
indeed.
-- 
Feri

On 11/10/16 04:38, Paul Emmerich via Syslinux wrote:
> Hi,
> 
>> Ferenc W?gner <wferi at niif.hu>:
>> for reading configuration files from disk.  Does it not work with HTTP/
>> TFTP for you?  Using that, load_file could be implemented in Lua as:
> 
> that does work, yes. It just looked like a file object should have a some
way
> to get the contents. I'm actually not using it in our scripts.
> 
It does seem useful.  So what you are saying here is that it isn't
actually broken, it is just suboptimal?

	-hpa

"H. Peter Anvin" <hpa at zytor.com> writes:
> On 11/10/16 04:38, Paul Emmerich via Syslinux wrote:
> 
>> Ferenc W?gner <wferi at niif.hu>:
>>
>>> for reading configuration files from disk.  Does it not work with
HTTP/
>>> TFTP for you?  Using that, load_file could be implemented in Lua
as:
>> 
>> that does work, yes. It just looked like a file object should have a
some way
>> to get the contents. I'm actually not using it in our scripts.
>
> It does seem useful.  So what you are saying here is that it isn't
> actually broken, it is just suboptimal?
No, this interface is completely missing at the moment.  You can read a
file using standard Lua I/O calls and work with its content.  Or you can
read a file using loadfile() provided by Syslinux, but then you can't
look at its contents from Lua.  You can run it as a kernel or use it as
an initramfs, but the only available Lua methods are name and size now.
Paul adds the data method to export the contents to Lua, mainly for the
sake of completeness, if I understand him correctly.
-- 
Feri