llvm dev - Apr 2013 - [LLVMdev] A new mechanism to compiler kernel modules using llvm: Defer type evaluation in clang?

On Mon, Apr 29, 2013 at 3:11 PM, Jean-Daniel Dupas
<devlists at shadowlab.org> wrote:
> Just out of curiosity, what would be the main benefit of this approach vs
DKMS which is already widely used ?
>
Thanks Dupas.

I checked DKMS you mentioned, basically DKMS is just a ko and its
sources management tool.

It's not easy to deploy ko source into target machine, and it's more
harder to build ko in target machine,
this is a very common case, especially in embedded Linux, you really
cannot put source into target device,
for example, you didn't see any kernel module source in android phone,
right?

What I really want to see is kernel module source don't need to to
change when kernel changes, whatever kernel
upgrade or minor structure change, DKMS cannot meet this.

what's the benefit of that?

we can replace kernel as we want, and don't need to worry ko cannot
work in new kernel, this is the true decouple,
not the decouple implementation in DKMS claimed.
Imaging that you want upgrade the android kernel to kernel mainline by
yourself, you really could do this if using the mechanism
I said, but you definitely cannot achieve this currently or use DKMS.

Another case is fast kernel debugging, just replace kernel, to check
functionality or performance in new kernel, without ko recompile,
ko may provided by different vendor, we cannot contact every vendor to
request they deliver new ko to let us try our new kernel.

On Mon, Apr 29, 2013 at 08:14:57PM +0800, Jovi Zhang
wrote:
> On Mon, Apr 29, 2013 at 3:11 PM, Jean-Daniel Dupas
> <devlists at shadowlab.org> wrote:
> > Just out of curiosity, what would be the main benefit of this approach
vs DKMS which is already widely used ?
> >
> Thanks Dupas.
> 
> I checked DKMS you mentioned, basically DKMS is just a ko and its
> sources management tool.
> 
> It's not easy to deploy ko source into target machine, and it's
more
> harder to build ko in target machine,
> this is a very common case, especially in embedded Linux, you really
> cannot put source into target device,
> for example, you didn't see any kernel module source in android phone,
right?
> 
> What I really want to see is kernel module source don't need to to
> change when kernel changes, whatever kernel
> upgrade or minor structure change, DKMS cannot meet this.
Hello,
Based on my many years of lurking on linux kernel development lists, IMHO,
that is never going to happen. You need the full support of the linux kernel
development community to succeed with your goal. You need to be talking to
the linux kernel development community. You are on the wrong list.

Folks who are truly linux kernel experts have no problem with embedded
development
environments. In my experience, linux server environments are orders of
magnitude
more complex than embedded environments. True linux kernel experts don't
even use
debuggers.

Of course I wish you well with your efforts. But you need to fully grasp the
ethos
of the linux kernel development community, before you can succeed at such a
goal.

enjoy,
Karen
-- 
Karen Shaeffer
Neuralscape, Mountain View, CA 94040

On Tue, Apr 30, 2013 at 4:53 AM, Karen Shaeffer
<shaeffer at neuralscape.com> wrote:
> On Mon, Apr 29, 2013 at 08:14:57PM +0800, Jovi Zhang wrote:
>> On Mon, Apr 29, 2013 at 3:11 PM, Jean-Daniel Dupas
>> <devlists at shadowlab.org> wrote:
>> > Just out of curiosity, what would be the main benefit of this
approach vs DKMS which is already widely used ?
>> >
>> Thanks Dupas.
>>
>> I checked DKMS you mentioned, basically DKMS is just a ko and its
>> sources management tool.
>>
>> It's not easy to deploy ko source into target machine, and it's
more
>> harder to build ko in target machine,
>> this is a very common case, especially in embedded Linux, you really
>> cannot put source into target device,
>> for example, you didn't see any kernel module source in android
phone, right?
>>
>> What I really want to see is kernel module source don't need to to
>> change when kernel changes, whatever kernel
>> upgrade or minor structure change, DKMS cannot meet this.
>
> Hello,
> Based on my many years of lurking on linux kernel development lists, IMHO,
> that is never going to happen. You need the full support of the linux
kernel
> development community to succeed with your goal. You need to be talking to
> the linux kernel development community. You are on the wrong list.
Thanks karen.
Main implementation part is engaging with LLVM, not the kernel, that's
why I send it to here firstly.
I'm also on the LKML many years, and I know how they will say:
"Just upstream your kernel module"

the point is "upstream my kernel module" doesn't fix the problem I
want to solve:
kernel module binary compatibility issue.

(Possibility I will involve LKML in some day as you suggest, thanks:) )
>
> Folks who are truly linux kernel experts have no problem with embedded
development
> environments. In my experience, linux server environments are orders of
magnitude
> more complex than embedded environments. True linux kernel experts
don't even use
> debuggers.
>
> Of course I wish you well with your efforts. But you need to fully grasp
the ethos
> of the linux kernel development community, before you can succeed at such a
goal.
>

On Tue, Apr 30, 2013 at 4:13 PM, James Courtier-Dutton
<james.dutton at gmail.com> wrote:
> Jovi,
>
> The problem with kernel module binary compatibility is that if you
> fixed the kernel module binary API, it will stifle innovation within
> the kernel.
> So, to the kernel developers, a changing module API is a bonus feature
> and not an issue. I expect you will have no support from the LKML.
> What is it in your current kernel driver that prevents it from being
> open sourced?
> What stops you from releasing a datasheet for your hardware?
>
Thanks James.

I can list hundreds of reasons on why there exist out of tree kernel modules:
code quality is not good to upstream;
kernel community think the feature is not very common, so it's not
worth to upstream;
kernel community reject the kernel modules;
company don't want to assign a long term module maintainer;
company don't want to release kernel modules source, kernel allow for
this(see EXPORT_SYMBOL);
the functionality is duplicated in mainline, but company must maintain
the interface for old customers;
company have many kernel version product, so it have to maintain
different modules sources, but mainline only maintain one version.
many famous open source kernel modules also not upstream, like
Systemtap from redhat, why this happen?
...

The truth is there, many kernel modules is out of tree, why we need to
argue this truth?

Frankly, we have a little out of topic now, I guess you don't have
engaged this kernel module
binary compatibility issue before, the world is not that simple as
upstream can solve anything.

(Look Microsoft, is all drive company upstream the driver source to
Microsoft? to solve driver compatibility issue)

Let's back to original topic: how to use LLVM mechanism to solve
kernel module binary compatibility issue in a good way?

Thanks.