ed at modk.it
2015-Jul-18 14:46 UTC
[LLVMdev] [Clang] [lld] [llvm-link] Whole program / dead-code optimization
Thanks Nick. I've been pursuing Gao's technique but can't seem to
get opt
to remove obviously dead code from even the following trivial example:
int mult(int a, int b){
return a*b;
}
int main(void){
return 0;
}
While mult is never called it still is not removed. I just can't seem to
get opt to understand it's seeing the whole program so it can remove this
globally accessible function. What am I missing? Seems related to the
missing -fwhole-program flag in clang. Perhaps this is not even possible?
If I can't get any answers here I may repost that specific question since I
didn't list [opt] in the original question subject.
Thanks,
Ed
On Fri, Jul 17, 2015 at 1:15 AM, Nick Lewycky <nicholas at mxc.ca> wrote:
> ed at modk.it wrote:
>
>>
>> Is there a reason why LLVM's link-time optimization won't
work for
>> you?
>>
>> http://llvm.org/docs/GoldPlugin.html
>> <
>>
https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_docs_GoldPlugin.html&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Mfk2qtn1LTDThVkh6-oGglNfMADXfJdty4_bhmuhMHA&m=rF94h73bKDdWVhxOWqRXpvw5pSMgvuHQXJ__qw8n2LU&s=PR31BXeMANGrAQP2Tt9Eg5psH82vj8Oq1WmyprGhyn8&e>>
>
>> http://llvm.org/docs/LinkTimeOptimization.html
>> <
>>
https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_docs_LinkTimeOptimization.html&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Mfk2qtn1LTDThVkh6-oGglNfMADXfJdty4_bhmuhMHA&m=rF94h73bKDdWVhxOWqRXpvw5pSMgvuHQXJ__qw8n2LU&s=PoqmeRXrssdG9xj6Fko_SKttwLPWqUVkxFH41dOcg4w&e>>
>
>>
>>
>> Well the primary motivation to move to LLVM is licensing which is why
we
>> also ditched binutils since we can't package gcc for iOS due to the
>> GPL. So in the end the gold plugin wouldn't work for licensing
reasons
>> even if we can get it to work technically but thanks for the links
I'm
>> still trying to wrap my head around the problem and any info helps.
>>
>
> The right future is a world where lld performs llvm lto for you.
>
> Until then, the technique in Gao's PDF is what I would recommend.
>
> Nick
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150718/695b22de/attachment.html>
ed at modk.it
2015-Jul-18 16:58 UTC
[LLVMdev] [Clang] [lld] [llvm-link] Whole program / dead-code optimization
After digging a bit more it seems we can achieve the same as gcc's
-fwhole-program by simply marking the mult function as "static" which
is
all -fwhole-program does anyway. From
https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html:
*-fwhole-program**Assume that the current compilation unit represents the
whole program being compiled. All public functions and variables with the
exception of main and those merged by attributeexternally_visible become
static functions and in effect are optimized more aggressively by
interprocedural optimizers.*
So we can accomplish that for now with a simple pass on the source. But
that had me thinking, how do we accomplish the same for unused C++ classes
or member functions within classes. I figured we could accomplish that by
changing the linkage type within the llvm IR. But it turns out these
already get linkonce_odr linkage. http://llvm.org/docs/LangRef.html states
"Unreferenced linkonce globals are allowed to be discarded"
class num{
private:
int number;
public:
num(int n):number(n){}
int mult(int other){
return number*other;
}
};
int main(void){
return 0;
}
If I compile the above to LLVM IR there is actually no trace of the num
class which kind of baffles me because what if I was compiling a library
and this class was needed in the library consumer?
Either way with this knowledge I think we can get the results we're looking
for in the short term and will follow up if we find or come up with
anything that could be generally useful to others.
Thanks,
Ed
On Sat, Jul 18, 2015 at 9:46 AM, ed at modk.it <ed at modk.it> wrote:
> Thanks Nick. I've been pursuing Gao's technique but can't seem
to get opt
> to remove obviously dead code from even the following trivial example:
>
> int mult(int a, int b){
>
> return a*b;
>
> }
>
>
> int main(void){
>
> return 0;
>
> }
>
>
> While mult is never called it still is not removed. I just can't seem
to
> get opt to understand it's seeing the whole program so it can remove
this
> globally accessible function. What am I missing? Seems related to the
> missing -fwhole-program flag in clang. Perhaps this is not even possible?
> If I can't get any answers here I may repost that specific question
since I
> didn't list [opt] in the original question subject.
>
>
> Thanks,
>
> Ed
>
> On Fri, Jul 17, 2015 at 1:15 AM, Nick Lewycky <nicholas at mxc.ca>
wrote:
>
>> ed at modk.it wrote:
>>
>>>
>>> Is there a reason why LLVM's link-time optimization
won't work for
>>> you?
>>>
>>> http://llvm.org/docs/GoldPlugin.html
>>> <
>>>
https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_docs_GoldPlugin.html&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Mfk2qtn1LTDThVkh6-oGglNfMADXfJdty4_bhmuhMHA&m=rF94h73bKDdWVhxOWqRXpvw5pSMgvuHQXJ__qw8n2LU&s=PR31BXeMANGrAQP2Tt9Eg5psH82vj8Oq1WmyprGhyn8&e>>>
>
>>> http://llvm.org/docs/LinkTimeOptimization.html
>>> <
>>>
https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_docs_LinkTimeOptimization.html&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Mfk2qtn1LTDThVkh6-oGglNfMADXfJdty4_bhmuhMHA&m=rF94h73bKDdWVhxOWqRXpvw5pSMgvuHQXJ__qw8n2LU&s=PoqmeRXrssdG9xj6Fko_SKttwLPWqUVkxFH41dOcg4w&e>>>
>
>>>
>>>
>>> Well the primary motivation to move to LLVM is licensing which is
why we
>>> also ditched binutils since we can't package gcc for iOS due to
the
>>> GPL. So in the end the gold plugin wouldn't work for licensing
reasons
>>> even if we can get it to work technically but thanks for the links
I'm
>>> still trying to wrap my head around the problem and any info helps.
>>>
>>
>> The right future is a world where lld performs llvm lto for you.
>>
>> Until then, the technique in Gao's PDF is what I would recommend.
>>
>> Nick
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150718/bbafff43/attachment.html>
Nick Lewycky
2015-Jul-19 04:25 UTC
[LLVMdev] [Clang] [lld] [llvm-link] Whole program / dead-code optimization
ed at modk.it wrote:> After digging a bit more it seems we can achieve the same as gcc's > -fwhole-program by simply marking the mult function as "static" which is > all -fwhole-program does anyway. From > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html: > > /-fwhole-program/ > /Assume that the current compilation unit represents the whole > program being compiled. All public functions and variables with the > exception of |main| and those merged by > attribute|externally_visible| become static functions and in effect > are optimized more aggressively by interprocedural optimizers. / > > So we can accomplish that for now with a simple pass on the source.The pass is called "-internalize". You can control its operation a bit with the -internalize-public-api-file=<filename> and -internalize-public-api-list=<list> flags. But> that had me thinking, how do we accomplish the same for unused C++ > classes or member functions within classes. I figured we could > accomplish that by changing the linkage type within the llvm IR. But it > turns out these already get linkonce_odr linkage. > http://llvm.org/docs/LangRef.html states "Unreferenced linkonce globals > are allowed to be discarded"The answer is still internalize. Don't include their names in the list of public APIs and they'll be switched to 'internal' linkage, then deleted by the LTO passes.> > classnum{ > > private: > > intnumber; > > public: > > num(intn):number(n){} > https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.htm > > intmult(intother){ > > returnnumber*other; > > } > > }; > > > intmain(void){ > > return0; > > } > > If I compile the above to LLVM IR there is actually no trace of the num > class which kind of baffles me because what if I was compiling a library > and this class was needed in the library consumer?There is no representation of a class in llvm or .o files. Instead, there's a plain struct which represents the typed memory (ie., storage for the non-static data members only), plus a pile of functions which take a pointer to that memory as their first argument (the member functions taking their 'this' argument). Note that 'static' functions also just change the linkage to internal. So does putting code in an anonymous namespace. The main thing you get out of linker integration into LLVM LTO is that the linker can look at the pile of non-llvm code and determine which symbols are required by the rest of the system and compute that public API list for internalize. Nick> Either way with this knowledge I think we can get the results we're > looking for in the short term and will follow up if we find or come up > with anything that could be generally useful to others. > > Thanks, > Ed > > On Sat, Jul 18, 2015 at 9:46 AM, ed at modk.it <mailto:ed at modk.it> > <ed at modk.it <mailto:ed at modk.it>> wrote: > > Thanks Nick. I've been pursuing Gao's technique but can't seem to > get opt to remove obviously dead code from even the following > trivial example: > > intmult(inta, intb){ > > returna*b; > > } > > > intmain(void){ > > return0; > > } > > > While mult is never called it still is not removed. I just can't > seem to get opt to understand it's seeing the whole program so it > can remove this globally accessible function. What am I missing? > Seems related to the missing -fwhole-program flag in clang. Perhaps > this is not even possible? If I can't get any answers here I may > repost that specific question since I didn't list [opt] in the > original question subject. > > > Thanks, > > Ed > > > On Fri, Jul 17, 2015 at 1:15 AM, Nick Lewycky <nicholas at mxc.ca > <mailto:nicholas at mxc.ca>> wrote: > > ed at modk.it <mailto:ed at modk.it> wrote: > > > Is there a reason why LLVM's link-time optimization > won't work for you? > > http://llvm.org/docs/GoldPlugin.html > <https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_docs_GoldPlugin.html&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Mfk2qtn1LTDThVkh6-oGglNfMADXfJdty4_bhmuhMHA&m=rF94h73bKDdWVhxOWqRXpvw5pSMgvuHQXJ__qw8n2LU&s=PR31BXeMANGrAQP2Tt9Eg5psH82vj8Oq1WmyprGhyn8&e> <https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_docs_GoldPlugin.html&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Mfk2qtn1LTDThVkh6-oGglNfMADXfJdty4_bhmuhMHA&m=rF94h73bKDdWVhxOWqRXpvw5pSMgvuHQXJ__qw8n2LU&s=PR31BXeMANGrAQP2Tt9Eg5psH82vj8Oq1WmyprGhyn8&e=>> > http://llvm.org/docs/LinkTimeOptimization.html > <https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_docs_LinkTimeOptimization.html&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Mfk2qtn1LTDThVkh6-oGglNfMADXfJdty4_bhmuhMHA&m=rF94h73bKDdWVhxOWqRXpvw5pSMgvuHQXJ__qw8n2LU&s=PoqmeRXrssdG9xj6Fko_SKttwLPWqUVkxFH41dOcg4w&e> <https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_docs_LinkTimeOptimization.html&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Mfk2qtn1LTDThVkh6-oGglNfMADXfJdty4_bhmuhMHA&m=rF94h73bKDdWVhxOWqRXpvw5pSMgvuHQXJ__qw8n2LU&s=PoqmeRXrssdG9xj6Fko_SKttwLPWqUVkxFH41dOcg4w&e=>> > > > Well the primary motivation to move to LLVM is licensing > which is why we > also ditched binutils since we can't package gcc for iOS due > to the > GPL. So in the end the gold plugin wouldn't work for > licensing reasons > even if we can get it to work technically but thanks for the > links I'm > still trying to wrap my head around the problem and any info > helps. > > > The right future is a world where lld performs llvm lto for you. > > Until then, the technique in Gao's PDF is what I would recommend. > > Nick > > >