Andrew Trick <atrick at apple.com> writes:>> When I asked about enhancing scheduler heuristics a month or so ago, I >> got a response about a MachineInstr scheduler and that that was the way >> of the LLVM future. Is that so? Is the ScheduleDAG going away? > > You sent a lengthy RFC on Apr 20 that demonstrated you aren't > following developments on trunk. That's perfectly fine, but if you > want to use the new scheduler before it is mature, you'll need to > follow trunk.Ok, but that doesn't answer the question. Is SchedulerDAG going away? If so, what's the timeframe for that? 3.2?> Feel free to use the patch and send your thanks to Hal. It doesn't > serve any purpose to mainline a partial solution only to replace it > before it can ever be enabled by default, which would require a major > performance investigation and introduces a huge risk (AliasAnalysis in > CodeGen has not be well tested).Er, but as I understand it the MachineInstr scheduler will also use alias analysis.>> So we are in a quandry. Do we continue our ScheduleDAG enhancements or >> do we wait for a MachineInstr scheduler? Will we have to throw away >> work on ScheduleDAG schedulers? > > If you're basing development on anything other than today's trunk, > then please continue working on the SelectionDag scheduler.But is that going to be throw away work? That's what we need to understand.> Either way, you won't need to throw much away. Target-specific > heuristics should be easy to transfer to an MI ScheduleDag from an SD > ScheduleDAG. ScheduleDAG doesn't change.Ok, if that's the case it will help. But we also need the flexibility provided by Hal's patch. I see that there is some work in that area in the MachineInstr scheduler. A timeline would be help as to when you expect this all to be ready.>> Is there a roadmap for the scheduler? When should we expect a >> MachineInstr scheduler to be usable? Can it be backported to 3.1? > > Only use MachineInstr scheduler now if it solves a pressing problem > for you. If you need it in 3.1, then develop your own MachineInstr > scheduler using the trunk implementation as a helpful reference. > > If you're developing based on 3.1, don't expect to upstream changes.Again, this doesn't answer the questions: - What's the expected release time for the new schedueler? 3.2? - How difficult do you expect a backport to 3.1 to be? We have to work from 3.1. Trunk is too buggy.>> In general, it would be very helpful for folks to post about major >> architectural changes like this before work begins so the rest of us can >> plan our work. After all, we are likely to run into the same problems >> that need solutions. As it is, it gets pretty frustrating to do a bunch >> of work in the current release only to have to throw it away because >> someone else did something different. > > There's a difference between having an intention to replace the > scheduler (for over a year now) and planning the work.Ok, but I don't recall any post about replacing the scheduler. If there was I missed it. That's easy to do on a list with hundreds of messages a day. It would be helpful to have a roadmap page on the LLVM web site to call attention to decisions like this. Such a page could be updated as timelines become firmer.> The actual planning coincided nicely with your RFC, providing a good > opportunity to air it on llvm-dev.I'm not sure what you mean by "planning." Do you mean design work? I think that's quite different from a roadmap with timelines. It's really the former that us non-Apple people need to have to effectively plan our work. The last thing we want to do is duplicate what's already planned to happen. -Dave
On May 9, 2012, at 8:34 AM, dag at cray.com wrote:> Andrew Trick <atrick at apple.com> writes: > >>> When I asked about enhancing scheduler heuristics a month or so ago, I >>> got a response about a MachineInstr scheduler and that that was the way >>> of the LLVM future. Is that so? Is the ScheduleDAG going away? >> >> You sent a lengthy RFC on Apr 20 that demonstrated you aren't >> following developments on trunk. That's perfectly fine, but if you >> want to use the new scheduler before it is mature, you'll need to >> follow trunk. > > Ok, but that doesn't answer the question. Is SchedulerDAG going away? > If so, what's the timeframe for that? 3.2? > >> Feel free to use the patch and send your thanks to Hal. It doesn't >> serve any purpose to mainline a partial solution only to replace it >> before it can ever be enabled by default, which would require a major >> performance investigation and introduces a huge risk (AliasAnalysis in >> CodeGen has not be well tested). > > Er, but as I understand it the MachineInstr scheduler will also use > alias analysis. > >>> So we are in a quandry. Do we continue our ScheduleDAG enhancements or >>> do we wait for a MachineInstr scheduler? Will we have to throw away >>> work on ScheduleDAG schedulers? >> >> If you're basing development on anything other than today's trunk, >> then please continue working on the SelectionDag scheduler. > > But is that going to be throw away work? That's what we need to > understand. > >> Either way, you won't need to throw much away. Target-specific >> heuristics should be easy to transfer to an MI ScheduleDag from an SD >> ScheduleDAG. ScheduleDAG doesn't change. > > Ok, if that's the case it will help. But we also need the flexibility > provided by Hal's patch. I see that there is some work in that area in > the MachineInstr scheduler. A timeline would be help as to when you > expect this all to be ready. > >>> Is there a roadmap for the scheduler? When should we expect a >>> MachineInstr scheduler to be usable? Can it be backported to 3.1? >> >> Only use MachineInstr scheduler now if it solves a pressing problem >> for you. If you need it in 3.1, then develop your own MachineInstr >> scheduler using the trunk implementation as a helpful reference. >> >> If you're developing based on 3.1, don't expect to upstream changes. > > Again, this doesn't answer the questions: > > - What's the expected release time for the new schedueler? 3.2? > > - How difficult do you expect a backport to 3.1 to be? We have to work > from 3.1. Trunk is too buggy.You've stated that trunk is too buggy for you to work from on multiple occasions. Can you elaborate? That doesn't match my experience, as I install a new compiler on my workstation from a trunk build every night and have only had a problem as a result once in the last year or more. It sounds like you've had a much different experience, which is unfortunate, and perhaps indicates a hole in our buildbot and nightly test infrastructure that could be fixed. -Jim> >>> In general, it would be very helpful for folks to post about major >>> architectural changes like this before work begins so the rest of us can >>> plan our work. After all, we are likely to run into the same problems >>> that need solutions. As it is, it gets pretty frustrating to do a bunch >>> of work in the current release only to have to throw it away because >>> someone else did something different. >> >> There's a difference between having an intention to replace the >> scheduler (for over a year now) and planning the work. > > Ok, but I don't recall any post about replacing the scheduler. If there > was I missed it. That's easy to do on a list with hundreds of messages > a day. It would be helpful to have a roadmap page on the LLVM web site > to call attention to decisions like this. Such a page could be updated > as timelines become firmer. > >> The actual planning coincided nicely with your RFC, providing a good >> opportunity to air it on llvm-dev. > > I'm not sure what you mean by "planning." Do you mean design work? I > think that's quite different from a roadmap with timelines. It's really > the former that us non-Apple people need to have to effectively plan our > work. The last thing we want to do is duplicate what's already planned > to happen. > > -Dave > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Jim Grosbach <grosbach at apple.com> writes:>> - How difficult do you expect a backport to 3.1 to be? We have to work >> from 3.1. Trunk is too buggy.> You've stated that trunk is too buggy for you to work from on multiple > occasions. Can you elaborate? That doesn't match my experience, as I > install a new compiler on my workstation from a trunk build every > night and have only had a problem as a result once in the last year or > more. It sounds like you've had a much different experience, which is > unfortunate, and perhaps indicates a hole in our buildbot and nightly > test infrastructure that could be fixed.We do a lot of Fortran which doesn't get covered all that well by the nightly tester. We also have a completely different frontend and optimizer meaning that we present code to LLVM that it's never seen before. We've seen major scalability problems in the past. We compile some absolutely gigantic codes here. Fortunately it appears most of these have been fixed but I think LoopStrengthReduce is still a problem (we've turned it off). Scheduling, regalloc, instcombine and dagcombine have all presented problems in the past. We've sent patches for the most egregious cases. Many of those patches get dropped, leading to less confidence on our side that issues will be fixed quickly. Others in the LLVM community have fixed other such problems. Again, these issues get fixed with new releases (and with some hacking by us in the interim) but the experience makes me very hesitant to make our development depend on trunk. We cannot afford to waste even a few days of interrupted release development work tracking down a new regression introduced in trunk. We also can't afford to have compiler builds break because someone upstream decided to change an API we rely on. We make monthly releases. We have to have some control over when we introduce such changes. We run hundreds of thousands of tests each week, both simple regression tests and large applications. I don't think the nightly tester will ever be capable of covering that. I have a long-term goal of setting up a tree that uses LLVM trunk for our builds both so I can track what's happening on trunk to more easily know what it means for us and to test out some of the new features, like the MachineInstr scheduler in this case. Unfortunately, time is limited so I have to do that piecemeal 10 minutes here and there. The git mirror will make that much easier. -Dave
On May 9, 2012, at 8:34 AM, dag at cray.com wrote:> Andrew Trick <atrick at apple.com> writes: > >>> When I asked about enhancing scheduler heuristics a month or so ago, I >>> got a response about a MachineInstr scheduler and that that was the way >>> of the LLVM future. Is that so? Is the ScheduleDAG going away? >> >> You sent a lengthy RFC on Apr 20 that demonstrated you aren't >> following developments on trunk. That's perfectly fine, but if you >> want to use the new scheduler before it is mature, you'll need to >> follow trunk. > > Ok, but that doesn't answer the question. Is SchedulerDAG going away? > If so, what's the timeframe for that? 3.2?SchedulerDAG is used for both SD scheduling and MI scheduling. It's not going away. SD scheduling is not going away in 3.2--it will be the first release with MI scheduling on by default. If all goes well, I expect SD scheduling to be removed by 3.3. That has not been discussed. Consider this the preliminary announcement. I'll post another announcement as soon as we have something that's more broadly interesting. In the current state it's only interesting for someone just beginning to write their own custom scheduler. Here's a more complete list of the implementation steps, but the real effort will be spent in performance analysis required before flipping the switch. Don't expect it to be an adequate replacement out-of-box for your benchmarks before 3.2. - Target pass configuration: DONE - MachineScheduler pass framework: DONE - MI Scheduling DAG: DONE - AliasAnalysis aware DAG option: In review (Sergei) - Bidirectional list scheduling: DONE - LiveInterval Update: WIP (simple instruction reordering is supported) - Target-independent precise modeling of register pressure: DONE - Register pressure reduction scheduler: WIP - Support for existing HazardChecker plugin - New target description feature: buffered resources - Modeling min latency, expected latency, and resources constraints - Heuristics that balance interlocks, regpressure, latency and buffered resources For targets where scheduling is critical, I encourage developers who stay in sync with trunk to write their own target-specific scheduler based on the pieces that are already available. Hexagon developers are doing this now. The LLVM toolkit for scheduling is all there--not perfect, but ready for developers. - Pluggable MachineScheduler pass - Scheduling DAG - LiveInterval Update - RegisterPressure tracking - InstructionItinerary and HazardChecker (to be extended) If you would simply like improved X86 scheduling without rolling your own, then providing feedback and test cases is useful so we can incorporate improvements into the standard scheduler while it's being developed.>> Feel free to use the patch and send your thanks to Hal. It doesn't >> serve any purpose to mainline a partial solution only to replace it >> before it can ever be enabled by default, which would require a major >> performance investigation and introduces a huge risk (AliasAnalysis in >> CodeGen has not be well tested). > > Er, but as I understand it the MachineInstr scheduler will also use > alias analysis.AliasAnalysis is important. We want it fully supported and enabled by default, but that requires effort beyond simply enabling it. Today, that effort is in the MI scheduler.>> The last thing we want to do is duplicate what's already planned > to happen.The information I provided above is the best I can do, and as early as I could provide this level of detail. If you follow trunk, you can see the direction things are heading, but until recently I would not have been able to tell you "plans" in the form of dates or release goals. -Andy
On Thu, 10 May 2012 20:33:53 -0700 Andrew Trick <atrick at apple.com> wrote:> On May 9, 2012, at 8:34 AM, dag at cray.com wrote: > > > Andrew Trick <atrick at apple.com> writes: > > > >>> When I asked about enhancing scheduler heuristics a month or so > >>> ago, I got a response about a MachineInstr scheduler and that > >>> that was the way of the LLVM future. Is that so? Is the > >>> ScheduleDAG going away? > >> > >> You sent a lengthy RFC on Apr 20 that demonstrated you aren't > >> following developments on trunk. That's perfectly fine, but if you > >> want to use the new scheduler before it is mature, you'll need to > >> follow trunk. > > > > Ok, but that doesn't answer the question. Is SchedulerDAG going > > away? If so, what's the timeframe for that? 3.2? > > SchedulerDAG is used for both SD scheduling and MI scheduling. It's > not going away. > > SD scheduling is not going away in 3.2--it will be the first release > with MI scheduling on by default. > > If all goes well, I expect SD scheduling to be removed by 3.3. That > has not been discussed. > > Consider this the preliminary announcement. I'll post another > announcement as soon as we have something that's more broadly > interesting. In the current state it's only interesting for someone > just beginning to write their own custom scheduler. > > Here's a more complete list of the implementation steps, but the real > effort will be spent in performance analysis required before flipping > the switch. Don't expect it to be an adequate replacement out-of-box > for your benchmarks before 3.2. > > - Target pass configuration: DONE > - MachineScheduler pass framework: DONE > - MI Scheduling DAG: DONE > - AliasAnalysis aware DAG option: In review (Sergei) > - Bidirectional list scheduling: DONE > - LiveInterval Update: WIP (simple instruction reordering is > supported) > - Target-independent precise modeling of register pressure: DONE > - Register pressure reduction scheduler: WIP > - Support for existing HazardChecker pluginIs support for the existing hazard detectors working now? [it does not say DONE or WIP here, but your comment below implies, I think, that it is at least partially working].> - New target description feature: buffered resources > - Modeling min latency, expected latency, and resources constraintsCan you comment on how min and expected latency will be used in the scheduling?> - Heuristics that balance interlocks, regpressure, latency and > buffered resources > > For targets where scheduling is critical, I encourage developers who > stay in sync with trunk to write their own target-specific scheduler > based on the pieces that are already available. Hexagon developers > are doing this now. The LLVM toolkit for scheduling is all there--not > perfect, but ready for developers. > > - Pluggable MachineScheduler pass > - Scheduling DAG > - LiveInterval Update > - RegisterPressure tracking > - InstructionItinerary and HazardChecker (to be extended) > > If you would simply like improved X86 scheduling without rolling your > own, then providing feedback and test cases is useful so we can > incorporate improvements into the standard scheduler while it's being > developed.Does this mean that we're going to see a new X86 scheduling paradigm, or is the existing ILP heuristic, in large part, expected to stay? Thanks again, Hal> > >> Feel free to use the patch and send your thanks to Hal. It doesn't > >> serve any purpose to mainline a partial solution only to replace it > >> before it can ever be enabled by default, which would require a > >> major performance investigation and introduces a huge risk > >> (AliasAnalysis in CodeGen has not be well tested). > > > > Er, but as I understand it the MachineInstr scheduler will also use > > alias analysis. > > AliasAnalysis is important. We want it fully supported and enabled by > default, but that requires effort beyond simply enabling it. Today, > that effort is in the MI scheduler. > > >> The last thing we want to do is duplicate what's already planned > > to happen. > > > The information I provided above is the best I can do, and as early > as I could provide this level of detail. If you follow trunk, you can > see the direction things are heading, but until recently I would not > have been able to tell you "plans" in the form of dates or release > goals. > > -Andy > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev-- Hal Finkel Postdoctoral Appointee Leadership Computing Facility Argonne National Laboratory
Andrew Trick <atrick at apple.com> writes:>> Ok, but that doesn't answer the question. Is SchedulerDAG going away? >> If so, what's the timeframe for that? 3.2? > > SchedulerDAG is used for both SD scheduling and MI scheduling. It's not going away.Oh! That's good news!> SD scheduling is not going away in 3.2--it will be the first release with MI scheduling on by default.Ok, that is helpful. Thanks!> If all goes well, I expect SD scheduling to be removed by 3.3. That has not been discussed.Is there any particular reason to remove it? Something has to convert from SDNodes to MachineInstrs so we'll at least need the "original order" SUnit scheduler, yes?> Here's a more complete list of the implementation steps, but the real > effort will be spent in performance analysis required before flipping > the switch. Don't expect it to be an adequate replacement out-of-box > for your benchmarks before 3.2.Understood.> - AliasAnalysis aware DAG option: In review (Sergei)This is going to be very important to us. I believe this accomplishes the same thing as Hal's patch and work we've done here. I'm really glad to see this here!> If you would simply like improved X86 scheduling without rolling your > own, then providing feedback and test cases is useful so we can > incorporate improvements into the standard scheduler while it's being > developed.Yep.>> Er, but as I understand it the MachineInstr scheduler will also use >> alias analysis. > > AliasAnalysis is important. We want it fully supported and enabled by > default, but that requires effort beyond simply enabling it. Today, > that effort is in the MI scheduler.Yes of course. Does this mean alias analysis in general will be available at the MachineInstr level? I've run into a desire for that multiple times.> The information I provided above is the best I can do, and as early as > I could provide this level of detail. If you follow trunk, you can see > the direction things are heading, but until recently I would not have > been able to tell you "plans" in the form of dates or release goals.This is really helpful, thanks! -Dave