llvm dev - Oct 2016 - [test-suite] making polybench/symm succeed with "-Ofast" and "-ffp-contract=on"

On Fri, Oct 14, 2016 at 10:50 AM, Hal Finkel <hfinkel at anl.gov>
wrote:
> ----- Original Message -----
>> From: "Renato Golin" <renato.golin at linaro.org>
>> To: "Sebastian Pop" <sebpop.llvm at gmail.com>
>> Cc: "Hal Finkel" <hfinkel at anl.gov>, "Sebastian
Paul Pop" <s.pop at samsung.com>, "llvm-dev" <llvm-dev
at lists.llvm.org>,
>> "Matthias Braun" <matze at braunis.de>, "Clang
Dev" <cfe-dev at lists.llvm.org>, "nd" <nd at
arm.com>, "Abe Skolnik"
>> <a.skolnik at samsung.com>
>> Sent: Friday, October 14, 2016 10:36:12 AM
>> Subject: Re: [test-suite] making polybench/symm succeed with
"-Ofast" and "-ffp-contract=on"
>>
>> On 14 October 2016 at 15:50, Sebastian Pop <sebpop.llvm at
gmail.com>
>> wrote:
>> > These 3 tests are passing with the following configurations:
>> > -O3 -ffp-contract=off
>> > -O3 -ffp-contract=on
>> > -O0 -ffp-contract=off
>> > -O0 -ffp-contract=on
>> >
>> > They are not passing at:
>> > -Ofast -ffp-contract=on
>> > -Ofast -ffp-contract=off
>>
>> Let's separate completely FP-contract and fast-math. They're
>> different
>> things and need different solutions.
>>
>>
>> > if(TEST_SUITE_USES_FAST_MATH)
>> >   add_definitions(-DFP_ABSTOLERANCE=1e0)
>> > else()
>> >   add_definitions(-DFP_ABSTOLERANCE=1e-5)
>> > endif()
>>
>> This doesn't make sense. If my program decreased precision by 5
>> orders
>> of magnitude with -ffast-math, I'd be *very* worried.
>
> Is this an absolute or a relative tolerance? If this is absolute, as the
name implies, then I'm not sure how to feel. We should probably use a
relative tolerance where possible.
>
This is absolute tolerance: here is the code I'm using to check.

        double V1 = A[i][j];
        double V2 = B[i][j];
        double Diff = fabs(V1 - V2);
        if (Diff > AbsTolerance) {
          fprintf(stderr, "A[%d][%d] = %lf and B[%d][%d] = %lf differ more
than"
                  " FP_ABSTOLERANCE = %lf\n", i, j, V1, i, j, V2,
AbsTolerance);
          return 0;
        }

Do you want me to add the relative tolerance for the 2 tests?
polybench/linear-algebra/kernels/symm, FP_ABSTOLERANCE=1e1
polybench/linear-algebra/solvers/gramschmidt, FP_ABSTOLERANCE=1e0
What should be a good relative tolerance to set for these two tests?

Here is the code to compute the relative tolerance from fpcmp:

  // Check to see if these are inside the absolute tolerance
  if (AbsTolerance < fabs(V1-V2)) {
    // Nope, check the relative tolerance...
    double Diff;
    if (V2)
      Diff = fabs(V1/V2 - 1.0);
    else if (V1)
      Diff = fabs(V2/V1 - 1.0);
    else
      Diff = 0;  // Both zero.
    if (Diff > RelTolerance) {
      fprintf(stderr, ("%s: Compared: %e and %e\n"
                       "abs. diff = %e rel.diff = %e\n"
                       "Out of tolerance: rel/abs: %e/%e\n"),
              g_program, V1, V2, fabs(V1-V2), Diff, RelTolerance, AbsTolerance);
      return true;
    }
  }
>  -Hal
>
>>
>> I hope that fast-math in Clang isn't that broken, so that's
probably
>> to do with the assumptions in the output reduction phase.
>>
>> But, as I said, let's do Ofast *later*. One thing at a time.
Agreed. I will update the patch, make sure all Polybench pass with
"-O3 -ffp-contract=on/off" and it will be ready to commit.

Thanks,
Sebastian

----- Original Message -----
> From: "Sebastian Pop" <sebpop.llvm at gmail.com>
> To: "Hal Finkel" <hfinkel at anl.gov>
> Cc: "Renato Golin" <renato.golin at linaro.org>,
"Sebastian Paul Pop" <s.pop at samsung.com>,
"llvm-dev"
> <llvm-dev at lists.llvm.org>, "Matthias Braun" <matze at
braunis.de>, "Clang Dev" <cfe-dev at lists.llvm.org>,
"nd"
> <nd at arm.com>, "Abe Skolnik" <a.skolnik at
samsung.com>
> Sent: Friday, October 14, 2016 5:05:01 PM
> Subject: Re: [test-suite] making polybench/symm succeed with
"-Ofast" and "-ffp-contract=on"
> 
> On Fri, Oct 14, 2016 at 10:50 AM, Hal Finkel <hfinkel at anl.gov>
wrote:
> > ----- Original Message -----
> >> From: "Renato Golin" <renato.golin at linaro.org>
> >> To: "Sebastian Pop" <sebpop.llvm at gmail.com>
> >> Cc: "Hal Finkel" <hfinkel at anl.gov>,
"Sebastian Paul Pop"
> >> <s.pop at samsung.com>, "llvm-dev" <llvm-dev at
lists.llvm.org>,
> >> "Matthias Braun" <matze at braunis.de>,
"Clang Dev"
> >> <cfe-dev at lists.llvm.org>, "nd" <nd at
arm.com>, "Abe Skolnik"
> >> <a.skolnik at samsung.com>
> >> Sent: Friday, October 14, 2016 10:36:12 AM
> >> Subject: Re: [test-suite] making polybench/symm succeed with
> >> "-Ofast" and "-ffp-contract=on"
> >>
> >> On 14 October 2016 at 15:50, Sebastian Pop <sebpop.llvm at
gmail.com>
> >> wrote:
> >> > These 3 tests are passing with the following configurations:
> >> > -O3 -ffp-contract=off
> >> > -O3 -ffp-contract=on
> >> > -O0 -ffp-contract=off
> >> > -O0 -ffp-contract=on
> >> >
> >> > They are not passing at:
> >> > -Ofast -ffp-contract=on
> >> > -Ofast -ffp-contract=off
> >>
> >> Let's separate completely FP-contract and fast-math.
They're
> >> different
> >> things and need different solutions.
> >>
> >>
> >> > if(TEST_SUITE_USES_FAST_MATH)
> >> >   add_definitions(-DFP_ABSTOLERANCE=1e0)
> >> > else()
> >> >   add_definitions(-DFP_ABSTOLERANCE=1e-5)
> >> > endif()
> >>
> >> This doesn't make sense. If my program decreased precision by
5
> >> orders
> >> of magnitude with -ffast-math, I'd be *very* worried.
> >
> > Is this an absolute or a relative tolerance? If this is absolute,
> > as the name implies, then I'm not sure how to feel. We should
> > probably use a relative tolerance where possible.
> >
> 
> This is absolute tolerance: here is the code I'm using to check.
> 
>         double V1 = A[i][j];
>         double V2 = B[i][j];
>         double Diff = fabs(V1 - V2);
>         if (Diff > AbsTolerance) {
>           fprintf(stderr, "A[%d][%d] = %lf and B[%d][%d] = %lf differ
>           more than"
>                   " FP_ABSTOLERANCE = %lf\n", i, j, V1, i, j, V2,
>                   AbsTolerance);
>           return 0;
>         }
> 
> Do you want me to add the relative tolerance for the 2 tests?
> polybench/linear-algebra/kernels/symm, FP_ABSTOLERANCE=1e1
> polybench/linear-algebra/solvers/gramschmidt, FP_ABSTOLERANCE=1e0
> What should be a good relative tolerance to set for these two tests?
What's the minimum relative tolerance that you need for them to pass?

 -Hal
> 
> Here is the code to compute the relative tolerance from fpcmp:
> 
>   // Check to see if these are inside the absolute tolerance
>   if (AbsTolerance < fabs(V1-V2)) {
>     // Nope, check the relative tolerance...
>     double Diff;
>     if (V2)
>       Diff = fabs(V1/V2 - 1.0);
>     else if (V1)
>       Diff = fabs(V2/V1 - 1.0);
>     else
>       Diff = 0;  // Both zero.
>     if (Diff > RelTolerance) {
>       fprintf(stderr, ("%s: Compared: %e and %e\n"
>                        "abs. diff = %e rel.diff = %e\n"
>                        "Out of tolerance: rel/abs: %e/%e\n"),
>               g_program, V1, V2, fabs(V1-V2), Diff, RelTolerance,
>               AbsTolerance);
>       return true;
>     }
>   }
> 
> >  -Hal
> >
> >>
> >> I hope that fast-math in Clang isn't that broken, so
that's
> >> probably
> >> to do with the assumptions in the output reduction phase.
> >>
> >> But, as I said, let's do Ofast *later*. One thing at a time.
> 
> Agreed. I will update the patch, make sure all Polybench pass with
> "-O3 -ffp-contract=on/off" and it will be ready to commit.
> 
> Thanks,
> Sebastian
> 
-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

On Fri, Oct 14, 2016 at 5:05 PM, Sebastian Pop <sebpop.llvm at gmail.com>
wrote:
> Agreed. I will update the patch, make sure all Polybench pass with
> "-O3 -ffp-contract=on/off" and it will be ready to commit.
I updated the patch https://reviews.llvm.org/D25346
Ok to commit as it is now, and then address the other problems with
-ffast-math in followup patches?

Thanks,
Sebastian

On Fri, Oct 14, 2016 at 6:10 PM, Hal Finkel <hfinkel at anl.gov>
wrote:
>> polybench/linear-algebra/kernels/symm, FP_ABSTOLERANCE=1e1
>> polybench/linear-algebra/solvers/gramschmidt, FP_ABSTOLERANCE=1e0
>> What should be a good relative tolerance to set for these two tests?
>
> What's the minimum relative tolerance that you need for them to pass?
Setting FP_ABSTOLERANCE=1e-5, the two tests are passing
when compiled with -Ofast for the following relative tolerance:

polybench/linear-algebra/kernels/symm, FP_TOLERANCE=1e-10
polybench/linear-algebra/solvers/gramschmidt, FP_TOLERANCE=1e5

gramschmidt passes validation against reference output,
symm does not pass validation against reference output:
I have opened a bug for that:
"__attribute__((optnone)) not respected with -ffast-math or -Ofast"
https://llvm.org/bugs/show_bug.cgi?id=30745

Sebastian