llvm dev - Oct 2016 - [test-suite] making polybench/symm succeed with "-Ofast" and "-ffp-contract=on"

On 14 October 2016 at 15:50, Sebastian Pop <sebpop.llvm at gmail.com>
wrote:
> These 3 tests are passing with the following configurations:
> -O3 -ffp-contract=off
> -O3 -ffp-contract=on
> -O0 -ffp-contract=off
> -O0 -ffp-contract=on
>
> They are not passing at:
> -Ofast -ffp-contract=on
> -Ofast -ffp-contract=off
Let's separate completely FP-contract and fast-math. They're different
things and need different solutions.

> if(TEST_SUITE_USES_FAST_MATH)
>   add_definitions(-DFP_ABSTOLERANCE=1e0)
> else()
>   add_definitions(-DFP_ABSTOLERANCE=1e-5)
> endif()
This doesn't make sense. If my program decreased precision by 5 orders
of magnitude with -ffast-math, I'd be *very* worried.

I hope that fast-math in Clang isn't that broken, so that's probably
to do with the assumptions in the output reduction phase.

But, as I said, let's do Ofast *later*. One thing at a time.

cheers,
--renato

----- Original Message -----
> From: "Renato Golin" <renato.golin at linaro.org>
> To: "Sebastian Pop" <sebpop.llvm at gmail.com>
> Cc: "Hal Finkel" <hfinkel at anl.gov>, "Sebastian Paul
Pop" <s.pop at samsung.com>, "llvm-dev" <llvm-dev at
lists.llvm.org>,
> "Matthias Braun" <matze at braunis.de>, "Clang
Dev" <cfe-dev at lists.llvm.org>, "nd" <nd at
arm.com>, "Abe Skolnik"
> <a.skolnik at samsung.com>
> Sent: Friday, October 14, 2016 10:36:12 AM
> Subject: Re: [test-suite] making polybench/symm succeed with
"-Ofast" and "-ffp-contract=on"
> 
> On 14 October 2016 at 15:50, Sebastian Pop <sebpop.llvm at gmail.com>
> wrote:
> > These 3 tests are passing with the following configurations:
> > -O3 -ffp-contract=off
> > -O3 -ffp-contract=on
> > -O0 -ffp-contract=off
> > -O0 -ffp-contract=on
> >
> > They are not passing at:
> > -Ofast -ffp-contract=on
> > -Ofast -ffp-contract=off
> 
> Let's separate completely FP-contract and fast-math. They're
> different
> things and need different solutions.
> 
> 
> > if(TEST_SUITE_USES_FAST_MATH)
> >   add_definitions(-DFP_ABSTOLERANCE=1e0)
> > else()
> >   add_definitions(-DFP_ABSTOLERANCE=1e-5)
> > endif()
> 
> This doesn't make sense. If my program decreased precision by 5
> orders
> of magnitude with -ffast-math, I'd be *very* worried.
Is this an absolute or a relative tolerance? If this is absolute, as the name
implies, then I'm not sure how to feel. We should probably use a relative
tolerance where possible.

 -Hal
> 
> I hope that fast-math in Clang isn't that broken, so that's
probably
> to do with the assumptions in the output reduction phase.
> 
> But, as I said, let's do Ofast *later*. One thing at a time.
> 
> cheers,
> --renato
> 
-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory

On Fri, Oct 14, 2016 at 10:50 AM, Hal Finkel <hfinkel at anl.gov>
wrote:
> ----- Original Message -----
>> From: "Renato Golin" <renato.golin at linaro.org>
>> To: "Sebastian Pop" <sebpop.llvm at gmail.com>
>> Cc: "Hal Finkel" <hfinkel at anl.gov>, "Sebastian
Paul Pop" <s.pop at samsung.com>, "llvm-dev" <llvm-dev
at lists.llvm.org>,
>> "Matthias Braun" <matze at braunis.de>, "Clang
Dev" <cfe-dev at lists.llvm.org>, "nd" <nd at
arm.com>, "Abe Skolnik"
>> <a.skolnik at samsung.com>
>> Sent: Friday, October 14, 2016 10:36:12 AM
>> Subject: Re: [test-suite] making polybench/symm succeed with
"-Ofast" and "-ffp-contract=on"
>>
>> On 14 October 2016 at 15:50, Sebastian Pop <sebpop.llvm at
gmail.com>
>> wrote:
>> > These 3 tests are passing with the following configurations:
>> > -O3 -ffp-contract=off
>> > -O3 -ffp-contract=on
>> > -O0 -ffp-contract=off
>> > -O0 -ffp-contract=on
>> >
>> > They are not passing at:
>> > -Ofast -ffp-contract=on
>> > -Ofast -ffp-contract=off
>>
>> Let's separate completely FP-contract and fast-math. They're
>> different
>> things and need different solutions.
>>
>>
>> > if(TEST_SUITE_USES_FAST_MATH)
>> >   add_definitions(-DFP_ABSTOLERANCE=1e0)
>> > else()
>> >   add_definitions(-DFP_ABSTOLERANCE=1e-5)
>> > endif()
>>
>> This doesn't make sense. If my program decreased precision by 5
>> orders
>> of magnitude with -ffast-math, I'd be *very* worried.
>
> Is this an absolute or a relative tolerance? If this is absolute, as the
name implies, then I'm not sure how to feel. We should probably use a
relative tolerance where possible.
>
This is absolute tolerance: here is the code I'm using to check.

        double V1 = A[i][j];
        double V2 = B[i][j];
        double Diff = fabs(V1 - V2);
        if (Diff > AbsTolerance) {
          fprintf(stderr, "A[%d][%d] = %lf and B[%d][%d] = %lf differ more
than"
                  " FP_ABSTOLERANCE = %lf\n", i, j, V1, i, j, V2,
AbsTolerance);
          return 0;
        }

Do you want me to add the relative tolerance for the 2 tests?
polybench/linear-algebra/kernels/symm, FP_ABSTOLERANCE=1e1
polybench/linear-algebra/solvers/gramschmidt, FP_ABSTOLERANCE=1e0
What should be a good relative tolerance to set for these two tests?

Here is the code to compute the relative tolerance from fpcmp:

  // Check to see if these are inside the absolute tolerance
  if (AbsTolerance < fabs(V1-V2)) {
    // Nope, check the relative tolerance...
    double Diff;
    if (V2)
      Diff = fabs(V1/V2 - 1.0);
    else if (V1)
      Diff = fabs(V2/V1 - 1.0);
    else
      Diff = 0;  // Both zero.
    if (Diff > RelTolerance) {
      fprintf(stderr, ("%s: Compared: %e and %e\n"
                       "abs. diff = %e rel.diff = %e\n"
                       "Out of tolerance: rel/abs: %e/%e\n"),
              g_program, V1, V2, fabs(V1-V2), Diff, RelTolerance, AbsTolerance);
      return true;
    }
  }
>  -Hal
>
>>
>> I hope that fast-math in Clang isn't that broken, so that's
probably
>> to do with the assumptions in the output reduction phase.
>>
>> But, as I said, let's do Ofast *later*. One thing at a time.
Agreed. I will update the patch, make sure all Polybench pass with
"-O3 -ffp-contract=on/off" and it will be ready to commit.

Thanks,
Sebastian