openssh unix dev - Apr 2009 - Improving Testing Infrastructure using an abstraction driven Framework - GSOC

Hi.

The following is my idea for Improving Testing Infrastructure for the Google
Summer of Code Program. Please do let me know if anything is vague or
anything needs to be improved.
*
A framework which provides abstractions for the different protocols that an
application uses, is always helpful in a testing process since a Test
Engineer need not bother about the implementation of the Protocol. Such a
framework can be useful for fuzzy testing, since the program for generating
the test cases will be greatly simplified. An additional library which may
be included during testing time only, can be useful to deal with the code
coverage issue.*

Thanks and regards.
Meher Anand

Hi Meher,

Meher Anand wrote:
> The following is my idea for Improving Testing Infrastructure for
> the Google Summer of Code Program. Please do let me know if
> anything is vague or anything needs to be improved.
You write "a framework" and "an additional library" but
nothing
really concrete?

How would it work? What testing problems would it solve? How would it
relate to the existing testing infrastructure? Would it replace it?
If so, how could the existing work be reused, if at all? If it would
run in parallell, could there still be some reuse? Etc etc.


//Peter

Hi.

I will be writing concrete code that will be using the libraries that I
intend to develop. The intention of using words like framework and library
is to emphasize on their re-usability. I havent yet completely thought out
about how I would relate it to the existing testing framework, but mostly it
is going to run in parallel.

The library for testing code coverage will be an additional header file
include that the tester would need to add to the existing code, along with a
few lines of code in the main() function that will call the functions in the
library. This library would monitor the coverage of the code, like the
function being called, the condition being executed etc. One way to
implement this would be to monitor the stack for all function calls and
conditions executed. The tester would  to create an array of all the
functions and if conditions in each file. I couldnt think of a better method
as of now.

And the framework that I mentioned about will be doing protocol level
testing. For eg:- To test the SSL protocol, there will be a few calls to the
SSL library of Open SSH. This set of calls will be passing a few parameters
to my functions and test its output, to see if it is returning the exact
values. This library, to test the SSL protocol, will be a part of the
framework along with libraries for testing the other protocols.

Each library in this framework will be independent of the other libraries in
the framework. When one needs to test one particular protocol, he will
include only those headers that are relevant to it. Thus, a protocol level
testing procedure is possible. Each of these libraries can be modified or
extended depending on the needs.

The operations in the above library will be called by functions which have
code for random string generation to generate random test inputs. This will
take care of the fuzzy testing part.

Please let me know if I need to detail on any of the points.

Thanks and regards.
Meher Anand

On Thu, Apr 2, 2009 at 1:29 AM, Peter Stuge <peter at stuge.se> wrote:
> Hi Meher,
>
> Meher Anand wrote:
> > The following is my idea for Improving Testing Infrastructure for
> > the Google Summer of Code Program. Please do let me know if
> > anything is vague or anything needs to be improved.
>
> You write "a framework" and "an additional library" but
nothing
> really concrete?
>
> How would it work? What testing problems would it solve? How would it
> relate to the existing testing infrastructure? Would it replace it?
> If so, how could the existing work be reused, if at all? If it would
> run in parallell, could there still be some reuse? Etc etc.
>
>
> //Peter
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev at mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>