Harry Butterworth
2006-Jun-16 18:49 UTC
[Xen-devel] [PATCH][XM-TEST] Fix bugzilla # 674 by changing the xm-test Console.py to wait for the command prompt.
Lots of xm-test tests are failing with console timeouts on some machines at the moment. I reproduced the problem and found it was due to some probing activity in the boot process of the -xen kernel. The behaviour of the current xm-test code is to assume that the boot process has finished after performing three one-second waits for input---when the probing activity introduces more than three one-second delays the test suite breaks. The patch below changes the test suite to wait for the command prompt before attempting to submit a command. The timeout is increased from 3 seconds to 3 minutes but the wait exits early when the prompt is found so the test suite doesn''t take any longer to run. DO NOT APPLY. This patch is FOR REVIEW ONLY at this stage since my python skills are pretty lame and it really needs to get some testing from people who have actually been experiencing problems. For me, xm-test works the same on my machine with and without this patch. Signed-off-by Harry Butterworth <butterwo@uk.ibm.com> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
David F Barrera
2006-Jun-16 20:06 UTC
[Xen-devel] Re: [PATCH][XM-TEST] Fix bugzilla # 674 by changing the xm-test Console.py to wait for the command prompt.
Harry, I tested your patch on two machines--lots of failures, not sure related to your changes. Attached are the logs. Harry Butterworth wrote:> Lots of xm-test tests are failing with console timeouts on some machines > at the moment. I reproduced the problem and found it was due to some > probing activity in the boot process of the -xen kernel. The behaviour > of the current xm-test code is to assume that the boot process has > finished after performing three one-second waits for input---when the > probing activity introduces more than three one-second delays the test > suite breaks. > > The patch below changes the test suite to wait for the command prompt > before attempting to submit a command. The timeout is increased from 3 > seconds to 3 minutes but the wait exits early when the prompt is found > so the test suite doesn''t take any longer to run. > > DO NOT APPLY. This patch is FOR REVIEW ONLY at this stage since my > python skills are pretty lame and it really needs to get some testing > from people who have actually been experiencing problems. For me, > xm-test works the same on my machine with and without this patch. > > Signed-off-by Harry Butterworth <butterwo@uk.ibm.com> > > ------------------------------------------------------------------------ > > diff -r ee3d10828937 -r 2dcc3cc7118e tools/xm-test/lib/XmTestLib/Console.py > --- a/tools/xm-test/lib/XmTestLib/Console.py Fri Jun 16 13:43:54 2006 > +++ b/tools/xm-test/lib/XmTestLib/Console.py Fri Jun 16 18:28:52 2006 > @@ -82,9 +82,6 @@ > > tty.setraw(self.consoleFd, termios.TCSANOW) > > - self.__chewall(self.consoleFd) > - > - > def __addToHistory(self, line): > self.historyBuffer.append(line) > self.historyLines += 1 > @@ -120,34 +117,47 @@ > output""" > self.PROMPT = prompt > > - > - def __chewall(self, fd): > + def __getprompt(self, fd): > timeout = 0 > - bytes = 0 > - > - while timeout < 3: > - i, o, e = select.select([fd], [], [], 1) > - if fd in i: > - try: > - foo = os.read(fd, 1) > - if self.debugMe: > - sys.stdout.write(foo) > - bytes += 1 > - except Exception, exn: > - raise ConsoleError(str(exn)) > - > - else: > - timeout += 1 > - > - if self.limit and bytes >= self.limit: > + bytes = 0 > + while timeout < 180: > + # eat anything while total bytes less than limit else raise RUNAWAY > + while (not self.limit) or (bytes < self.limit): > + i, o, e = select.select([fd], [], [], 1) > + if fd in i: > + try: > + foo = os.read(fd, 1) > + if self.debugMe: > + sys.stdout.write(foo) > + bytes += 1 > + except Exception, exn: > + raise ConsoleError(str(exn)) > + else: > + break > + else: > raise ConsoleError("Console run-away (exceeded %i bytes)" > % self.limit, RUNAWAY) > - > - if self.debugMe: > - print "Ignored %i bytes of miscellaneous console output" % bytes > - > - return bytes > - > + # press enter > + os.write(self.consoleFd, "\n") > + # look for prompt > + for prompt_char in "\r\n" + self.PROMPT: > + i, o, e = select.select([fd], [], [], 1) > + if fd in i: > + try: > + foo = os.read(fd, 1) > + if self.debugMe: > + sys.stdout.write(foo) > + if foo != prompt_char: > + break > + except Exception, exn: > + raise ConsoleError(str(exn)) > + else: > + timeout += 1 > + break > + else: > + break > + else: > + raise ConsoleError("Timed out waiting for console prompt") > > def __runCmd(self, command, saveHistory=True): > output = "" > @@ -155,7 +165,7 @@ > lines = 0 > bytes = 0 > > - self.__chewall(self.consoleFd) > + self.__getprompt(self.consoleFd) > > if verbose: > print "[%s] Sending `%s''" % (self.domain, command) > @@ -176,7 +186,7 @@ > "Failed to read from console (fd=%i): %s" % > (self.consoleFd, exn)) > else: > - raise ConsoleError("Timed out waiting for console") > + raise ConsoleError("Timed out waiting for console command") > > if self.limit and bytes >= self.limit: > raise ConsoleError("Console run-away (exceeded %i bytes)" >-- Regards, David F Barrera Linux Technology Center Systems and Technology Group, IBM "The wisest men follow their own direction. " Euripides _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
David F Barrera
2006-Jun-16 20:35 UTC
Re: [Xen-devel] [PATCH][XM-TEST] Fix bugzilla # 674 by changing the xm-test Console.py to wait for the command prompt.
Harry, I tested your patch. I would like you to look at the logs as there are lots of failures. Please contact me and I will tell you where to get the logs (I tried sending them, but the file is probably too big). Harry Butterworth wrote:> Lots of xm-test tests are failing with console timeouts on some machines > at the moment. I reproduced the problem and found it was due to some > probing activity in the boot process of the -xen kernel. The behaviour > of the current xm-test code is to assume that the boot process has > finished after performing three one-second waits for input---when the > probing activity introduces more than three one-second delays the test > suite breaks. > > The patch below changes the test suite to wait for the command prompt > before attempting to submit a command. The timeout is increased from 3 > seconds to 3 minutes but the wait exits early when the prompt is found > so the test suite doesn''t take any longer to run. > > DO NOT APPLY. This patch is FOR REVIEW ONLY at this stage since my > python skills are pretty lame and it really needs to get some testing > from people who have actually been experiencing problems. For me, > xm-test works the same on my machine with and without this patch. > > Signed-off-by Harry Butterworth <butterwo@uk.ibm.com> > > ------------------------------------------------------------------------ > > diff -r ee3d10828937 -r 2dcc3cc7118e tools/xm-test/lib/XmTestLib/Console.py > --- a/tools/xm-test/lib/XmTestLib/Console.py Fri Jun 16 13:43:54 2006 > +++ b/tools/xm-test/lib/XmTestLib/Console.py Fri Jun 16 18:28:52 2006 > @@ -82,9 +82,6 @@ > > tty.setraw(self.consoleFd, termios.TCSANOW) > > - self.__chewall(self.consoleFd) > - > - > def __addToHistory(self, line): > self.historyBuffer.append(line) > self.historyLines += 1 > @@ -120,34 +117,47 @@ > output""" > self.PROMPT = prompt > > - > - def __chewall(self, fd): > + def __getprompt(self, fd): > timeout = 0 > - bytes = 0 > - > - while timeout < 3: > - i, o, e = select.select([fd], [], [], 1) > - if fd in i: > - try: > - foo = os.read(fd, 1) > - if self.debugMe: > - sys.stdout.write(foo) > - bytes += 1 > - except Exception, exn: > - raise ConsoleError(str(exn)) > - > - else: > - timeout += 1 > - > - if self.limit and bytes >= self.limit: > + bytes = 0 > + while timeout < 180: > + # eat anything while total bytes less than limit else raise RUNAWAY > + while (not self.limit) or (bytes < self.limit): > + i, o, e = select.select([fd], [], [], 1) > + if fd in i: > + try: > + foo = os.read(fd, 1) > + if self.debugMe: > + sys.stdout.write(foo) > + bytes += 1 > + except Exception, exn: > + raise ConsoleError(str(exn)) > + else: > + break > + else: > raise ConsoleError("Console run-away (exceeded %i bytes)" > % self.limit, RUNAWAY) > - > - if self.debugMe: > - print "Ignored %i bytes of miscellaneous console output" % bytes > - > - return bytes > - > + # press enter > + os.write(self.consoleFd, "\n") > + # look for prompt > + for prompt_char in "\r\n" + self.PROMPT: > + i, o, e = select.select([fd], [], [], 1) > + if fd in i: > + try: > + foo = os.read(fd, 1) > + if self.debugMe: > + sys.stdout.write(foo) > + if foo != prompt_char: > + break > + except Exception, exn: > + raise ConsoleError(str(exn)) > + else: > + timeout += 1 > + break > + else: > + break > + else: > + raise ConsoleError("Timed out waiting for console prompt") > > def __runCmd(self, command, saveHistory=True): > output = "" > @@ -155,7 +165,7 @@ > lines = 0 > bytes = 0 > > - self.__chewall(self.consoleFd) > + self.__getprompt(self.consoleFd) > > if verbose: > print "[%s] Sending `%s''" % (self.domain, command) > @@ -176,7 +186,7 @@ > "Failed to read from console (fd=%i): %s" % > (self.consoleFd, exn)) > else: > - raise ConsoleError("Timed out waiting for console") > + raise ConsoleError("Timed out waiting for console command") > > if self.limit and bytes >= self.limit: > raise ConsoleError("Console run-away (exceeded %i bytes)" > > ------------------------------------------------------------------------ > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- Regards, David F Barrera Linux Technology Center Systems and Technology Group, IBM "The wisest men follow their own direction. " Euripides _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Harry Butterworth
2006-Jun-18 23:02 UTC
[Xen-devel] Re: [PATCH][XM-TEST] Fix bugzilla # 674 by changing the xm-test Console.py to wait for the command prompt.
On Fri, 2006-06-16 at 15:06 -0500, David F Barrera wrote:> Harry, > > I tested your patch on two machines--lots of failures, not sure related > to your changes. Attached are the logs.Quite a lot of the failures are from Thursday which was before I wrote the patch so I think there is a good chance that those are not related. Of the remainder, I there is one instance of a "timed out waiting for console prompt" which is the new timeout in my patch but this is for the memset_random test which fails for me for a different reason so the timeout is probably correct in this case. The remainder seem to be completely unrelated. So, I think my patch is probably good and the rest of your problems are probably just more issues in the xm-test code or possibly real bugs. Harry _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
David F Barrera
2006-Jun-19 16:17 UTC
Re: [Xen-devel] [PATCH][XM-TEST] Fix bugzilla # 674 by changing the xm-test Console.py to wait for the command prompt.
Harry, I tested your patch against the same code base as with the xm-test ran without your patch. The results look much better. Attached are the xm-test reports from both runs, with and without your patch, on two machines, one a SLES 9 based x335, and the other a RHEL 4 based HS20 Blade running a 32-bit OS. Harry Butterworth wrote:> Lots of xm-test tests are failing with console timeouts on some machines > at the moment. I reproduced the problem and found it was due to some > probing activity in the boot process of the -xen kernel. The behaviour > of the current xm-test code is to assume that the boot process has > finished after performing three one-second waits for input---when the > probing activity introduces more than three one-second delays the test > suite breaks. > > The patch below changes the test suite to wait for the command prompt > before attempting to submit a command. The timeout is increased from 3 > seconds to 3 minutes but the wait exits early when the prompt is found > so the test suite doesn''t take any longer to run. > > DO NOT APPLY. This patch is FOR REVIEW ONLY at this stage since my > python skills are pretty lame and it really needs to get some testing > from people who have actually been experiencing problems. For me, > xm-test works the same on my machine with and without this patch. > > Signed-off-by Harry Butterworth <butterwo@uk.ibm.com> > > ------------------------------------------------------------------------ > > diff -r ee3d10828937 -r 2dcc3cc7118e tools/xm-test/lib/XmTestLib/Console.py > --- a/tools/xm-test/lib/XmTestLib/Console.py Fri Jun 16 13:43:54 2006 > +++ b/tools/xm-test/lib/XmTestLib/Console.py Fri Jun 16 18:28:52 2006 > @@ -82,9 +82,6 @@ > > tty.setraw(self.consoleFd, termios.TCSANOW) > > - self.__chewall(self.consoleFd) > - > - > def __addToHistory(self, line): > self.historyBuffer.append(line) > self.historyLines += 1 > @@ -120,34 +117,47 @@ > output""" > self.PROMPT = prompt > > - > - def __chewall(self, fd): > + def __getprompt(self, fd): > timeout = 0 > - bytes = 0 > - > - while timeout < 3: > - i, o, e = select.select([fd], [], [], 1) > - if fd in i: > - try: > - foo = os.read(fd, 1) > - if self.debugMe: > - sys.stdout.write(foo) > - bytes += 1 > - except Exception, exn: > - raise ConsoleError(str(exn)) > - > - else: > - timeout += 1 > - > - if self.limit and bytes >= self.limit: > + bytes = 0 > + while timeout < 180: > + # eat anything while total bytes less than limit else raise RUNAWAY > + while (not self.limit) or (bytes < self.limit): > + i, o, e = select.select([fd], [], [], 1) > + if fd in i: > + try: > + foo = os.read(fd, 1) > + if self.debugMe: > + sys.stdout.write(foo) > + bytes += 1 > + except Exception, exn: > + raise ConsoleError(str(exn)) > + else: > + break > + else: > raise ConsoleError("Console run-away (exceeded %i bytes)" > % self.limit, RUNAWAY) > - > - if self.debugMe: > - print "Ignored %i bytes of miscellaneous console output" % bytes > - > - return bytes > - > + # press enter > + os.write(self.consoleFd, "\n") > + # look for prompt > + for prompt_char in "\r\n" + self.PROMPT: > + i, o, e = select.select([fd], [], [], 1) > + if fd in i: > + try: > + foo = os.read(fd, 1) > + if self.debugMe: > + sys.stdout.write(foo) > + if foo != prompt_char: > + break > + except Exception, exn: > + raise ConsoleError(str(exn)) > + else: > + timeout += 1 > + break > + else: > + break > + else: > + raise ConsoleError("Timed out waiting for console prompt") > > def __runCmd(self, command, saveHistory=True): > output = "" > @@ -155,7 +165,7 @@ > lines = 0 > bytes = 0 > > - self.__chewall(self.consoleFd) > + self.__getprompt(self.consoleFd) > > if verbose: > print "[%s] Sending `%s''" % (self.domain, command) > @@ -176,7 +186,7 @@ > "Failed to read from console (fd=%i): %s" % > (self.consoleFd, exn)) > else: > - raise ConsoleError("Timed out waiting for console") > + raise ConsoleError("Timed out waiting for console command") > > if self.limit and bytes >= self.limit: > raise ConsoleError("Console run-away (exceeded %i bytes)" > > ------------------------------------------------------------------------ > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- Regards, David F Barrera Linux Technology Center Systems and Technology Group, IBM "The wisest men follow their own direction. " Euripides _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Harry Butterworth
2006-Jun-19 16:36 UTC
[Xen-devel] [PATCH][XM-TEST] Fix bugzilla # 674 by changing the xm-test Console.py to wait for the command prompt.
On Fri, 2006-06-16 at 19:49 +0100, Harry Butterworth wrote:> Lots of xm-test tests are failing with console timeouts on some machines > at the moment. I reproduced the problem and found it was due to some > probing activity in the boot process of the -xen kernel. The behaviour > of the current xm-test code is to assume that the boot process has > finished after performing three one-second waits for input---when the > probing activity introduces more than three one-second delays the test > suite breaks. > > The patch below changes the test suite to wait for the command prompt > before attempting to submit a command. The timeout is increased from 3 > seconds to 3 minutes but the wait exits early when the prompt is found > so the test suite doesn''t take any longer to run.This patch has now had adequate testing and David Barrera has shown that it fixes a substantial chunk of the current breakage in xm-test. Signed-off-by Harry Butterworth <butterwo@uk.ibm.com> Please apply. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Reasonably Related Threads
- [PATCH] [Xm-Test] A test case doing local migration 3 times in a loop
- [PATCH] nr_cpus calculation problem due to incorrect sockets_per_node
- RE: [RFC][PATCH] create an initrd for dom0 in install.shscript
- Bug in use of grant tables in blkback.c error path?
- withr::set_makevars