Skip to content

Commit

Permalink
runtest: Treat test timeouts as fatal
Browse files Browse the repository at this point in the history
If a test times out, that means it hasn't returned a prompt within
reasonable time.  Continuing to type at it is unlikely to be helpful,
since the lack of a prompt indicates that the subprocess has hung, so
subsequent tests are likely to time out as well.

This is particularly annoying in step 5, where the line that usually
times out, "(def! res2 (sum2 10000 0))" has its output ignored, so a
timeout is treated as success, meaning that runtest proceeds to the next
test, which of course also times out.
  • Loading branch information
bjh21 committed Jun 1, 2020
1 parent 13404d9 commit 9d7fb0f
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion runtest.py
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,9 @@ def assert_prompt(runner, prompts, timeout):
soft_fail_cnt = 0
failures = []

class TestTimeout(Exception):
pass

while t.next():
if args.deferrable == False and t.deferrable:
log(t.deferrable)
Expand Down Expand Up @@ -293,7 +296,10 @@ def assert_prompt(runner, prompts, timeout):
res = r.read_to_prompt(['\r\n[^\s()<>]+> ', '\n[^\s()<>]+> '],
timeout=args.test_timeout)
#print "%s,%s,%s" % (idx, repr(p.before), repr(p.after))
if (t.ret == "" and t.out == ""):
if (res == None):
log(" -> TIMEOUT (line %d)" % t.line_num)
raise TestTimeout("TIMEOUT (line %d)" % t.line_num)
elif (t.ret == "" and t.out == ""):
log(" -> SUCCESS (result ignored)")
pass_cnt += 1
elif (re.search(expects[0], res, re.S) or
Expand Down

0 comments on commit 9d7fb0f

Please sign in to comment.