"Autograding in progress" never completes

[magsol]

Your environment
Let us know details such as

• AutoLab at University of Georgia
• Ubuntu 20.04
• Latest version of AutoLab (3.0.2)

Steps To Reproduce
Make a submission to the autograder.

Current behavior
Autograder technically "finishes" but scores of 0 are posted and some UI elements do not seem to update to recognize that the job has completed.

Also, the job feedback page never updates with the final score, despite the job showing as "Finished" on the Jobs page.

Strangest of all, in the Tango courselabs folder, the output for the job in the text file does show the autoresult, but it's not reflected anywhere else that I can find in AutoLab.

The scoreboard also remains blank, with the message "There are currently no submissions" despite that being objectively incorrect.

I can't find any logs or errors that would indicate a problem. I double-checked that the problem name in AutoLab matches that in the autoresult, but otherwise I am at a loss. Any help would be greatly appreciated.

[magsol]

To provide some additional clarification / context here:

Everywhere in the AutoLab UI (that I can tell), it lists these submitted jobs as either "successfully complete" or as zeros.

For instance, in the "Jobs" page, the submissions look complete:

And then from the "Manage Submissions" and "Gradesheet" pages, they look as if they've all received a grade of 0 (or just... blank?).

If I click on any of these "Completed" jobs to see the details, the summary suggests that everything completed successfully. The runtime trace seems to show everything working fine, too:

    2025-01-28 11:33:42 -0500 | Added job [email protected]:10 to queue
    2025-01-28 11:33:42 -0500 | Dispatched job [email protected]:10 [try 0]
    2025-01-28 11:33:42 -0500 | Assigned job [email protected]:10 existing VM prod-1001-autograding_image
    2025-01-28 11:33:42 -0500 | Job [email protected]:10 waiting for VM prod-1001-autograding_image
    2025-01-28 11:33:42 -0500 | VM prod-1001-autograding_image ready for job [email protected]:10
    2025-01-28 11:33:42 -0500 | Input copied for job [email protected]:10 [status=0]
    2025-01-28 11:33:44 -0500 | Job [email protected]:10 executed [status=0]
    2025-01-28 11:33:45 -0500 | Output copied for job [email protected]:10 [status=0]
    2025-01-28 11:33:45 -0500 | Success: Autodriver returned normally

But there is no feedback provided; it's just blank. This despite the fact that I've gone into the Tango docker container, opened up the text files created in the job traces (the Output copied for job [email protected] step), and in those files I'll see the JSON autoresults for successful runs, and error messages for runs that crashed, but none of this information makes it back to AutoLab for reasons I do not understand and cannot diagnose.

Any advice would be appreciated.

[magsol]

Ok, another update, and this one is important. I've been trying a lot of things, none with any success, so I decided to start from the ground-up.

I installed a brand new instance of AutoLab on my home desktop (running Ubuntu 24.04). I did not set up SMTP, and I explicitly disabled TLS/SSL so I could access it via http://localhost. Otherwise, the setup followed the instructions in the documentation.

I then tried one of my class assessments, and the same problem cropped up: from the UI, the job appears to never complete; according to the list of Jobs, it did complete, and the logs show it completed successfully, but no feedback or output from the job was recorded anywhere, and the score still does not show up in the Gradebook. Only when I navigate into the Tango container and access the job output text files directly, can I see all the actual job output including the autoresult (which all looks perfectly fine). No errors (that I know of) show up anywhere to explain why this output isn't making it back to AutoLab and the database.

So finally, I tried one of your pre-made assessment examples: specifically, the C-hello. I created the assessment by importing the tarball, so all the settings would be directly set without my needing to take action. I then submitted the provided hello.tar, so I didn't change a single line of what was written in the example.

Same effect: the job never appears to complete, no score is recorded, no feedback/output shown in the web UI; only when I navigate into the Tango container and cat the output text file does all the output show up, and it's all there and looks fine, no errors.

I would very much like some help in diagnosing this, as I am all but out of ideas!

[damianhxy]

Hey there! If this is your first time deploying Autolab, I suspect this might be the issue: https://docs.autolabproject.com/installation/docker-compose/#autograder-logs-not-appearing

[magsol]

That fixed the local deployment, and it can now run both your example assessments and my own assessments!

Does this mean the /etc/hosts needs to be edited on my production/instructional AutoLab server? If so, what does it need to be changed to?

[magsol]

Yep, that was the problem: I needed to add the public IP and FQDN of my production server to its /etc/hosts. For some reason, AutoLab < v3 didn't seem to require this, hence why it's been broken ever since upgrading.

So instead of my /etc/hosts looking like this (as it did with AutoLab v1 and v2):

127.0.0.1       autolab.cs.uga.edu      autolab

Now it looks like this (where 123.456.789.000 is its publicly-accessible IP):

123.456.789.000       autolab.cs.uga.edu      autolab
127.0.0.1       autolab.cs.uga.edu      autolab

Now it all works.

[damianhxy]

Awesome, glad you figured it out! We'll leave this issue open until we update the docs with this information to help future Autolab users.