Stanford CS149 Assignment 4 getting red error [TEN404] Internal tensorizer error: TensorInitialization:Expect NeuronReduceMacro!

[kpodosin]

Here is the full error:
(aws_neuron_venv_pytorch_p310) ubuntu@ip-172-31-37-252:~/cs149assign4/part2$ python3 test_harness.py
Running correctness test for conv2d kernel with smaller images...[TEN404] Internal tensorizer error: TensorInitialization:Expect NeuronReduceMacro! - Please open a support ticket at https://github.com/aws-neuron/aws-neuron-sdk/issues/new. You may also be able to obtain more information using the 'XLA_IR_DEBUG' and 'XLA_HLO_DEBUG' environment variables.
Traceback (most recent call last):
File "/home/ubuntu/cs149assign4/part2/test_harness.py", line 183, in
test_result = test_correctness_conv2d_kernel(conv2d, use_larger_images=False)
File "/home/ubuntu/cs149assign4/part2/test_harness.py", line 85, in test_correctness_conv2d_kernel
out = kernel(*args, **kwargs)
File "neuronxcc/nki/compile.py", line 95, in neuronxcc.nki.compile.GenericKernel.call
File "neuronxcc/starfish/penguin/targets/nki/TraceKernel.py", line 174, in neuronxcc.starfish.penguin.targets.nki.TraceKernel.Kernel.call
File "neuronxcc/starfish/penguin/targets/nki/TraceKernel.py", line 422, in neuronxcc.starfish.penguin.targets.nki.TraceKernel.BaremetalKernel.post_process_call
File "neuronxcc/starfish/penguin/targets/nki/TraceKernel.py", line 425, in neuronxcc.starfish.penguin.targets.nki.TraceKernel.BaremetalKernel.post_process_call
File "neuronxcc/starfish/penguin/targets/nki/TraceKernel.py", line 508, in neuronxcc.starfish.penguin.targets.nki.TraceKernel.BaremetalKernel._compile
RuntimeError: Compilation failed for fused_conv2d_maxpool with error Command '['neuronx-cc', 'compile', '--framework', 'XLA', 'penguin.py', '--internal-tensorizer-opt-level=nki', '--pipeline', 'compile', 'SaveTemps', '--target', 'trn1', '--disable-internal-io-dge', '--output=file.neff']' returned non-zero exit status 70.

Not sure what to do. I restarted my AWS instance and tried to run again but still getting the error.

[AWSNB]

hi @kpodosin

sorry you are getting this issue, please see couple of other threads that had TEN404 error to see if those tips and suggestions work for you:

#1055
#1052
#1049

and while we are working to make sure compiler output is much more helpful, there may be already useful errors from the compiler that may possibly be hidden by the test harness, see tip from @aws-serina-tan in : #1054 (comment)

[kpodosin]

The issue is that I was using a +=. Thanks