Pipez pipes consistently causing the most lag, taking up most of the tick time

[Nevrai]

Bug description

Pipez are consistently the most lag-inducing tile entities in my world, and many others I’ve spoken with have the same issue. For item and fluid transport, I’ve been using Pipez, Mekanism, LaserIO, Integrated Dynamics and Integrated Tunnels, and Modular Routers, but Pipez is always taking up the most tick time and causing the most lag out of any of these—as well as the most lag out of any of the mods I have installed.

I’m playing in a single-player world. I’m using Netherite upgrades in most of my pipes and have a lot of Pipez pipes in my world. None of the mod’s config options have been changed from their defaults. My tps is also very low most of the time (although Forge’s tps command usually shows a higher tps than spark’s tps command, so I’m not sure what that’s about, but my tps is poor either way).

Screenshots and descriptions

Click the spoiler below for screenshots of profiles done with Observable and a profile done with spark, with descriptions.

Click to expand

 

These screenshots show profiles run using Observable and spark, showing how much tick time tile entities are taking up. My Pipez pipes are, by far, some of the worst offenders each time I run a profile.

Item pipe pulling out of an EnderChests Ender chest, which is getting items from an RFTools Builder builder quarry (so it is importing many items)

Universal pipe getting energy from a Flux Networks flux plug and pulling items from a Functional Storage storage drawer

Item pipe pulling items from an EnderChests Ender chest. Pipe is being filled with a Cable Tiers creative exporter, but the chest is full, as is the inventory it’s connected to, and it’s not exporting any items at this time, yet the pipe is still causing lag.

Fluid pipe pulling lava from an EnderTanks Ender tank. The tank is full, and the machines using the lava are not running, yet the pipe is still causing lag.

Profile (profiling ticks that took more than 100 ms to run over a span of about a minute and a half) done with spark, showing that Pipez is the most lag-inducing mod in my world.

Profile done with Observable, showing that Pipez’ pipes are taking up far more tick time than practically anything else, causing a lot of lag

Steps to reproduce the issue

Not sure exactly, but use Pipez’ pipes to pull items or fluids (primarily), with Netherite upgrades, and run an Observable profile and a spark profile (or look at the tps when you have a lot of Pipez pipes in use by using /forge tps overworld—or /spark tps with spark installed).

Versions

• Modpack: All the Mods 7 (0.4.29—latest)
• Minecraft: 1.18.2
• Forge: 0.4.69 (also tested with 40.1.68)
• Pipez: 1.1.5 (latest for 1.18.2)

[tsbreuer]

From what i've observed (same modpack), it seems to be worse the more filled up the inventory its trying to pull from is.
Using a Integrated dynamics to make a buffer chest that will only turn on the previous pipe with < than 4 slots used in the chest will drastically cut down the us/t from arround ~700 to about 160.

There's some diminishing returns as lowering it to < 2 slots will still be about 110 us/t, but its still pretty good improvement over the original 700.

A really easy solution would be to give us an option to delay every X ticks to check for target inventories to have an available slot (To slow it down a bit)... Or using something like a level emitter to gauge wether it needs to be on at all will literally cut down the us/t to near zero.

However, the level emitter option doesn't really work when mass producing items like in the first picture where i'm moving enough items that i need to use pipez netherite upgrade over mekanism's ultimate logistical transporter.

I can see a big enough base using these pipes with netherite upgrades on backed up inventories creating a lag problem.

EDIT; I just realized i was somewhat okay because i was using regular minecraft chests. I can only imagine how bad the us/t would be if it was a big chest like a diamond chest filled with items backed up.

[BabyDead]

Just came here to confirm. I wanted to make a post of my own but then saw this one.

I tested my Pipez systems with all upgrades.
There is a noticeable lag increase with every upgrade. I literally only need to upgrade one single pipe (length does not seem to matter in the slightest, nor inventory size, what it's linked to, etc).

As soon as I get to the diamond level upgrade, I get a noticeable lag on both a server and single player mode, where a block will reappear for a second after destroying it. As soon as I put in a Netherite level upgrade, you might as well stop playing. The lag caused by this is absolutely insane. No, it's criminal, even. I can build a massive automated world which runs perfectly fine and smooth, but add one single Netherite upgrade to the Pipez system and everything falls apart.

I notice this post is from September. Is there a fix on the way? Any kind of update?

Small edit: I have tried blocking all the pipes out of sight so they wouldn't need to render, but that changes nothing.

Edit 2: This problem seems to exist exclusively on item and univeral Pipez, not energy. I have not tested fluid.

[NordicGamerFE]

Just came here to confirm. I wanted to make a post of my own but then saw this one.

I tested my Pipez systems with all upgrades. There is a noticeable lag increase with every upgrade. I literally only need to upgrade one single pipe (length does not seem to matter in the slightest, nor inventory size, what it's linked to, etc).

As soon as I get to the diamond level upgrade, I get a noticeable lag on both a server and single player mode, where a block will reappear for a second after destroying it. As soon as I put in a Netherite level upgrade, you might as well stop playing. The lag caused by this is absolutely insane. No, it's criminal, even. I can build a massive automated world which runs perfectly fine and smooth, but add one single Netherite upgrade to the Pipez system and everything falls apart.

I notice this post is from September. Is there a fix on the way? Any kind of update?

Small edit: I have tried blocking all the pipes out of sight so they wouldn't need to render, but that changes nothing.

Edit 2: This problem seems to exist exclusively on item and univeral Pipez, not energy. I have not tested fluid.

This does also happen with fluid. on 1.18.2 play stoneblock 3

[xBlackPoison357x]

Had to shutdown a factory that was feeding around 60 salvagers from a chest of items to break down using Pipez because it was using 66% of tick time. https://spark.lucko.me/goZx5CUvVz

[LilMxPlagueRat]

Just came here to confirm. I wanted to make a post of my own but then saw this one.
I tested my Pipez systems with all upgrades. There is a noticeable lag increase with every upgrade. I literally only need to upgrade one single pipe (length does not seem to matter in the slightest, nor inventory size, what it's linked to, etc).
As soon as I get to the diamond level upgrade, I get a noticeable lag on both a server and single player mode, where a block will reappear for a second after destroying it. As soon as I put in a Netherite level upgrade, you might as well stop playing. The lag caused by this is absolutely insane. No, it's criminal, even. I can build a massive automated world which runs perfectly fine and smooth, but add one single Netherite upgrade to the Pipez system and everything falls apart.
I notice this post is from September. Is there a fix on the way? Any kind of update?
Small edit: I have tried blocking all the pipes out of sight so they wouldn't need to render, but that changes nothing.
Edit 2: This problem seems to exist exclusively on item and univeral Pipez, not energy. I have not tested fluid.

This does also happen with fluid. on 1.18.2 play stoneblock 3
I'm also having this issue in StoneBlock 3 I've just torn apart a large Lava Generation set up because it had reduced my fps from over 100 to 2

[CrazyDev05]

For me, it got to the point of a server Crash
crash-2023-01-21_13.06.57-server.txt

[jpeaglesandkatz]

Can you guys check if this happens only with say round robin settings on the filter or also on the other modes, if there is a difference.. I've been looking through the code and from my observations, this mainly happens with the 'insert ordered' method...

[jpeaglesandkatz]

From what i've observed (same modpack), it seems to be worse the more filled up the inventory its trying to pull from is. Using a Integrated dynamics to make a buffer chest that will only turn on the previous pipe with < than 4 slots used in the chest will drastically cut down the us/t from arround ~700 to about 160.

There's some diminishing returns as lowering it to < 2 slots will still be about 110 us/t, but its still pretty good improvement over the original 700.

A really easy solution would be to give us an option to delay every X ticks to check for target inventories to have an available slot (To slow it down a bit)... Or using something like a level emitter to gauge wether it needs to be on at all will literally cut down the us/t to near zero.

However, the level emitter option doesn't really work when mass producing items like in the first picture where i'm moving enough items that i need to use pipez netherite upgrade over mekanism's ultimate logistical transporter.

I can see a big enough base using these pipes with netherite upgrades on backed up inventories creating a lag problem.

EDIT; I just realized i was somewhat okay because i was using regular minecraft chests. I can only imagine how bad the us/t would be if it was a big chest like a diamond chest filled with items backed up.

Whether you are using a bigger chest or not should not matter.. If it is full, it is full

[Robj91]

I noticed this issue too. Seems to cause more lag as the inventory it is trying to pull from fills up more, and the contents have no valid destination. Also the more possible destinations that exist(but can't accept the items) causes more lag as well. In pic 2 below, if you break a few of the full destination chests, the us/t number decreases a lot for each invalid destination taken away. This is just what I've noticed, and it doesn't matter what method is selected on the pipe (round robin, nearest first, whatever, same result). These pics are with a ultimate pipe upgrade but no filtering added.

[jpeaglesandkatz]

Well I've looked at the code but unfortunately either my knowledge is too limited or it is not pipez related... But I'm hoping the mod author will finally take a look at this..

But I can definitely confirm that there is a lot of lag (contrary to the author's performance comparison statement) in 1.19.2 (ATM8) as well.

[jpeaglesandkatz]

Appears this mod is now abandoned looking at the lack of response from the dev for months even though there are some serious gamebreaking bugs present. Shame, it had potential.

[Th3Fanbus]

EDIT; I just realized i was somewhat okay because i was using regular minecraft chests. I can only imagine how bad the us/t would be if it was a big chest like a diamond chest filled with items backed up.

Well... With 8 diamond chests with a single Stack Upgrade Tier 4 (x16), a single pipe makes TPS horrendous. Mod versions:

sophisticatedcore-1.19.2-0.5.38.203.jar
sophisticatedstorage-1.19.2-0.6.16.276.jar

The chests mainly contain unstackable items (e.g. tools, armor). The diamond ones are filled to the brim and the netherite chest the pipe is extracting from is about three quarters full. Without the netherite upgrade, the pipe still takes 27 ms to tick (which is still unacceptable, it's more than half of the maximum tick time to maintain 20 TPS).

Profile data after letting the game run for a while, with the netherite upgrade:

spark-profile.zip

[henkelmax]

The tick time of the pipe is mostly dependent on the implementation of the insertion and extraction logic of the blocks you are pulling from and inserting to.
Just because Observable tells you the pipe takes up a lot of tick time, doesn't mean it is caused by the Pipez mod.
As you can see in @Th3Fanbus post, the sophisticated storage mod is actually taking up the greatest part of the time.

I did some tests and can confirm that this is not caused by the Pipez mod.

We still went ahead and optimized the pipe performance as well as we could (Will be in the next release). But the lag you all are experiencing is not caused by us.