Remote IES modification system

[celophi]

Summary
Changes to the client may be made dynamically by remotely modifying IES data through specified packets. It is extremely likely that server operators running Melia will want to customize the experience for players to their liking, and since distributing modified clients is not legal, the ability to use these packets will be highly desired.

Problem
In order to send a modification, intimate knowledge of how an IES is structured must be known to the application. Further, information required may change with each client release, and an incorrect structure can crash the client. We need a system that allows for making changes easily and safely. Such a system should make it easy on the server operators to add/remove new changes without needing to spend so much effort to find required information.

Data Requirements
The following data is required in order to make an IES change:

• Filename of the IES.
• ClassID of the row to be modified.
• Column name identifying the modification.
• Previous value to be overwritten. (Necessary when revoking a revision).

Items to Consider

• Modifications should most likely exist as a table in the database.
• Should modifications be applicable to single characters/accounts? (More complex, but maybe there is a reason someone might want this?)
• A large number of modifications should not be sent as one packet. They need to be split up.
• Should we make a feature to allow GMs to make modifications through CZ_CHAT? (Validation needs to be robust since a bad packet can crash clients.)
• Modifications are not permanent by default and reset only once the client is closed. (It seems like there is code to flush changes to the files themselves).

[exectails]

Modifications should most likely exist as a table in the database.

I was thinking about making it scriptable, so it can easily be done on the fly by anyone, be reloaded, and maybe even included in scripts that you download, when they require specific values for some reason.

addiesmod("sharedconst", "1", "Value", "0.0", "10.0")

Should modifications be applicable to single characters/accounts?

I don't think it needs to be integrated into the base system, but it wouldn't be hard to do it, so why not. Although I really can't think of a use case.

A large number of modifications should not be sent as one packet. They need to be split up.

Why is that? Does the client have problems handling larger numbers?

Should we make a feature to allow GMs to make modifications

I can't imagine why a normal GM would have that kind of power, and a dev could change a script and then reload it. I was even thinking about maybe supporting hot reloading at some point, with that you would only need to change the file.

Previous value to be overwritten.

You always need to know the previous value to make a change? That's a little unfortunate =|

a bad packet can crash clients

In which cases does the client crash?

[celophi]

I was thinking about making it scriptable,

There are good reasons to do that. The reason I suggested a database was because it will generate a date for you which can be used to generate the revision number easily.

Why is that? Does the client have problems handling larger numbers?

It won't break the game, but there is a very recognize-able client freeze while it is processing all of the patches. I ran a few tests, and for me anything close to ~100 revisions at a time was about a second of delay.

I can't imagine why a normal GM would have that kind of power,

I mention it only because IMC allows this currently. It would be trivial to interpret a packet and broadcast it to all clients.

hot reloading

You could do that. Authors need to know that the revision number must be increased each time if the same cells are changed. With a script, how do you implement rollbacks? There are more complicated issues with using scripts, but really they only apply if Melia runs on multiple servers which is a long ways off.

You always need to know the previous value to make a change?

No, only in the case of a rollback. And that's a big if. I haven't been able to understand how to do a rollback correctly yet to test it.

[exectails]

The reason I suggested a database was because it will generate a date for you which can be used to generate the revision number easily.

I guess with scripts we'd just need to take them as they come, with the user taking on the responsibility of ordering them correctly in the script loading list. Though in an open system there should probably not even be multiple changes to one column. If multiple scripts change the same value, there are bound to be problems, all of them expecting certain outcomes.

[exectails]

anything close to ~100 revisions at a time was about a second of delay.

There go my dreams of custom shops, where we blank the entire shop table to create a new one dynamically whenever a shop is opened, just for that client =P Had it all figured out o~o

[celophi]

There go my dreams of custom shops,

v.v;;

How many revisions do you expect for a custom shop? We should still try a test to see what the experience will be like.

[exectails]

We should still try

Yea, absolutely^^ We'd need to change as many rows as the largest custom shop has items in the shop table, and, I guess that can be done on login, the price table, which has a few hundred (~580) entries.

The way this would work is that we have a price table, where every price is 1, and a shop table that we modify on the fly whenever a user opens a shop. On as many rows as we have items in the custom shop we change the shop name to our new one, the item to one of the items, and the price multiplier to the actual price of our item. Now the client would display our items, with our prices once we open it.

Replacing them dynamically like this is necessary if we can't insert new entries, because there really aren't that many shop items in the client, and users like to go a little crazy with those.

Quick calculation, a shop of the size of Dunkel's weapon shop in Klaipeda would require 44*3=132 changes before the shop is actually opened. (44 rows, 3 columns each) The absolute max number of items possible would be the number of rows in the shop table, which are a few hundred.

[exectails]

I did a quick test right now, modifying the price of a ledge theme in the barracks, and it seems like modifying IES properties can be pretty straight forward. What I did was to simply send one update for a property every login, keeping the revision number and previous value the same, but changing the new value. When opening the window for changing the lodge, it always showed the price I just sent. Though it seems to get more finicky if you want to change values over and over at run-time. That means as a simple on-the-fly patching tool this could very well be used to change a couple IES properties on login.

I'm also wondering about the lag that was mentioned before when doing a lot of changes. While experimenting with client scripting I noticed that some dbs appear to get cached, which makes it difficult to modify them from Lua. If they need to be recached after IES changes though, that might explain some lag, because that would take a moment.

[exectails]

In other news, I learned that the shops are cached and apparently don't reload, so you actually can't modify them with IES mods at all 🙄 It does work on items though, so that will probably be useful. Then again, you don't seem to be able to add or remove objects, so the use is limited.

[exectails]

The basic functionality was now implemented and I didn't notice any problems switching between servers or anything yet.