Multi-Version Compatibility

[exectails]

Supporting multiple versions of a game's client is difficult. It's a lot of work and it can be complicated. However, especially in a game that has changed as much as ToS has over the years, such functionality might be highly desired by users. Maybe you want a feature back that was removed, maybe you want to use a client from a region that isn't supported anymore, or maybe you just want to experience the game as it was during the beta. There are situations where modifying the client a little bit doesn't cut it, or where readding features would take a considerable amount of work. Not to mention that not every user has the technical knowhow to do these changes.

Originally, we just started emulating iToS because that would become our hometurf, and we were too busy getting the basics to work to worry about keeping multiple clients compatible. That being said, if we do want multi-version compatibility... it should be kept in mind rather sooner than later, because that decision will affect the design of several systems, and adding support for it later on might get messy.

Advantages

• Ability to (potentially) use any client you want, particularly one that might be closest to your ideal version of the game
• Ability to keep using one client version even if the server is kept getting updated

Disadvantages

• Work. A lot of work. And headaches. Ulcers... the usual.

This was already briefly discussed once in #47, but ultimately I decided against it at the time. That was pre-Rebuild though, and I'd like to point your attention at the first comment on that issue.

I don't think this game will last longer than 4-5yrs. Not sure there will be some milestones like RO ones:
classic -> pre-re (12-13episode) -> renewal

Even if in future there will be some gamebreaking feature, some limitations or stupid exploitable mechanic it's should be in emu configs (like limited trade: 0 or 1) etc.

Point is - if this feature (multi-ver) will be ever used by some player base? probably not

The game did last longer than 5 years, it did already have one major update that is on the level of Renewal, and I would wager that quite a few users might be interested in going back to pre-Rebuild. Not to mention all the other smaller changes, and who knows what will come in the future.

[exectails]

If we did want this feature, I believe there are two fundamental ways to implement it in a viable manner with C# for this game.

1) Ifs and lookup tables

We do runtime checks for packet changes and dynamic lookups for enum values. Over the years I've come up with a way to handle enums that doesn't suck completely, which I'm using in other projects, and it works pretty well in my opinion. For example, here's what a packet sender might look like.

public static void ShowEffect(Character character, EffectId effectId, EffectParameters parameters)
{
	if (Game.Version < 20050311)
		throw new GameVersionException("Not supported by this version.");

	var networkEffectId = MEnum<EffectId>.GetValue(effectId);

	var packet = new Packet(Op.NET_SHOW_EFFECT, character.ObjectId);
	packet.PutInt(networkEffectId);

	if (Game.Version >= 20060802)
		parameters?.AddTo(packet);

	character.Region.Broadcast(packet, character);
}

The EffectId enum is just a simple enum without explicit enumeration, which works independently from the values expected by the client. Inside the sender, hidden from the coder, this value is then converted to the value that the client expects, similar to how HostToNetworkOrder might affect the data sent over the network. MEnum is initialized from a file for every enum it needs to support and then just converts the values back and forth. The files it loads the values from use a C-like, macro-esque, enum-focused layout.

StopMusic
PetGaze = 1000
PetBuff
#if VERSION >= 20060801
	MagicalMusic
	PetPvp
#endif
Curtain

This is one way we could handle enums. In the context of ToS, we could look at the property ids for an example of how this might be used. In the normal code, a normal enum would be used, but when the properties are supposed to get sent to the client, we look up the value the property id should have for the currently selected version and send that instead.

Maintaining the enums would require some effort, but I believe it's manageable, and throwing in a few simple ifs when adding new packet fields isn't that much work.

2) Branches

This option was suggested by Tachiorz back when we first discussed this. Basically, we would have compatibility branches for certain versions, so if you wanted kToS support for example, you might use the branch compatible-kr-latest, or something along those lines, and that branch would only be compatible to the latest version of kToS.

I imagine that with this approach, you might keep one branch that is relatively free from version dependent enums, and then create branches off of it, that add the version dependent files, like ops, properties, etc. Finally, you might make master the default compatibility branch for the latest iToS, since it will be the most-used version for us.

This approach would presumably be pretty clean, but it would require a lot of branch hopping and potentially confuse users and contributors. It would also let us only support so many versions, because we can't viably create dozens or hundreds of compatbility branches.

Non-Viable

Two options that aren't viable in my opinion are #define and OOP. C#'s defines aren't on the level of C's. We'd have to create defines like VERSION_20210831 for every single version, and filtering which part of the code we actually want to be active would be a mess. Similarly, I believe any kind of OOP solution is out, because the game's packets and ops just change way too much. We would end up with thousands of variations of packet handlers and writers, to a completely unmaintainable degree.

[exectails]

Okay, in that case it sounds like you're proposing the branching model I described, just with plugins added on top^^ Focus on major versions and use branches to keep logic and version dependent code organized. The advantages and disadvantages outlined above should apply then.

you are the main driving force behind this project (so far anyway)

Well, as the original developer, I kind of get roped into working on Melia every time someone shows some interest in the project again, and I'd be happy to see it actually take off after all those years, but honestly, in its current form, Melia isn't high on my list of interesting projects anymore, because I have absolutely no interest in the current version of the game.

are you sure that we shouldn't skip the latest revisions and just focus on that instead

I'm not sure about sticking to the latest version at all, and it's not what I would want personally, but frankly, working without packet logs is an ordeal. Not difficult, or almost impossible, it's just tedious, and it takes time. Ideally you also want to have a little bit of reverse engineering experience, to understand the client code. That's not exactly well suited for a community project, because it would hinge on someone digging up the packet structures that weren't logged in the past, wheras, with the latest version, someone can just log into iToS and get whatever they want.

This is in part why I opened this discussion, because with multi-version support and a solid foundation you can have both.

If the leak really contains the source files, we wouldn't even need an emulator, as we could just alter the code, even though that would put us in a tough spot with legality of the project.

You said it, it would be illegal. Sharing the source code or the binaries publically in any form would be kind of crazy. And I certainly wouldn't recommend setting up public servers with those files. They could be useful for research however. But even if we assume that it's just a matter of time until they become available, we don't know when that might be yet.

[ovflowd]

e. That's not exactly well suited for a community project, because it would hinge on someone digging up the packet structures that weren't logged in the past, wheras, with the latest version, someone can just log into iToS and get whatever they want.

Reverse Engineering can be way harder and would require way more commitment with the contributors. I love reverse engineering. But we would need a specialist that understands that. I have friends that might be willing to volunteer for that. Yet, I definitely agree just using a Packet Logger is way easier because you get what you want.

[Adephx]

Just a reminder that ToS doesn't really recieve updates in a traditional sense. It should be relatively easy to convert the current state of the game back to what it was, even with the current client. Drops are handled on the server side, so are stats and access to skills.

[exectails]

It should be relatively easy to convert the current state of the game back to what it was

Drops, stats, and skills surely are part of it, but there are always things you can't just easily revert back. But if I understand you correctly, you don't mind much either way, new version, old version, as long as the base systems are working in the end, right? Though you seem to be against multi version support, because it adds a lot more work that might never be put to good use.

[Adephx]

It should be relatively easy to convert the current state of the game back to what it was

Drops, stats, and skills surely are part of it, but there are always things you can't just easily revert back. But if I understand you correctly, you don't mind much either way, new version, old version, as long as the base systems are working in the end, right? Though you seem to be against multi version support, because it adds a lot more work that might never be put to good use.

That would be the TLDR of it yeah.