AGS 4: Touch Input API (2)

[ivan-mogilko]

Resolves #1538
This is a recreation of #2440.

Added following API to the script:

managed struct TouchPointer {
  /// Gets this touch pointer's identifier.
  import readonly attribute int ID;
  /// Gets whether this touch pointer is currently down (in touching state).
  import readonly attribute bool IsDown;
  /// Gets this touch pointer's X screen coordinate.
  import readonly attribute int X;
  /// Gets this touch pointer's Y screen coordinate.
  import readonly attribute int Y;
};

builtin struct Touch {
  /// Number of touch pointers registered by the game engine.
  readonly import static attribute int TouchPointerCount;  // $AUTOCOMPLETESTATICONLY$
  /// Accesses a touch pointer state by its identifier.
  readonly import static attribute TouchPointer* TouchPointers[];  // $AUTOCOMPLETESTATICONLY$
};

The Touch.TouchPointers[] indexed property has an underlying list that only grows during a game session, but never shrinks or resets. It cannot be preallocated, because the backend does not always have ability to tell how many pointers / fingers are supported by the connected touch devices. That is why it starts at zero size, and keep growing as more simultaneous touches are detected.

TouchPointer struct describes a single "touch move" by a Nth pointing device, whatever that might be (a finger, or a mouse in case of mouse to touch emulation, etc). It's ID matches the index of TouchPointers[]. IsDown tells whether this pointer is currently down (touches the sensor device), and where at. More properties could be added later if necessary.

Normally user should iterate over TouchPointers array, looking for the states of all recorded TouchPointers, and maybe remember their states and compare with the last saved state to see how they are changed.

Because multiple touch events may occur within a single game frame, there are 3 new script callbacks supported, that let to react to them with more precision:

function on_pointer_down(int pointer, int x, int y);
function on_pointer_up(int pointer, int x, int y);
function on_pointer_move(int pointer, int x, int y);

where "int pointer" argument is the TouchPointer index, and x,y is the new/last position of a pointer.

Other additions: added eInputTouch to InputType enum for consistency.

---

NOTES on implementation.

I had to create duplicate "touch" events recorded as SDL_UserEvent in our input queue. Primary reason is that input queue is processed with a delay, and our internal "pointer id" cannot be reliably calculated anymore after the finger is unpressed. So it has to be recorded along with the event in queue. This user event stores AGS_TouchPointerEventData with pointer id, and a position already in absolute window coordinates (SDL finger events have it in [0,1] relative coordinates).

At the time touch events do not affect anything in game state, rather than calling script callbacks. I.e. they do not skip display boxes, cutscenes etc. But this may be implemented later.
(At least display boxes skip must be implemented, I suppose, as no script is run when they are shown, so user cannot script their removal with a touch.)

---

TEST GAME

ags4-touch-test.zip
ags4-touch-test-compiled.zip

Built APK (14 MB):
https://www.dropbox.com/scl/fi/6pqtttp2pwzyshqgn9ceg/ags4-touch-test-apk.zip?rlkey=764b7unhlrm9lafv3b4qnuy2f&st=f7h1x5um&dl=0

---

TODO

• Besides few things mentioned above, I think we may benefit from a config setting that enables mouse-to-touch emulation, so that one could test at least 1 finger touch without having a touch device.
• Skip display boxes on touch.
• Make a simple demo game which requires at least 2 fingers to control, and test on e.g. Android.

[ivan-mogilko]

Made a simple test game, borrowing the idea from the screenshot in #1538, where there's a on-screen "joystick" controlling player's movement, and a separate "pad" with buttons on it for the second finger:

ags4-touch-test.zip
ags4-touch-test-compiled.zip

Built APK (14 MB):
https://www.dropbox.com/scl/fi/6pqtttp2pwzyshqgn9ceg/ags4-touch-test-apk.zip?rlkey=764b7unhlrm9lafv3b4qnuy2f&st=f7h1x5um&dl=0

[ivan-mogilko]

Ok, good news, my test game works well on Android, letting to control with 2 fingers using new API.

[ericoporto]

I the undefined state of the touch is -1, -1, I wonder if its better to use some big negative number there, in case there is a black border all around the game screen - like when the game is run in integer scaling but the screen is bigger in some number that is not a multiple in any axis.

[ivan-mogilko]

I the undefined state of the touch is -1, -1, I wonder if its better to use some big negative number there, in case there is a black border all around the game screen - like when the game is run in integer scaling but the screen is bigger in some number that is not a multiple in any axis.

Input touch position is always clamped to the game screen, they can never go behind these borders.
But I can make these INT32_MIN.