Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Render order #251

Closed
konsumer opened this issue Feb 1, 2021 · 35 comments
Closed

Render order #251

konsumer opened this issue Feb 1, 2021 · 35 comments

Comments

@konsumer
Copy link

konsumer commented Feb 1, 2021 •
edited
Loading

In some tile-based engines (like godot, with some config) render order is determined by a combination of layer height + X/Y position. This makes it possible to have a thing the player can be "in front of" and "behind" in a zelda-style map.

I made an example map that has player layer under some stuff. When there is overlap (like with the bird feeder) it would be better if it check Y position to determine which to render first, so it draws the player over it, when they are below it, in Y position.

Peek 2021-01-31 22-18
@konsumer
Copy link
Author

konsumer commented Feb 1, 2021

Looks like #166 is related.

@karai17
Copy link
Owner

karai17 commented Feb 1, 2021 via email

@konsumer
Copy link
Author

konsumer commented Feb 2, 2021

Thanks. I will look into that.

I'd prefer to not have to duplicate information in the map, if I can help it, and not draw anything multiple times. I realize you aren't suggesting this, just mentioning it as part of my goals, here. I'm trying to come up with a basic system that is extremely asset-driven, so you can mostly edit your whole game in tiled, with a few properties and all entity behaviors defined in lua files. Kinda like solarus, but for tiled/love2d.

Maybe layer-grouping would help? Like add a property to layer-group such as z="y", which means "figure out render-order for layers in this group, based on Y position."

Screenshot from 2021-02-02 14-06-44

Is this something that would be useful to others, as a plugin, or does it make more sense as just layer-code in my app?

@karai17
Copy link
Owner

karai17 commented Feb 2, 2021 via email

@konsumer
Copy link
Author

konsumer commented Feb 2, 2021

I started work on a general solution for this at sti-renderorder, but it's just a stub for now. I will close this issue. Thanks for taking a look.

@konsumer konsumer closed this as completed Feb 2, 2021
@karai17
Copy link
Owner

karai17 commented Feb 3, 2021 via email

@konsumer
Copy link
Author

konsumer commented Feb 3, 2021 •
edited
Loading

I have a basic system that can draw a layer "in front" or "behind":

local function in_front(location, layer)
  -- always "behind"
  return false
end

local function sti_renderorder(map, renderorder_layer, location)
  local orig_draw = renderorder_layer.draw

  -- find layers that could overlap & disable drawing (keeping a ref to original draw)
  local layers = {}
  for i=1,#map.layers do
    local layer = map.layers[i]
    if layer.properties.ysort then
      layer.orig_draw = layer.draw
      function layer:draw() end
      table.insert(layers, layer)
    end
  end

  -- overwrite the renderlayer's draw function to account for front/behind
  function renderorder_layer:draw()
    orig_draw()
    for _,layer in pairs(layers) do
      if in_front(location, layer) then
        layer:orig_draw()
        orig_draw()
      else
        layer:orig_draw()
      end
    end
  end
end

But I am having trouble with how to implement in_front(). It works ok if I set it to always true or always false and the location is correctly updated, so I think it's an ok start, but it is drawing the player-layer multiple times (either under, or under/over.) I can live with that, if it works, and optimize it later. I have demo code here. What can I do in in_front() to determine if the layer wants to draw in the current location space? I realize I am calling in_front in a loop and the in_front will probly also be a loop, so this will probly suck for performance, but I'm not sure how I could further optimize that. Maybe somehow assign z to each layer in { playerlayer, ...zlayers } for a single-pass sort? Any help would be greatly appreciated.

@konsumer
Copy link
Author

konsumer commented Feb 4, 2021 •
edited
Loading

I got a bit closer by just exploiting the fact that there is layer.data[y][x]. I would still appreciate any input:

local function sti_renderorder(map, renderorder_layer, location)
  local orig_draw = renderorder_layer.draw

  -- find layers that could overlap & disable drawing (keeping a ref to original draw)
  local layers = {}
  for i=1,#map.layers do
    local layer = map.layers[i]
    if layer.properties.ysort then
      layer.orig_draw = layer.draw
      function layer:draw() end
      table.insert(layers, layer)
    end
  end

  -- overwrite the renderlayer's draw function to account for front/behind
  function renderorder_layer:draw()
    orig_draw()
    local targetx, targety = map:convertPixelToTile (location.x, location.y)
    targetx = math.floor(targetx)
    targety = math.floor(targety)
    for _,layer in pairs(layers) do
      if layer.data[targety] and layer.data[targety][targetx] then
        layer:orig_draw()
        orig_draw()
      else
        layer:orig_draw()
      end
    end
  end
end

Peek 2021-02-03 22-17

The choppiness is in the gif, it seems to run ok.

@karai17
Copy link
Owner

karai17 commented Feb 4, 2021 via email

@konsumer
Copy link
Author

konsumer commented Feb 4, 2021

Yeh, I swapped them initially in the description, but in code, I have it [y][x].

@konsumer
Copy link
Author

konsumer commented Feb 5, 2021

I am still having a great deal of trouble getting this to work in a decent way. it seems like around any mention of tiled + z-ordering people say "use a custom layer that sorts the tiles correctly" but I can't find any actual code examples of this. You have used this technique, and offered this advice in the forums and here. Can you link me to an example of actually doing this?

@karai17
Copy link
Owner

karai17 commented Feb 5, 2021

https://github.com/excessive/rpg/blob/master/states/gameplay.lua#L222-L246

This code is a bit old to say the least, but I believe this is where we wete doing this in a test game we were working on.

@konsumer
Copy link
Author

konsumer commented Feb 6, 2021

Ah, I see, you just treat things that can be over/under the player like you would the player position, placing on the map after, in the right order. I totally get that approach, and it does seem much simpler. I will fall back to that, if I can't figure this out, soon.

I like the idea of the map just knowing how to do it correctly, using tile locations (not exact pixel positions, but 32x32 grid or whatever) & spritebatches, for max efficiency. I'm going to keep playing with it.

@konsumer
Copy link
Author

konsumer commented Feb 8, 2021

I am still having trouble with this.

I tried a few different methods. I put all the objects with gids (tile-objects) in a separate table, then set visible to false on all their (object) layers. After this, I want to update the player-layer to reorder the tiles, but they are all in SpriteBatchs, so I'm not sure how to reorder & draw them, individually.

so if I do this, it will draw all the tile objects from an object-layer, without accounting for player:

    function self.map.layers.player.draw()
       for _, batch in pairs(self.map.layers.objects.batches) do
         love.graphics.draw(batch, 0, 0)
      end
    end

I also have a collection of the original objects with gid, that I collected in an earlier step called self.object_tiles and I can get the tile-instances like this:

    function self.map.layers.player.draw()
      for _,tile in pairs(self.object_tiles) do
        -- what do do here?
        local tileInstance = self.map.tileInstances[tile.gid]
      end
    end

I imagine I can do 2 loops, 1 under, then draw the player, then 1 for over-player, but I can't figure out how to separate the tiles like this.

Since I can get the tile-instance, the sprite-batches, etc, it seems like I am very close, but I still don't understand how to order and draw the tiles separate from their spritebatch. Am I thinking about the problem incorrectly?

@karai17
Copy link
Owner

karai17 commented Feb 9, 2021

Instead of Tile Objects, I'd use Point Objects to denote locations for things to be, and then use that info to load custom objects into a custom layer and sort those objects by their Y value. You could batch them each frame and draw the whole layer only once and it should still be very fast.

You could use a temporary tile layer to place tiles down where you'd want them to be so you can design your maps and then in the object layer place points on the objects.

A little trick to help with aligning your objects to the tile drid would be to divide the point's XY location by tile size, floor those values, and them multiply them back to pixels~

Below is some pseudocode that should help you figure out exactly what you need to do, it's been a while so i don't quite remember love's syntax off the top of my head

local layer       = map.layers["Objects and Stuff"]
local old_objects = layer.objects
local clayer      = map:createCustomLayer("Fancy Stuff Here!")
clayer.objects    = {}
local objects     = clayer.objects
clayer.batch      = love.graphics.newSpriteBatch() -- etc

-- Align objects to tile grid even if the points in the map data are loosely placed
for i, object in ipairs(old_objects) do
   objects[i] = {}
   local o = objects[i]
   o.x = math.floor(object.x / map.tilewidth) * map.tilewidth
   o.y = math.floor(object.y / map.tileheight) * map.tileheight
   o.texture = some_texture -- texture all your objects will be from, for batching
   o.sprite  = some_quad -- the area of the texture to be drawn for this particular object
end

-- Add player to the custom layer
table.insert(clayer.objects, player)

function clayer:update(dt)
   table.sort(self.objects, function(a, b)
      return a.y < b.y
   end)

  -- clear self.batch
  -- re-batch from self.objects
end

function clayer:draw()
   love.graphics.draw(self.batch)
end

@konsumer
Copy link
Author

konsumer commented Feb 10, 2021 •
edited
Loading

I think I understand, and maybe have a different way, using regular quads, so I can display things that are just tiles, and also things that are dynamic objects (animated NPC for example) and also do per-pixel drawing. But either way, I could use help getting some_texture and some_quad in your example.

First I load up a bunch of objects:

function noop() end

  for k, object in pairs(self.map.objects) do
    -- disable the layer's draw & update, as I will be taking these over in player-layer
    object.layer.draw = noop
    object.layer.update = noop

    -- copy id because bump leaves it out
    object.properties.id = object.id
    if object.type and object.type ~= '' then
      if objectTypes[ object.type ] then
        -- add an instance of the type class, if it exists
        self.objects[object.id] = objectTypes[object.type](object, self)

        -- fill in some basics from map to make objects nicer to use
        self.objects[object.id].id = object.id
        self.objects[object.id].gid = object.gid
        self.objects[object.id].type = object.type
        self.objects[object.id].x = object.x
        self.objects[object.id].y = object.y
        self.objects[object.id].width = object.width
        self.objects[object.id].height = object.height

        -- HOW DO I GET image/quad IN self.objects[object.id].tile HERE?
        
        if object.type == "player" then
          self.player = self.objects[object.id]
        end
      else
        -- non-fatal warning that no behavior has been setup
        self.objects[object.id] = object
        print("No script found for " .. object.type .. "(".. object.id ..")")
      end
    end
  end

Then, in player layer I do all the sorting and drawing:

    function self.map.layers.player.update(layer, dt)
      -- run update for every object
      for k,object in pairs(self.objects) do
        if object.update then
          object:update(dt)
        end
      end

      -- this manages physical collisions (`collidable` layers)
      local playerX = self.player.x + (self.player.move[1] * (dt + self.player.speed))
      local playerY = self.player.y + (self.player.move[2] * (dt + self.player.speed))
      local actualX, actualY, collisions, len = self.world:move(self.player, playerX, playerY)
      self.player.x = actualX
      self.player.y = actualY

      -- sort self.objects by Y
      table.sort(self.objects, function(a, b)
        if  a and b then
          return a.y > b.y
        end
      end)
    end

    function self.map.layers.player.draw()
      for o,object in pairs(self.objects) do
        -- run the objects map_draw, to draw in context, or draw it's tile
        if object.map_draw then
          object:map_draw()
        else
          if object.tile then
            --  I DON'T HAVE THIS PART
            love.graphics.draw(object.tile.image, object.tile.quad, object.x, object.y)
          end
        end
      end
    end

Where does some_texture and some_quad in your example come from? I need to pre-cache it in object.tile in my code, but can't figure out how to pull that out of the map from an object, and I think this is the last piece I need. From docs, it seems like map.tileInstances[object.gid] should have it (using x/y, and batch) but I don't see it.

@karai17
Copy link
Owner

karai17 commented Feb 10, 2021 via email

@konsumer
Copy link
Author

So the data doesn't exist in the map, anywhere? If I could at least get to the spritebatch, I could do batch:getImage() but I still don't know how to get the quad coordinates.

@karai17
Copy link
Owner

karai17 commented Feb 10, 2021

you can probably partially automate it by adding some info to the custom properties of your objects or having some lookup table that matches an object's name to quad coords.

local lookup = {
   "Tree_01" = { x=5, y=6, w=2, h=3 }
}

for _, o in ipairs(objects) do
   if lookup[o.name] then
      local asdf = lookup[o.name]
      o.quad = love.graphics.newQuad(asdf.x*map.tilewidth-map.tilewidth, asdf.y*map.tileheight-map.tileheight, asdf.w*map.tilewidth, asdf.h*map.tileheight) -- or whatever the syntax is
      layer.batch:addSprite(o.quad)
   end
end

@karai17
Copy link
Owner

karai17 commented Feb 10, 2021 •
edited
Loading

my thought here is that your objects are gonna have more than a single tile involved. like a tree might be a 2x3 structure but instead of treating it as 6 tiles you treat it as a single large tile, which is why you need to put in a bit of leg work since tiled itself doesn't really have that sort of feature, as far as i know.

@konsumer
Copy link
Author

you can probably partially automate it by adding some info to the custom properties of your objects

I am so sure the image + quad must be somewhere in the map object, I just can't seem to find it. STI is using that info to draw object-layers, with no meddling, so it had it, at least at one time, even if it doesn't expose it.

like a tree might be a 2x3 structure but instead of treating it as 6 tiles you treat it as a single large tile

The objects can be a placeholder for something else (that is why I have object.map_draw() for things like NPCs that are just linked to with graphical objects), but the position (in x,y, and z, which is layer + Y-order) & props tell my engine what animation to load, where to put it, etc. Things like this are easy to deal with in custom per-type drawing code, if needed.

Aside from that, If I have a 2x3 tree, I want it to render with correct z-order (based on Y) for all of the tiles, and I can put a collision hitbox in the center of the trunk so the player can't stand in the middle of the tree trunk, or end up "under" things it should be "over". then I put the top of the tree on a layer over the player, so it's always over everything. this is similar to trees in zelda alttp, where they are made with several tiles, there are tree tops that can obscure the player, and you can walk in front of and behind some trunk sprites. I'm fairly sure this is done by sorting by layer+Y. I have seen this limitation of tiled come up in the forums as a reason it can't do what I am trying to do, but I've also seen other tiled loaders, in other languages, that don't have a problem with it.

My issue is right now is with single-tile objects (although I think it will carry over to all tiles) that the player might be under or over, depending on Y. A single-tile sign is an example. I add a sign tile object to the map, on a object layer, and set it's type to sign. I load the code to handle user-trigger & touching, and show a sign. That's all working. I just need the player to be able to walk behind the sign, without having to use a collision box to block them out of being able to overlap (or end up "behind".)

Now, I am attempting to solve this by giving up on spritebatches for objects, and just place them all manually, using the image & quad it links to, if no object.map_draw is found, but it's still very challenging because I can't get the images or quads that must be in the map-data, somewhere (but seemingly not where the STI docs say they are, unless I am looking at it wrong.)

If it helps to put it into context of a complete project, here is the game I'm working on.

I appreciate all your help, but If I can't work out what I'm trying to do with STI, I can just go back to writing my own tiled-loader. I made some progress, and I think I can work it out that way. I'm fairly comfortable with tiled (I contributed to the plugin API, and have worked on importers and exporters for other things) and am getting comfortable with love, so I think it's totally doable, if not a pain to duplicate a lot of effort.

I also noticed there is no per-tile linked collisions in STI (like in tiled, choose collision-editor) which is another feature I really want, so I don't have to draw all my collisions over and over, on a seperate layer. I looked around at other tiled-map loaders, in other languages and many of them support these things (for example flixel's tiled loader.)

@karai17
Copy link
Owner

karai17 commented Feb 10, 2021

https://github.com/karai17/Simple-Tiled-Implementation/blob/master/sti/init.lua#L175-L219

STI generates quad data and batches for tile layers based on the width and height of the map and texture, etc. this data just doesn't exist in the tiled map format. you basically need to re-implement this code to handle your own game-specific needs for custom layers. whether you use STI or your own map loader you'll still need to do this~

@konsumer
Copy link
Author

Yep, as I'm digging into how tile-objects work, I am starting to see what you mean!

Here is one object:

{ 
  gid = 1025,
  height = 32,
  id = 1,
  name = '',
  properties = { 
    direction = 'S' 
  },
  rotation = 0,
  shape = 'rectangle',
  type = 'player',
  visible = true,
  width = 32,
  x = 798.203,
  y = 801.822 
}

In this case, gid is 1025 and looks like this in the tileset:

    {
      name = "objects",
      firstgid = 1025,
      filename = "objects.tsx",
      tilewidth = 32,
      tileheight = 32,
      spacing = 0,
      margin = 0,
      columns = 10,
      image = "objects.png",
      imagewidth = 320,
      imageheight = 320,
      objectalignment = "unspecified",
      tileoffset = {
        x = 0,
        y = 0
      },
      grid = {
        orientation = "orthogonal",
        width = 32,
        height = 32
      },
      properties = {},
      terrains = {},
      tilecount = 100,
      tiles = {}
    }
  },

So in STI or in my own attempts, it's the same problem, and it's more complicated than regular tiles. I can get the image from the tileset, and the quad has to be gathered from doing some tricky (and possibly totally wrong) math with other fields in tileset (using columns, tilewidth, tileheight, spacing, margin, etc) If I can assume that gid's are linear (they start at firstgid, and increment 1 to the right/down, etc) then I think I can do it, but it's not really a problem with STI, as it has the same map-info. I had to dig into parsing it myself, to understand that code you linked to, but I totally get it, now.

if that code you linked to is parsing tile objects, the same as the regular map tiles, then it should be in

map.tiles[object.gid].quad
map.tilesets[object.tileset].image

If I am understanding the code right, that is exactly what I was looking for! If I can get this working, I'd much rather use STI.

I will look closer at this. In my earlier experiments, these weren't showing in pprint, I think because it doesn't always print complex class objects (I have seen this with images and quads) so it might have been there all along. Or, in re-reading your comments, you are saying I will need to do similar to what tiles are doing, for objects? That is totally workable, still, as I can make a little util that uses that snippet of code to grab the image+quad, and cache it in objects, before render.

I think I could make a layer.draw function that ignores the spritebatch, and then does z-ordering of all on a single layer with Y, which should solve my original issue. In tiled, I will just put all the things that need to be z-ordered together on a layer, and let them figure out what is over/under. I could probly make a pretty simple & reusable function you can just insert like layer.draw=drawzfight when you want a layer to act like this (maybe via a layer-prop?) Should I PR for this, in the shape of a plugin?

As for composite-collisions (made in tiled's collision-editor) if you are interested, a terrain tile with one looks like this:

{
          id = 311,
          terrain = { 19, -1, 19, -1 },
          objectGroup = {
            type = "objectgroup",
            draworder = "index",
            id = 2,
            name = "",
            visible = true,
            opacity = 1,
            offsetx = 0,
            offsety = 0,
            properties = {},
            objects = {
              {
                id = 1,
                name = "",
                type = "",
                shape = "polygon",
                x = 13.5166,
                y = 0.500615,
                width = 0,
                height = 0,
                rotation = 0,
                visible = true,
                polygon = {
                  { x = 0, y = 0 },
                  { x = 5.00615, y = 5.79283 },
                  { x = 0, y = 11.0135 },
                  { x = 4.29098, y = 20.0246 },
                  { x = 3.9334, y = 24.8162 },
                  { x = -1.21578, y = 31.2527 },
                  { x = 18.2367, y = 31.3242 },
                  { x = 18.2367, y = -0.429098 }
                },
                properties = {}
              }
            }
          }
        },

This is a tile of a coastline terrain-brush, and the collision is the shape of water in that tile. The sub-field in the tile objects has the shape of the collision polygon. In my own parser, I was thinking I'd keep a separate map.collisions that holds the shapes of these (along with the gid of the parent tile they are embedded in) + shapes on collidable layers, so it'd be faster to loop over them all at once, and check for collisions in other libraries.

I will look into doing this, in STI plugin-space, with HC (which can support more complex polygons than bump) and if I can get it all working, I will make a PR to add the plugin for STI. It might be a good general approach to physics/collisions, and make it easier to write other plugins (they can just loop over map.collisions, instead of worrying about layers.)

Sorry for all the back-and-forth, and thanks again for all your help.

@konsumer
Copy link
Author

konsumer commented Feb 10, 2021 •
edited
Loading

I might even be able to pull out map:getImageForGid(gid) that jumps through all the hoops, and returns an image (by pulling out the quad and image from tileset) so you have a drawable all ready to go for your tile-objects.

@konsumer
Copy link
Author

I made a plugin called yz that I think does mostly the right thing, but it's flashing when 2 objects overlap. I sort the table by Y, on update.

I overwrite the layer.draw, like this:

layer.draw = function()
    for _, object in pairs(layer.objects) do
        -- this allows overwriting with custom in-place drawing (like animation or whatever)
        if object.draw_yz then
            object.draw_yz()
        else
            if object.yz_tile then
                local x, y, w, h = object.yz_tile.quad:getViewport()
                lg.draw(object.yz_tile.image, object.yz_tile.quad, object.x, object.y-h)
            end
        end
    end
end

I am pretty sure the object.yz_tile is setup right, because it's displaying all the right tiles. Any idea why it's flashing? I also noticed I needed to do object.y-h where h is tile-height, otherwise it was off by 1.

@konsumer
Copy link
Author

here is the whole thing, for context.

@konsumer
Copy link
Author

As a sidenote, if I disable the sort on update, it stops flashing, but also stops negotiating z-order.

@karai17
Copy link
Owner

karai17 commented Feb 11, 2021 via email

@konsumer
Copy link
Author

konsumer commented Feb 11, 2021 •
edited
Loading

Yeh, I tried <= and < with same prob. I also tried putting it in draw() instead. The only thing that fixes it is to disable the sort, so it seems like it's not z-fighting (as the same number of things occupy the same space without making the problem.)

I read somewhere that the nil in draw-table has problems, so I did this:

function map:yz_sort_objects(a, b)
    if a and b then
        return a.y > b.y
    end
end

layer.update = function(dt)
    for i,o in pairs(layer.objects) do
        if not o then
            table.remove(layer.objects, i)
        end
    end
    table.sort(layer.objects, map.yz_sort_objects)
end

but that also failed. This seems like not a STI problem, just can't figure out how to make it work.

@karai17
Copy link
Owner

karai17 commented Feb 11, 2021 via email

@konsumer
Copy link
Author

my suspicion is that it changes a bit every frame

Isn;t that what I'm shooting for?

It moves too fast, with a regular print, so I did this in my top-level draw():

for o, object in pairs(self.map.layers.objects.objects) do
  love.graphics.print(object.id, 0, (o-1)*12)
end

2021-02-11 03 02 20

It definitely shouldn't be swapping order that fast.

@konsumer
Copy link
Author

konsumer commented Feb 11, 2021 •
edited
Loading

In the sort function I noticed there were nils so I tried this too, with no effect:

layer.update = function(dt)
    for i,o in pairs(layer.objects) do
        if not o then
            table.remove(layer.objects, i)
        end
    end
    table.sort(layer.objects, map.yz_sort_objects)
end

@konsumer
Copy link
Author

konsumer commented Feb 11, 2021 •
edited
Loading

weirdly first, middle, and last seem to stay the same.

@karai17
Copy link
Owner

karai17 commented Feb 11, 2021 via email

@konsumer
Copy link
Author

yeh, I was thinking that about pairs(), like issue reminds me of the old days of javascript with associative-array objects. Like the order used to be random (or at least "not guaranteed") unless you used an array to keep the order. I could build a separate numeric array of just the gids, then lookup in the sorting function. I will experiment with that and pre-setting Z, then adjusting it based on Y. It seem unlikely to me that they are all on the same Y value, which would cause the whole "I don't care which order because they are the same Y" problem, in sorting, so it must be some other problem with sort.

@konsumer konsumer mentioned this issue Oct 11, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@konsumer @karai17