-
Notifications
You must be signed in to change notification settings - Fork 115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rewrite/v2.0 General Discussion Thread #50
Comments
@movitto, @alexeld, @AdrienMorgan, @delvarworld, @stemkoski - Just tagging you guys to let you know the existence of this thread. If you have anything to add, please do. Not expecting anything, but comments/criticism of the current version would be very handy. Cheers. |
hey @squarefeet thanks for creating this thread. I read SuckerPunch presentation on their particle engine for their ps4 game "infamous". The slides are at:
but this is only for the sake of behaviour of particles themselves, another thing is appearance and there things like reaction to lights/shadows and casting of shadows is apparently a lot more important, I'm quite certain it would be pretty hard to formulate without heavily coupling with some particular rendering pipeline, so I'm just leaving it out there. Trouble with encoding things like "spiral" or "straight line" or whatever else is an interesting motion shape someone might want directly into the engine is lack of flexibility. I believe a great engine is:
if you could supply emitter with following position function:
you could essentially give people freedom to define spirals, however it would give them a lot more than that. And maybe it is a good way, to just allow injection of a snippet of code into shader, where shader includes some useful function like noise inside of it already. Another thing in terms of implementation that i'm quite certain of - it has to remove JS out of the loop almost entirely, best option for that right now seems to use 2 buffers and swap them between simulation steps, use one for lookup and other for writing new state. I believe there are better ways potentially, but they aren't supported by what's enabled in a browser by default (without toggling flags in chrome or equivalent elsewhere). From my personal experience - keeping things as generic and as simple as possible usually pays out in the long run. |
I agree on what @Usnul said about injectible function. An injectable functions means fully generic system. Adding a gravity would be as simple as
and you can switch from a cylinder to a sphere easily by changing two parameters. If you've heard about superformula you'll definetely understand what I mean. This was also an issue I felt when I was coding Hydrogen atom probablity density viewer with SPE because I had a formula for probablity distribution and I had to inject it to place points to the related coordinates but the current library didn't help me about that and I had to change many parts. This also may lead an issue about easiness of code because a developer should find out the right formula to generate desired shape of particle system. We should consider it. |
I don't really see this as an issue @cihadturhan, there is an option of creating a library of these parameterizable Emitters, something along the lines of:
let people use these as much as they like, I'd imagine it would serve large portion of users. PS: |
Yep, on a second thought I think that won't be a problem. Thanks @Usnul |
This is all great stuff. I'd never thought of having custom position calculations... I'm assuming these will either be written in GLSL and injected at runtime (or, if an emitter is already running and its pos calculation is changed, it's shader's rebuilt)? For the sake of playing devil's advocate, I have some issues with it:
To argue against myself:
I'm liking the idea of moving the functions that currently reside in What I would absolutely love to be able to do is get values back out of the shaders. At the moment (and please do correct me if I'm wrong) but I'm pretty sure that we can only whack values into a shader, but not get the result back into JS. WebCL should solve this problem, but I believe that is quite a way off from being included in browsers as standard. Dreaming aside, though, this is a good step forward. @cihadturhan: That hydrogen density sim is really awesome - great work! |
PS. Just done a v. quick lookup on the state of WebCL. The 1.0 spec was released in March this year, but Firefox, at least, isn't going to implement it. They seem to be erring on the side of OpenGL ES 3.1's compute shaders instead. I haven't yet found a timeline for ES 3.1's adoption. |
Hey @squarefeet, I like your reasoning. Let me try and address some issues:
I could see every emitter class (not instance) use a separate shader, but at the same time - even hundreds wouldn't present much of a challenge, when thinking about games today that employ thousands of shaders at the same time. |
here's a promising thread from three.js on the similar concept of using double buffering: |
A couple of things,
|
@cihadturhan I'm a little confused on this point. if we use an FBO - then vertex shader is pretty much excluded, you have a quad and that's it, each fragment however represents a particle, and looks up position of that particle from a texture (FBO). Having UV (0.5,1) for instance, we'd go to texture and lookup pixel at at that location, then we'd unpack state from it, including position of the particle. In a way UV only serves as ID of that particle. |
@Usnul I understand how you want to use FBO but I have no knowledge on how FBO works in deep. Will the coordinates change in the next frame like in |
@cihadturhan here's a basic rundown:
okay, that's great, and here's what's happening inside fragment shader:
the proviso here is that texture for lookup and render targets are identical in structure, this is why you can quite easily identify where to write, and why you can swap them too. As you can see, particle state would correspond to a specific pixel, so it is preserved between render cycles. Inside the shader you have access to that state and also have the opportunity to use that "old" state when creating new one. A simple static shader, for instance, could just copy old state so that new state remains the same as the old one. |
@Usnul Thank you very much for explaining, I'm new to texture stuff. I see what you want to do and it's very clever 👍 If the number of particles are n * m, then we'll use all of the pixels :) PS: we can assign m = 1 everytime so that's not a problem though. |
@squarefeet @Usnul this all seems reasonable / cool. Am also relatively new to GLSL so can't comment too much but agree it would be awesome to parameterize the shader w/ custom algorithms (if it's feasible / practical). Also agree on addressing the performance implications, perhaps in addition to the solutions presented above, we could support multiple modes w/ different shaders optimized for different scenarios. In general would be good to try to do this as a pluggable architecture, where at the base we present simple classes / interface able to be extended/used for custom scenarios. Would be glad to help where/when I can. |
@Usnul The FBO stuff does certainly seem to overcome our problem with particle state between render cycles - great call to use that. From my extremely limited understanding, though, wouldn't we only have 4 values per pixel to describe a particle's state (R/G/B/A)? I think I'm just missing the connection between how I describe a particle's state in my mind (pos, color, velocity, acceleration, opacity, angle), and how the shader would describe the particle's state using an FBO. Though I guess we would only really need to store the position? Maybe, if necessary, we could use more than one texel to describe a vertex... As far as linking a particle/vertex to a specific texel co-ordinate, could we not just have a FloatArray that stores UV co-ords? Obv. two entries per vertex ( uniforms: {
texels: new Float32Array(...)
}
attributes: {
coords: { type: 'f', value: [...]
} I know I'm probably bringing the discussion backwards a little here, but I just want to be sure I fully understand your proposal :) @movitto Great to have you on board. I think it's more than feasible to have custom shader algorithms - as @Usnul mentioned, shader re-compilation doesn't seem to pose as much of a problem as we first thought. That, and the changing of shader algorithm(s) during runtime would probably be a small use-case anyway. The idea of different shaders for different scenarios is a good idea - I reckon for ease of development, though (and for DRY purposes), it would be prudent to make our shader code as modular as possible; broken up into small-ish chunks that we could just tie together to make new shaders as and when necessary ( similar to what THREE.js does ). That said, I think that's what you might've meant when you talked about the 'pluggable architecture'! |
Okay, been doing a bit of reading into the FBO stuff, and @Usnul - I followed your link to the double-buffering thread in the THREE.js repo. It helped quite a lot, thanks. About the 4 values p/pixel issue: looking at this example of GPGPU boids/flocking (which is just stunning, btw) from here, it appears that @zz85 is using multiple textures per "particle" (one for position, another for velocity, etc.) Following this pattern would solve the issue of only 4 values p/pixel. His SimulationRenderer.js file would be a good inspiration for the FBO-swapping... |
Just got a quick practicality question:
What would you guys prefer? At this stage, it'd just be sketches, tests, etc. |
I'd say a SPE mark.2 would make sense :) |
So I've been getting my head around GPGPU/FBO/render-to-texture stuff, and I've re-written the THREE.js gpgpu birds example. It sounds a little strange to have done that but it gave me a good way to learn FBO stuff, as well as make a base "class" that we can use in the next version of the particle engine. I've submitted this as a possible replacement for the existing SimulationRenderer for THREE.js here. Hopefully I'll get some useful feedback either here or on that thread and it can move forward into a useful little "class". In short, this version of the Simulation Renderer takes care of generating data textures, swapping renderer targets / buffers, and automatically passing generated textures into shaders. It's fairly straightforward to use, so I haven't added many comments, but let me know if it's not as straightforward as I think it is and I'll comment the crap out of it :) @Usnul: Once this SimulationRenderer is declared usable/useful, then I'll start up the SPE v2.0 repo. |
I'm quite comfortable with FBO stuff now, so I'm starting to think about how we should organise what textures hold what data. To start with, lets keep things simple and have the following per-particle attributes:
We could have one texture for each of these, but I'm wondering if we could pack some of them together. Like using the unused We could also work around having one render-to-texture pass for each of the textures as well - maybe by using the velocity calculations from the existing SPE shaders that extrapolate velocity values based on particle age. These velocity calculations could be integrated into the position texture shader directly, so that's one less render pass to do. This may cause problems later on when we come to add vector fields and stuff like that, though. So... Issues that need discussing:
|
could make a mapping mechanism for variables, something along the lines of: |
I've been playing around with a few things tonight and wanted to get feedback on a possible "core" API. I'm not going to paste any of the internals I have going at the moment, because they're an absolute mess, but as far as using the thing goes, this is what I have: var myParticles = new Particles();
myParticles.addProperty( 'acceleration', new THREE.Vector3( 0, 1, 0 ) );
myParticles.addCalculation( 'velocity', 'velocity += acceleration * delta' );
myParticles.addCalculation( 'position', 'position += velocity * delta' );
For each Note that I haven't defined any initial values for Also, for each Oh, and one more thing, I haven't done anything re. emitter groups. Just this core API work. Thoughts? I'm thinking that whilst this gives us the most freedom we need (particularly taking care of custom position calculations), it does seem to possibly be at the cost of usability. |
that looks pretty good @squarefeet. Regarding groups, i suggest using a hash of sprite URLs: var sprites = {};
function Particles(url){
var texture;
if(sprites.hasOwnProperty(url)){
texture = sprites[url];
}else{
texture = THREE.TextureUtils.load(url);
sprites[url] = texture;
}
...
} also, regarding syntax: i do think it's verbose, but there are ways of reducing that. One way would be to write a glsl parser which would be a small investment and would probably be done later in the project. Another way would be to do something like this: myParticles.properties.add('acceleration',new THREE.Vector3(0,1,0));
myParticles.calculations.add('velocity += acceleration * delta')
.add('position += velocity * delta'); parsing this would be relatively easy, as all you'd have to do is tokenize it and exclude reserved tokens like float, int etc. and only keep ID tokens ([a-zA-Z_][a-zA-Z0-9_]*) |
I'm not sure why we'd need a GLSL parser? EDIT: Ah, wait. I think I understand why now... you've removed the first argument for the |
yup :) |
glsl parser project in JS: |
I've taken a look through this, and it's absolutely superb, @Usnul. Exactly what I was looking to do. I reckon it'll serve as a good base for sorting out FBOs and assignments :) I want to have a good ol' tinker with it, and see if I can build up a double-buffered render chain from it. Once that's done, I'll start trying to work it into an emitter to see how it holds up. I'll open up a new repo for all of this v. soon. Thanks! |
Hey guys, So... it's been a long, long while since the last message in this thread, so quite a lot to update you all on:
About the alpha version:
I would love to try to get particle lights implemented but I fear I'd have to crack open Three.js's internals to do that, which would be beyond the scope of the project. Having the particles react to a scene's lights might be do-able, though... Must look into that. Talking of which, I'd like to get shadows implemented, but that brings with it a whole new issue of particle sorting, which is a very expensive operation. I've experimented with radix sort on the GPU, but it's not smooth enough for my tastes (it sorts them over multiple frames). There are also multiple ways of setting most parameters. For example, you can set the velocity value to be a THREE.Vector3, or a set of THREE.Vector3s that will be linearly interpolated through over the age of the particle, or as a random value between minimum and maximum bounds. I'm working on getting parameters to support sets of values that will change over the lifetime of the emitter as well as the lifetime of a particle. These value options replace all of the So yeah, it's a very modular system. Create an emitter, then create as many modules as you want (each with various parameters) and add them to the emitter. Pretty much it. There's currently no grouping, though, and I'm not sure whether this is a viable with the architecture I've made so far. With all that said..! At the moment, I'm running only four passes regardless of how many modules are enabled (spawn, forces [vel., accel., noise, basically anything that affects velocity], color, and position/draw), and things are speeding up rather nicely. It also helps that I'm pushing less data to/from the GPU. As far as custom calculations go, that's going to be added quite near the end of this re-write but it's at the front of my mind whilst I'm writing the rest of it. With everything running on the GPU and JS only really doing the buffer swapping, there's not really a way to get per-frame access to a particular particle's properties. Luckily, in this re-write most of the parameters are dynamic, with only the one texture holding both velocity and position data. So things can still be changed on a per-frame basis without repopulating the texture buffer (which is an exponentially expensive operation). I think that pretty much covers most things... hit me with any questions. It might be hard without seeing any code, but if you want examples of it's current state, just let me know and I'll do some copypasta. I'll whack it up here under a new repo once I'm happy with it and it's stable enough for people to poke around and experiment with. PS. Thanks @Usnul for pointing me in the direction of textures/FBO/GP-GPU. It led me down a wild garden path of joyous discovery! |
Wow, you've made huge amount of work. That's awesome. I'm looking forward to see the demos live. Is this blue one called curl noise? I'm on mobile now, didn't watch the video. By the way, if you're studying you can request github students pack which includes free github subscription for a year. |
Hey @squarefeet , that's amazing progress. Would be interesting to have a look at the code. You mentioned "per-particle" access, i'm actually not too worries about that since you can always use canvas operations to from JS side to work with pixels in a buffer, or if it's a binary array - things are even simpler. Would be silly to manipulate all particles this way and defeat the purpose of GPU engine, but for some small pools of say 200 particles or so it could be a nice addition. Regarding particle lighting, there's an interesting introductory set of slides which you probably have seen already: I've been reading around particle engines for a while now, and i'm quite convinced now that integration and dynamic properties of particles are more important than count overall. By that I mean aspects which make particles "feel" more like they are a part of the world, here are a few examples:
I've seen many examples where only a handful of particles (>500) were used to achieve visually amazing effects. That's not to say that really high particle count is a great thing on its own :) One more piece to consider is screen-space, some effects such as rain or snow are often done in screen space, because you can get higher density in 2d with same count than you can in 3d. Lastly, regarding cracking open three.js - it's remarkably nicer than it was several months ago, shaders for all materials are now composed out of fragments (chunks/snippets), so you might be able to create a custom material by manipulating order of these chunks and introducing your own. At the end of the day, i'd personally be more in favour of having strong support for Three.js, but no core integration, allowing particle engine to be used with different rendering engines. Babylon is getting a lot of traction, and there are many projects which go beyond stock three.js. |
Hey guys, @cihadturhan: Yep, that's curl noise. In both of the screenshots, actually. I'm past my studying days - I just take free things where available ;) @Usnul: I'm totally with you re. appearance of particles more important than the count. Sure, higher counts are more impressive initially but really of little real-world value (unless of course, you're building a tech demo!) I think once I've got the new shader up and running (I've called it BigDaddyShader... it does velocity, position, colour, and as-yet-undefined "other" all in one pass, yay) and functioning with the modules from the previous rewrite, I'm definitely going to start tackling the task of using depth buffers to do various things. I'm definitely keen on getting soft particles introduced. Maybe that should be my first port of call instead of collision...I never make things easy for myself! You're right about THREE's internals, btw. I nicked the idea of using shader "chunks" to build shaders, and I daresay I'll be able to bastardise their lighting shaders to whack in my draw shader. Maybe even the shadow stuff. I think when it comes down to this engine being used as a part of a bigger system (games, environments, etc., rather than nerdy simulations), the sorting issue won't be too much to worry about, as the particle count will be fairly low. At least I hope so. About this library supporting not just THREE: Definitely in favour of that. I'll need to get my "native" WebGL chops up to scratch, but it's something I've definitely considered and researched already. I'm not too sure about Babylon, but I daresay if it's anything like THREE re. ease-of-use, then it shouldn't be a problem. The "native" WebGL version will be the trickiest IMHO. |
Using chunks is the most primitive way to generate code, nothing wrong with primitive, but there is a lot more to code generation. Next step is probably templating, beyond that is tree transform (aka parse-tree -> AST -> transform -> transform -> generate code from AST) You could use point cloud from threejs, but you would be bound to a vertex buffer then, which i assume isn't a good fit for your engine. Three JS works with shadows for geometry, so i doubt that you'd be able to make it work without changes. Good news is that shadow maps are pretty easy to code - it's just a B/W projection. Collisions. Bullet physics engine has recently been updated (about a year ago now actually) to do solving on GPU, but i'm not sure what kind of shaders they use - probably geometry or compute, still might be a useful source of info as it's open source. I wouldn't go beyond planes, cubes, spheres and maybe heightmaps for collision solving. I'm still certain that it would/will be a pain to do this with just the vertex and fragment shaders. |
Ha, I don't know whether going down the AST would is completely necessary, here; pretty sure chunking it up will suffice! I'm currently using the PointCloud... I'm not sure why being bound to a vertex buffer is a bad thing? Not sure of the alternative..? Unless you're thinking that shadows are more easily calculated/cast using faces? I'll definitely dig into the bullet engine and see how they're doing the depth collision. I'm also digging into UE4's internals to discover how they do their depth collision... hopefully I can get enough pointers. Failing that, this page has a very good tutorial. Just missing the bit about projecting a particle's position into clip-space (damn). Also, just because it's kinda pretty, here's a screenshot of the BigDaddyShader (I really need to rename that...) in action. Next up:
|
having vertex buffer is actually great, I think. The projection itself is somewhat easy, but for light you will need a normal as well as position, otherwise it's back to billboards. MAX_TEXTURE_SIZE is important to consider |
Very useful library, looking forward to the new features and flexibility. Just want to vouch for updating the editor with these changes too, as it is a really valuable tool for getting the right look for an effect (and made me decide that this was the library to use). |
@Usnul "BigDaddy" is just the internal filename for the shader... I don't think that's what I'm going to call the final version! Maybe as a nickname... ;) @eldog Thanks, I wasn't aware anyone was using the Editor! That has reminded me to push it to the master branch... Completely forgot to do that. I think there are some issues around exporting and whatnot, though..? There definitely will be an Editor for v2, but it will come once v2 is stable. I'm also going to get a site up for v2 specifically (that's not gh-pages), with some tutorials, maybe some videos, full and proper documentation, and the editor. Lots to do! |
Got stuck doing particle collision with the depth buffer... Posted a question on stack overflow and linking it here in case anyone has any ideas! |
i have a suspicion that you're not taking clipping planes into account (far, near). I'm not sure what space you use to resolve collisions, but if you are using screen-space - you would have to first project position of a particle into screen space coordinate system, otherwise you would have to apply inverse of that transform to depth buffer. If you are using the depth buffer - you'll have to get normals for texels as well, some SSAO algorithms implement inference of normals from depth buffer and can help with that. |
I've tried taking clipping planes into account, in fact I use that already in the draw shader: http://jsfiddle.net/1dchvbge/1/ (see the HTML section, and scroll down until you see the draw shader section) The problem I have is that I can't seem to project the particle position into screen space correctly. I need to use the particle's screenspace Once I manage to find out how to (correctly) project into screenspace without using gl_FragCoord, then the rest of it will be nice and easy... Famous last words, hey? |
Ah-ha! Figured it out. I've added an answer to my own question (!) on my SO thread: New fiddle showing it in action is here: |
I can't be bothered to put up a fiddle for this just yet, so here's a video I made of collisions with (probably buggy) collision response: Vimeo's processing it at the moment, so should be ready in about 30mins. Will post a fiddle link when I get around to it! Was recorded on my Macbook Air, so performance isn't great when capturing the screen. |
Okay, here's a fiddle: http://jsfiddle.net/vhw6o7ef/1/ |
yup, looks the same. Looks awesome actually :) |
have you seen this? |
I haven't... super impressive stuff. I came across his Flow stuff before, though. It's interesting how he does an incremental odd-even merge sort for the particles... Must look into that! |
There are also webworkers to assist with sorting, even mobile devices have more than 1 core. Allowing sorting to be incomplete could produce visual artifacts, but with large number of particles this may be an acceptable trade-off, considering you'd only need this when number of particles is large :) |
Ah-ha, good call... I'll take a look at how best that could be implemented :) |
Okie dokie, here's a quick update on the (latest) rewrite:
// Create emitter
var emitter = new Emitter();
// Set acceleration value. Behind the scenes, this sets
// define values in the shader to 'activate' the acceleration
// module.
emitter.acceleration = new THREE.Vector3( 1, 2, 3 );
// Create emitter
var emitter = new Emitter();
// Add a module.
// This style allows for much easier customisation. Custom emitters
// with custom GLSL calculations are SO easy, now!
emitter.addModule( new AccelerationModule( {
value: new THREE.Vector3( 1, 2, 3 )
} );
// Repeat adding various modules until emitter is complete.
// Unlike the rewrite before this, no need to call `emitter.prepare();`
// after all modules have been added.
So, I'm pretty sure I've hit a little bit of a jackpot re. combining the best bits from previous rewrites. I've just got to stabilise the emitter code, then tackle a few important internal issues:
Once I've rewritten the remaining modules into the new format, I'll definitely whack this up on Github, I promise! As ever, any questions, comments, or criticisms welcome :) |
I like this, "module" feels a bit off though. If you already have the whole shader compiled and only defs are being manipulated, maybe consider using a flag instead:
just a thought though.
|
I hear that, but with the module style, it will allow for custom modules to be created really easily... Once it's up on Github and you can see the Module code, I daresay you might agree :)
Good shout. I'll think a bit more about grouping, as I daresay it will affect some of the internals quite a lot. Something to do as soon as the Module stuff is fully supported by the Emitter.
About the spawn rate issue: It'll be a lot easier to explain once the code is publicly accessible. Quite hard to explain otherwise without a huge wall of text. Needless to say, it's not a floating-point precision issue! |
@squarefeet |
@Usnul Ha, braver than I! I'll try and put it up soon for you to peruse. In the meantime, here's something else that would be good to get your opinion on: Two constructors, both alike in dignity, in fair JavaScript where we set our scene... One constructor, the A question about the implementation... which do you think is more logical? var emitter = new Emitter();
emitter add( new ShaderPass( {
name: 'velocity',
// etc...
} ) );
emitter.add( new Module( {
name: 'acceleration',
value: new THREE.Vector3( 1, 2, 3 ),
affectsShaderPass: 'velocity' // Assign this module to the ShaderPass created above
} ) ); Or: var emitter = new Emitter(),
pass = new ShaderPass( {
name: 'Velocity',
// etc...
} ),
module = new Module( {
name: 'acceleration',
value: new THREE.Vector3( 1, 2, 3 )
} );
pass.addModule( module );
emitter.addPass( pass ); EDIT: Quick clarification: As far as the end user goes, most people won't have to create their own ShaderPass instances or Modules. There will be predefined ones in the library (like 'VelocityPass', CurlNoise module, etc.). This is mainly for custom stuff, but these constructors will also be used to create the predefined modules/passes. |
@squarefeet Seems like a "has a" relationship, maybe i'm just not getting something. Module seems to need a pass or many passes, then module should be responsible for a pass, no? And thanks for edit, for common use-cases this does seems like a beast of an API :D |
Yeah, that's what I was thinking as well - the 2nd option "feels" correct to me. Long story short, it goes like this: An A Also, if a And yes... it's a dirty ol' beast of an API, but that's the way I like it :D At least it'll make adding new modules/passes nice and easy in the future. Expansions made easy! |
Just had another thought: using this API, you could theoretically create a |
Following incredibly valid points from @Usnul and @cihadturhan, in order to get this library into the state that it deserves, a rewrite would appear to be necessary. This thread is for anyone to add their 2-cents/pennies/other-currency worth of thoughts on improvements, new features, etc. The more critical, the better (IMHO) - that's one of the major ways to ensure the new version is as easy-to-use and performant as possible.
I do ask that any requests/comments should follow the same format as below: one feature per bullet-point, and grouped into either Required or Nice-to-Have (or both, if you're feeling generous).
Thanks!
Required
vec3(0,0,0)
position.Nice-to-haves
The text was updated successfully, but these errors were encountered: