JoMME modifications

Get support for anything JK2 related, find tutorials for in-game and modding topics, and post any suggestions for the site here.
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

JoMME modifications

Post by Tom Arrow »

I was wondering how hard it would be to implement some fucked up rendering modes to JoMME for shits and giggles.

For example, I would like to simulate a rolling shutter. The algorithm would be very simple: divide each frame into XX time samples (where XX is the amount of pixels in the frame), then scan one pixel for each time sample and put them all together into one picture. It would read the pixels in a row after row basis. The result would be a simulation of the rolling shutter that is very typical in modern cameras.

One could control the strength of this effect through a variable that determines how much % of the frame duration should be used. One could also, if it is not possible to render pixel-based, make it faster by just taking entire rows at once.

That's just one idea, tho. I would love to be able to program this shit myself, but I don't know where to start. I found some sourcecode on github, but I don't even know how to compile it, how to understand it, or whether it is up-to-date.

Any info would be appreciated.
Kevin
Administrator
Posts: 393
Joined: 07 Jun 2015, 08:36

Re: JoMME modifications

Post by Kevin »

How do you know it's easy if you don't know much about programming? :)
User avatar
fau
Staff
Posts: 433
Joined: 16 Aug 2015, 01:01
Location: Warsaw / Poland
Contact:

Re: JoMME modifications

Post by fau »

Functionality you're describing is already present in jomme, it's motion blur with a half frame-time gap (don't remember exact cvar names). JK2 at 24 fps with rolling shutter simmulation looks awful though. I made few clip for comparison a year or two ago, but don't have this specific one uploaded.

I implemented motion blur and described it here (including rolling shutter example): https://github.com/aufau/jedi-outcast/b ... ME-MME.txt you can achieve the same in jomme, although cvars are different.
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »

Kameleon wrote:How do you know it's easy if you don't know much about programming? :)
Well, for one, I am a PHP developer. There is definitely a lot more to the native languages, but I am confident I can learn that stuff. I know some C++ basics too.

Second, I did not say it was easy. In fact, my initial question was "how hard would it be to...". I only said the algorithm was simple, because it is. It would be easy for me to implement this, say, in Javascript, if I had some straight-forward rendering API from which to fetch the frames. I don't know if JK2 features this, of course.
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »

fau wrote:Functionality you're describing is already present in jomme, it's motion blur with a half frame-time gap (don't remember exact cvar names). JK2 at 24 fps with rolling shutter simmulation looks awful though. I made few clip for comparison a year or two ago, but don't have this specific one uploaded.

I implemented motion blur and described it here (including rolling shutter example): https://github.com/aufau/jedi-outcast/b ... ME-MME.txt you can achieve the same in jomme, although cvars are different.
I looked up the file you linked, but all I could find in it was an example with negative blurOverlap for the "gap". This does simulate a shutter, but not a rolling shutter. Are you sure you know what I mean?

As for it looking awful (rolling shutter does), it's what I seek. 24 fps I am not interested in either way. Frankly, I don't get the hype.

Can you point me to some resources on how to start programming for jomme?
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »

fau wrote:Functionality you're describing is already present in jomme, it's motion blur with a half frame-time gap (don't remember exact cvar names). JK2 at 24 fps with rolling shutter simmulation looks awful though. I made few clip for comparison a year or two ago, but don't have this specific one uploaded.

I implemented motion blur and described it here (including rolling shutter example): https://github.com/aufau/jedi-outcast/b ... ME-MME.txt you can achieve the same in jomme, although cvars are different.
One more thing. That file says the JK2 engine can't run above 1000fps, which I doubt, because I recorded 2000fps scenes a few years back. I also tried joMME with 32 blurframes and 200 fps (6400 fps). Seemed to have worked without troubles.

Thanks for linking the file anyway, has some interesting stuff in there!
User avatar
fau
Staff
Posts: 433
Joined: 16 Aug 2015, 01:01
Location: Warsaw / Poland
Contact:

Re: JoMME modifications

Post by fau »

I thought you meant this: https://en.wikipedia.org/wiki/Rotary_disc_shutter

In original engine time is counted in milliseconds, so at 2000 fps every other frame is simply a copy of previous one. However there is a simple hack to get movement at fps higher than 1000fps which I wanted to describe, but looks like jomme already has it implemented so you're right, it should work.

How to start programming for jomme… do you know C?

Answering to your "how hard it would be?" question – rather easy (although making a usable cvar interface for this rather than one-time hack may be slightly more engaging). I understand JK2 is a medium-large project and can be intimidating at start, I could give you instructions what should be done.

Also keep in mind you could just render at huge fps and use some post-processing tool or a script that would generate rolling shutter effect combinig these frames, it would be awfully slow though. For vertical rolling shutter effect at 720p and 30 fps you'd need to render 720 * 30 = 21600 frames per second.
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »

fau wrote:I thought you meant this: https://en.wikipedia.org/wiki/Rotary_disc_shutter

In original engine time is counted in milliseconds, so at 2000 fps every other frame is simply a copy of previous one. However there is a simple hack to get movement at fps higher than 1000fps which I wanted to describe, but looks like jomme already has it implemented so you're right, it should work.

How to start programming for jomme… do you know C?

Answering to your "how hard it would be?" question – rather easy (although making a usable cvar interface for this rather than one-time hack may be slightly more engaging). I understand JK2 is a medium-large project and can be intimidating at start, I could give you instructions what should be done.

Also keep in mind you could just render at huge fps and use some post-processing tool or a script that would generate rolling shutter effect combinig these frames, it would be awfully slow though. For vertical rolling shutter effect at 720p and 30 fps you'd need to render 720 * 30 = 21600 frames per second.
No, I didn't mean rotary disc shutter, but I am glad that this one is already implemented. Cool stuff.

Yeah, well, this 'render at huge fps' thing is nothing I am willing to do. 21600 frames, exactly. And that's just for vertical rolling shutter. For full HD pixel-based rolling shutter at 50fps, you get 103.680.000 frames per second. Pretty undoable. :)

I know PHP very well and some C++ basics. But you wouldn't have to teach me that, there's plenty of material on the internet for the finer syntax questions; that is, outside of specifics that regard the JK2 code. Basically, my biggest questions are: 1. How and where to get the up2date code and how to compile it so that it actually works. (Windows 7 64bit) 2. How the project is structured and how to look at it; I guess, understanding the flow. And maybe 3. Some kind of basic steps, as you hinted at, for this idea of mine.
Last edited by Tom Arrow on 04 Apr 2020, 18:26, edited 1 time in total.
User avatar
fau
Staff
Posts: 433
Joined: 16 Aug 2015, 01:01
Location: Warsaw / Poland
Contact:

Re: JoMME modifications

Post by fau »

Yeah, I will have a look and try to describe how I would take on it. I'd rather do it here in public, maybe it will be useful to someone else.
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »

fau wrote:Yeah, I will have a look and try to describe how I would take on it. I'd rather do it here in public, maybe it will be useful to someone else.
Aight!
User avatar
fau
Staff
Posts: 433
Joined: 16 Aug 2015, 01:01
Location: Warsaw / Poland
Contact:

Re: JoMME modifications

Post by fau »

This is what I wrote so far, there are no suggestions on actual implementation yet. I have few ideas but tell me if you come up with something yourself first. Unfortunately it's harder to implement in jomme because video capturing code it's rather complex already.

First of all please use git, clone jomme repository from
https://github.com/entdark/jk2mp, set up your IDE and make sure
everything builds correctly. jomme README may contain required
information, I can't help you on Windows.

So our goal is to get rolling shutter effect, you wrote yourself that
method is rather obvious. How can coding it in the engine make
it faster than rendering thousand frames and combining then later
though? Hopefuly we can ask OpenGL to render only pixels we need from
a given frame. This how OpenGL and other graphics libraries work - you
request specific pixels and they figure out what needs to be done to
return correct color values. You can see this in action by moving part
of your jk2 window outside of your desktop - FPS should
increase. It doesn't increase infinitely, game engine needs to
run physics simulation and fill renderer with objects, effects,
entities etc each frame.

Now we need to figure out how to tell OpenGL to render only specific
row of pixels (only rows for now) and combine them into a frame.

I'll try to be brief and will explain things in more detail if you
have questions. First of all, how do you figure out where is coded a
functionality you're interested in modifying? Try thinking of an entry
point of some kind. These can be many things that appear in game and
you can find in code base by doing simple "grep" (string search),
usually for command and cvar names. Eg in jomme's case I'd start with
greping for "capture" command and explore from there, although you
don't need to do it for now.

JK2 code consists of several subsystems with varying
level of connectivity and abstraction between them. We will be
concerned mostly with two of them: client and renderer. Client code
resides in `client/` directory and controls all the non-server logic,
and calls other modules like renderer to do their work. The most
important client procedure for us is CL_Frame, which is being called
every frame from the main game loop. You can see what functions
it provides to topmost subsystems in the "client interface" part
of qcommon.h but don't bother understanding all of them now.

You may want to go through CL_Frame now without dwelling into other
procedures it calls like CL_CheckTimeout or CL_SendCmd. There is a
substantial code for us that initiates capturing frame starting with
`if (cl_avidemo.integer > 0 && msec)` line. `msec` comes to CL_Frame
as an argument and contains a number of millisecond that client should
move ahead in normal (playing game) scenario. When capturing you can
see it's adjusted depending on video framerate and motion blur.

You'll find re.Capture call in there. `refEntity_t re` is a struct
containing pointers to functions exported by renderer (or refresh)
module. You can see in `renderer/tr_init.cpp (GetRefAPI)` a line:

re.Capture = R_MME_Capture

so `R_MME_Capture` is where re.Capture call redirects us to. Now
if you look at the content of R_MME_Capture, there is another thing
that requires some explanation. It is a special type of procedure that
command of type "RC_CAPTURE" to the "render queue". Render queue
is simply a queue of renderering instructions that will be executed
when drawing a scene. Each render command goes to the queue together
with few parameters, in this case fps, focus, radius and name. To find
out how RC_CAPTURE gets interpreted, search for "RC_CAPTURE"
references in renderer code. There is only one more
in `renderer/tr_backend.cpp (RB_ExecuteRenderCommands)` and this
is where commands from render queue get finally executed. Now
R_MME_CaptureShotCmd is clearly not the place where anything gets
captured. It only configures shotData structure so it can be used
somewhere else. If you search for shotData.take tests in the same
file, you'll find out where. Not going to explain why it works this
way, but this is a rather unfortunate design decision.

Finally we arrived at R_MME_TakeShot where the actual work gets
done. The procedure is a bit complicated since jomme does a lot more things
than simply taking screenshots each frame.

As you can see even navigating in JK2 source code needs some
understanding how subsystems are connected and work.

Now how do we want to implement our row-by-row capturing? There
is already somewhat similar motion blur
functionality. blurControl->totalFrames holds the number of frames to
be combined and blurControl->totalIndex holds current frame index in a
cycle. We can either merge it with motion blur code (how do you apply
motion blur to rolling shutter?) or write a separate procedure that
will disregard motion blur or any other effects (easier?).
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »

Alrighty! Thanks for this writedown. I am only doing this by-the-side, that's why I respond so lately.

I have downloaded the project and opened it in Visual Studio 2013 Express. I tried a quick compile (just hit the Build button) and got this error:
error C1083: Cannot openinclude file 'ft2build.h': No such file or directory.

It comes from line 81 in the file tr_mme_font.cpp. I assume I need to somehow download the FreeType sources and use them, but where do I put them so that Visual Studio recognizes them?

As for the rest of what you wrote, I looked into the code. The main capturing seems to happen in 'R_MME_GetShot'.

This is the code that creates the actual data:
qglReadPixels( 0, 0, glConfig.vidWidth, glConfig.vidHeight, GL_RGB, GL_UNSIGNED_BYTE, 0 );

Then it seems to be fetched by this code:
qglBindBufferARB(GL_PIXEL_PACK_BUFFER_ARB, pboIds[index]);
GLubyte* ptr = (GLubyte*)qglMapBufferARB(GL_PIXEL_PACK_BUFFER_ARB, GL_READ_ONLY_ARB);

I can't find an actual writedown of that function, so I assume it's some kind of OpenGL API thing. (Sorry, never did anything with game or 3D code)

Anyhow, it looks easy enough to specify a single pixel for reading. Depending on the API, it may be possible to do the qglReadPixels call for each pixel and then keep the rest of the code as it is. Or it may be necessary to do the read-out after every pixel and then somehow put all the resulting data together into one variable.

I suppose the other big question is: How do I, within R_MME_GetShot, advance the time for a few nanoseconds and then make another read-out? (if possible at all). I assume that there are separate functions that actually move all the objects etc based on the time and I assume these functions are somwhere in the main loop. Although I wonder how this plays together with R_MME_TakeShot, as this one seems to already incorporate all the motion blur stuff etc, without referencing any time variables at all.

I guess I'll get a better understanding if I take a deeper look at the code, but if you can provide any quick answers to guide me through delving into the code further, I'll be happy.

As for how to implement it, I would opt for (if possible) adjusting the code assuming that motion blur is disabled, which should do the trick for a quick-n-dirty approach. Once I get that working, I can think about how to make it work together with the motion blur. Or not, depending on how difficult that would seem.
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »

Btw, one "physically correct" way to implement motion blur with rolling shutter, I think, would be to apply the motion blur to each pixel.

For instance, if we had an fps of 25 and 10 blurFrames and a capture resolution of 10x10 (just for example's sake).

We'd have an effective fps per pixel of 25*10*10 = 2500 fps. Now, my first idea here is to simply take each pixel and further divide by 10 blurFrames into 25.000 fps and blend the 10 sub-pixels into one, but that is not really the right way. A real camera would hardly get enough light if it worked like that. Also, there would hardly be any noticeable blur, as the movement would be too miniscule.

Instead, it would actually need to be blurred over the timelength of the whole frame. That is, we would have 25*(10 blurFrames) = 250 fps for the blurring. But then, the rolling shutter would be gone with the current implementation.

Now, how it really works in a camera is that after a pixel (or, more probably, a row) is read-out, it already starts to capture light again. Which means that there is simply a delay for each pixel.

Let's say we start at sec 1.

The first pixel would consist of 10 pixels from the following secs: 1, 1+1/250, 1+2/250, ..., 1+10/250.

The second pixel would consist of the same sec positions with a slight delay of the pixel-duration of 1/2500. In other words: 1+1/2500, 1+1/2500+1/250, 1+1/2500+2/250, ..., 1+1/2500+10/250.

The last pixel would use these times: 1+100/2500+1/250+1+100/2500+2/250, ..., 1+100/2500+10/250.

In other words, the whole rolling shutter really is nothing more than a delay based on pixels (or rows).

A pixel-based rolling shutter with motion blur would likely be much too slow for practical use, but maybe a row-based rolling shutter with motion blur can be implemented.

Another way to control the rolling shutter then would be to specify a kind of ... yeah, shutter angle. 360 shutter angle would mean that the delay of a pixel is basically framelength/amount of pixels, which would be rather extreme, especially on low framerates. Then, say, 90 shutter angle would mean that the delay is framelenght/amount of pixels/4. Which would be more realistic.

I suppose most of this speculation, for now, is just intellectual jerking off. Let's see what the code can actually support first. It will take some time for me to wrap my head around it anyway.
User avatar
fau
Staff
Posts: 433
Joined: 16 Aug 2015, 01:01
Location: Warsaw / Poland
Contact:

Re: JoMME modifications

Post by fau »

Tom Arrow wrote:Alrighty! Thanks for this writedown. I am only doing this by-the-side, that's why I respond so lately.

I have downloaded the project and opened it in Visual Studio 2013 Express. I tried a quick compile (just hit the Build button) and got this error:
error C1083: Cannot openinclude file 'ft2build.h': No such file or directory.

It comes from line 81 in the file tr_mme_font.cpp. I assume I need to somehow download the FreeType sources and use them, but where do I put them so that Visual Studio recognizes them?
I've heard jomme has somewhat broken build system on windows. Try setting BUILD_FREETYPE preprocessor macro to 0 in cmake.
As for the rest of what you wrote, I looked into the code. The main capturing seems to happen in 'R_MME_GetShot'.

This is the code that creates the actual data:
qglReadPixels( 0, 0, glConfig.vidWidth, glConfig.vidHeight, GL_RGB, GL_UNSIGNED_BYTE, 0 );

Then it seems to be fetched by this code:
qglBindBufferARB(GL_PIXEL_PACK_BUFFER_ARB, pboIds[index]);
GLubyte* ptr = (GLubyte*)qglMapBufferARB(GL_PIXEL_PACK_BUFFER_ARB, GL_READ_ONLY_ARB);

I can't find an actual writedown of that function, so I assume it's some kind of OpenGL API thing. (Sorry, never did anything with game or 3D code)
qgl* functions correspond to the same OpenGL functions with gl prefix instead of qgl. You can easily google their documentation: https://www.opengl.org/sdk/docs/man2/xh ... Pixels.xml
Anyhow, it looks easy enough to specify a single pixel for reading. Depending on the API, it may be possible to do the qglReadPixels call for each pixel and then keep the rest of the code as it is. Or it may be necessary to do the read-out after every pixel and then somehow put all the resulting data together into one variable.
Single pixel per frame won't work. Let's do some estimations. Game engine does not only render frames, but also runs game physics, processes user input etc. each frame. No matter what you do it probably won't run at fps higher than few thousands. If you want to grab only one pixel each frame, at 720p you need 1280 x 720 = 921600 frames to generate a single "combined" frame! 921600 / 1000 = 921,6 seconds for a single output frame. This is way too slow for any practical usage. I thought of capturing one row per frame, this means only 720 in-game frames for one output frame.

On a sidenote,
another idea came to my mind for approximating pixel per frame rolling shutter. Simply calculate linear interpolation of each pixel's color. Say you have captured two adjacent frames A and B separated by 0.025s. pixel with coordinates (x, y) should be captured 0.025 * (x + 1280 * y) / 921600 seconds later than frame A. So you can approximate it's color value using linear approximation: C = A + ((x + 1280 * y) / 921600) * (B - A). Repeat for each pixels and if original framerate was high enough, effect should be convincing, although there are corner cases where it won't work very well. One could implement it as a funny filter for some video editing program.
I suppose the other big question is: How do I, within R_MME_GetShot, advance the time for a few nanoseconds and then make another read-out? (if possible at all). I assume that there are separate functions that actually move all the objects etc based on the time and I assume these functions are somwhere in the main loop. Although I wonder how this plays together with R_MME_TakeShot, as this one seems to already incorporate all the motion blur stuff etc, without referencing any time variables at all.
Just advance whole game by one frame. R_MME_TakeShot doesn't generate blurred frame in one shot. It accumulates (see "accum" functions) current frame to motion blur buffer and only when it combines enough these frames blended together create a single, blurred output frame.
I guess I'll get a better understanding if I take a deeper look at the code, but if you can provide any quick answers to guide me through delving into the code further, I'll be happy.

As for how to implement it, I would opt for (if possible) adjusting the code assuming that motion blur is disabled, which should do the trick for a quick-n-dirty approach. Once I get that working, I can think about how to make it work together with the motion blur. Or not, depending on how difficult that would seem.
Quick comment on your second post (haven't noticed): instead of grabbing the same row for few frames and combining them, you can just grab few rows each frame and combine these rectangles for the same effect. When you look at it this way you can see that motion blur doesn't produce much overhead at all because you need to render all these frames anyway for rolling shutter effect.
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »

I managed to include the Freetype2 library through the include paths, but I get dozens of syntax errors. It almost seems like it is written in a different language or something. You advise me to compile without FT, but is FT not necessary for any text to be displayed in-game or in the menues? It would be rather hard to work with JoMME without any text display at all. (Wild guess about it being necessary for text display)
qgl* functions correspond to the same OpenGL functions with gl prefix instead of qgl. You can easily google their documentation: https://www.opengl.org/sdk/docs/man2/xh ... Pixels.xml
Thanks, I found that out too. :)
Single pixel per frame won't work. Let's do some estimations. Game engine does not only render frames, but also runs game physics, processes user input etc. each frame. No matter what you do it probably won't run at fps higher than few thousands. If you want to grab only one pixel each frame, at 720p you need 1280 x 720 = 921600 frames to generate a single "combined" frame! 921600 / 1000 = 921,6 seconds for a single output frame. This is way too slow for any practical usage. I thought of capturing one row per frame, this means only 720 in-game frames for one output frame.
Well, only trying out wil provide an answer to that, but your pixel-approximation idea is funny as hell, might as well try it. Besides, the pixel-based effect of rolling shutter I can probably live without. It is hardly noticeable anyway, my guess.

My JK2MV runs at about 200fps, and that is with 32xCSAA and whatnot. Somebody on a server said that his JK2 runs at 1000fps, which I guess is easily posssible without the fancy GPU stuff. I wonder how much of that goes for game physics etc and how much for actual rendering. We will see.
Just advance whole game by one frame. R_MME_TakeShot doesn't generate blurred frame in one shot. It accumulates (see "accum" functions) current frame to motion blur buffer and only when it combines enough these frames blended together create a single, blurred output frame.
Ah, aight. Seems I have some understanding to do yet.
Quick comment on your second post (haven't noticed): instead of grabbing the same row for few frames and combining them, you can just grab few rows each frame and combine these rectangles for the same effect. When you look at it this way you can see that motion blur doesn't produce much overhead at all because you need to render all these frames anyway for rolling shutter effect.
You mean, taking the whole frames and then simply combining the relevant rows for each final shot? Well, I suppose we better put this on the back burner, until I get anywhere with the basic effect.
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »

Aye, I successfully compiled with your tip to switch off that flag.

Now, I can start it and I can start a demo and I see the overlay of MME. Alas, all the key controls don't work and although the 'capture' command seems to work in the console, it doesn't list any 'mme_' or '_mov' cvars. Maybe I have an outdated version of JoMME or something? I got it from here: https://github.com/entdark/jk2mp/
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »

I take that back. The 'mme_' and 'mov_' cvars are there, but the key controls still don't work. But I am having the guess that those key bindings are in the old cfg that came with the actual release ...
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »

Okay. It works now. Cool. Sorry for those useless posts. I'll report back when I find out anything interesting.
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »

Aight, I have a real question.

CL_Frame is what is called when the actual frame is already 'taking place'. But I don't see any game physics stuff in it. At best, I can find a call to SCR_UpdateScreen, which seems to run some kind of 'VM' (?).

Where's the mechanism that actually calls the CL_Frame and decides, based on framerate for instance, what frame will be served next, and does all the game physics stuff and the likes? I feel that's important when it comes to having the right next frame loaded. Or does it really all work with the fps parameter that is passed to re.Capture? Anyhow, I'd like to understand this whole loop more deeply, or I'll be forever walking in the dark.
User avatar
fau
Staff
Posts: 433
Joined: 16 Aug 2015, 01:01
Location: Warsaw / Poland
Contact:

Re: JoMME modifications

Post by fau »

VM = Virtual Machine. In id tech 3 there are 3 modules (subsystems) which can be loaded either as native shared libraries or as special qvm code for quake virtual machine. This separation exists to make modding the game easier, their source code was released in old SDKs and you can easily distribute them as mods:
game - controls server mechanics
cgame - controls client mechanics
ui - draws UI
They can be accessed only with VM_Call procedure, eg. VM_Call( cgvm, CG_DRAW_ACTIVE_FRAME, clc.serverTime, …) is required to draw (and play) current frame using information from server or demo data.

Game physics stuff is mostly server-side but during demo playback it doesn't need to be evaluated, game physics snapshots are saved in demo files. cgame module handles mostly interpretation and presentation of this data.

What calls CL_Frame? The main game loop. If you configured your IDE correcty you should be able to examine it yourself. msec argument to CL_Frame contains current game time in milliseconds, although it's ignored when rendering video because every frame needs to be captured regardless of framerate. It's overridden in if (cl_avidemo->integer > 0) block.

If you're interested in overall engine architecture you can read some rough explanation here: http://fabiensanglard.net/quake3/ but it's not really necessary for what we're doing. There are many subsystems doing different jobs, you don't need to know or understand all of them.
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »

I tend to get obsessed with understanding all the finer details of code, otherwise I feel ... stunted. I'm weird that way. This is why I hate working with frameworks. I can't just let things be and work with the API, heh.

Anyhow, that is interesting.

I played around a little with the code and here's a little thing I did: I replaced the msec variable and other variables that save time from int to float. I had some difficulties, but I can now use a very low timescale like 0.001 in-game (not while recording) and it works, althoguht it stutters a little. A problem I encountered was that I had to convert the float to int for the VM_Call of UI_Refresh and CG_DRAW_ACTIVE_FRAME, otherwise the game just fucked up (weird thingies in main menu stopped moving and game crashed after loading demo).

My guess is that the VM_Call has no explicit types (how could it ...) and the float-bits get simply interpreted as integer-bits which causes some confusion. Now I did find the _UI_Refresh function for instance and it does have int in its declaration. I am wondering if it is possible to pass a float through the VM_Call without fucking everything up. Probably would need a little deeper understanding of the VM-stuff.

But yeah, things are slowly starting to make sense.

Few more questions:
1. A very low timescale on the vanilla jomme code did not do anything beyond 0.15 or so. I have found that there is a limit for the msec going < 1 in the jk2 code. I removed that. After that, the game just got stuck on values below 0.15 or so. After I replaced everything else with the float values, it seems to work quite nicely. BUT, what I was wondering is why it is that even lower timescale values DO work on vanilla code if capturing is activated. What sorcery is this?
2. I do see the calculation of the higher framerate based on blurFrames in CL_Frame, but I can't see any apparent implementation of the DoF mechanism. Where is it hidden and how does it achieve what it does (changing camera position and basically re-rendering without advancing time)?
User avatar
fau
Staff
Posts: 433
Joined: 16 Aug 2015, 01:01
Location: Warsaw / Poland
Contact:

Re: JoMME modifications

Post by fau »

1. A very low timescale on the vanilla jomme code did not do anything beyond 0.15 or so. I have found that there is a limit for the msec going < 1 in the jk2 code. I removed that. After that, the game just got stuck on values below 0.15 or so. After I replaced everything else with the float values, it seems to work quite nicely. BUT, what I was wondering is why it is that even lower timescale values DO work on vanilla code if capturing is activated. What sorcery is this?
jomme already has a hack for this I mentioned earlier, search for cg.timeFraction. It's used only in few places where it really matters. Converting whole code to using floating point time would be a huge work and unnecessary. Also you'd need to use double not float. Server time can go up to 2^31 milliseconds (24 days) and a float can hold only 23 bits of precision.
2. I do see the calculation of the higher framerate based on blurFrames in CL_Frame, but I can't see any apparent implementation of the DoF mechanism. Where is it hidden and how does it achieve what it does (changing camera position and basically re-rendering without advancing time)?
I don't know how DoF is implemented in jomme, there are few more simple and fast methods other than what you described. Follow cvars controlling DoF to find out.
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »



Alrighty, I played around a little with extremely dirty code, which basically looks a little like this (in GetShot):

static int progress = 9;
Com_Printf("Current progress: %d\n",progress);
void* displacedOutput = (byte*)output + progress* 3 * glConfig.vidWidth*glConfig.vidHeight / 10;
qglReadPixels(0, glConfig.vidHeight / 10 *progress, glConfig.vidWidth, glConfig.vidHeight / 10, GL_RGB, GL_UNSIGNED_BYTE, displacedOutput);
progress--;
if (progress == -1) progress = 9;

The result you see above. I guess I am happy I didn't get a segfault or something like that (right?), because JoMME seems to be reusing the frame buffer that is passed to the 'output' parameter. (EDIT: Of course I didn't ...) Anyhow, I tried this with adjusting the fps in CL_Frame:

fps = 10* cl_avidemo->integer * com_timescale->value * (float)blurFrames;

But that doesn't seem to do anything at all. You don't need to give an answer to that btw, I am sure I will figure it out when I look more into the code.

Anyhow, thought you might be pleased to see the shitty 'result'.
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »



Alright, first proof-of-concept. The sound is messed up, because I did it by simply excluding 9 out of 10 frames from SaveShot (including the audio).

Surprisingly, the effect is very mild. I suppose I need a faster scene to tickle out a little more. I needed to go as low with the fps as 10, to make it reasonably recognizable. It's also possible that the fact that I only divide by 10 blocks negatively impacts the effect, as the curvature does not become really nicely visible. Well, anyhow, I suppose this is a nice first draft of an 'Lowlight Iphone' look. You can already recognize a little of that 'wobbling' effect that is so typical for rolling shutter.

Btw, fau, I found out why changing the fps did not do anything. You pointed me in the wrong direction.

This block:

if (cl_avidemo->integer > 0 && msec) { ... }

is actually obsolete. It never gets called, because the capture command is independent of the cl_avidemo. In reality, it all happens within the DRAW_ACTIVE_FRAME thingy and, more concretely, in cg_demos.c.

Anyhow, I tried the whole thing with 100 row blocks, but it messed up somehow. Maybe I missed one spot to change it to 100 or something. Or 1080 somehow is not perfectly divisible by 100 (EDIT: note to myself: derp!). I got quite the pixelsalad trying it.

Does anybody here have a rather fast-paced demo with lots of quick mouse-movements for further tests? Does not have to feature any notable skill, just high speed.
Tom Arrow
Posts: 92
Joined: 21 Sep 2016, 03:05

Re: JoMME modifications

Post by Tom Arrow »

Ah, and I didn't find a way to make it render just a part-frame yet. It seems that as it is currently implemented, openGL always renders the entire frame to the window and then the qgl commands simply read out what has already been rendered. So there's really no speed boost. Doing the 100-row-blocks test was quite slow. But maybe there's some way around it. If not, I suppose it will simply be slow, heh. But then we can forget about the pixel-based approach definitely.
Post Reply
Created by Matti from StylesFactory.pl and Warlords of Draenor (modified by jk2.info)
Powered by phpBB® Forum Software © phpBB Limited