• New Horizons on Maelstrom
    Maelstrom New Horizons


    Visit our website www.piratehorizons.com to quickly find download links for the newest versions of our New Horizons mods Beyond New Horizons and Maelstrom New Horizons!

Feature Request Improve Performance on Store Goods Interface

Pieter Boelen

Pieter Boelen

Navigation Officer
Administrator
Storm Modder
Hearts of Oak Donator
I just noticed that GetStoreGoodsQuantity gets called all the time in the Store Goods Interface too.
That probably isn't helping performance there either.
 
Will take a look at it later. If the function isn't heavy its no problem, but I guess it isn't.
 
Well, *some* stuff there is heavy for sure because performance in that interface isn't really stellar....
 
Marked this as bug ;).
 
Just curious when is the worst lag in this interface?
Is it during scrolling trough the goods?
 
Levis said:
Just curious when is the worst lag in this interface?
Is it during scrolling trough the goods?
Click to expand...
The interface takes a while to load, if I recall. And the scrolling can be a bit slow at times as well.
But I don't think anybody ever had any major complaints about it, so I wouldn't consider this a huge priority.
 
If there is lag in the store interface, and I recall that it has tremendous lag, there is something very wrong with the code or scripts, because there is no performance issues with a simple scroller interface component. If it has, then something behind the scene is doing something amazingly wrong. It is either repeating work that absolutely does not need to be repeated, or it is waiting for an event that doesn't need wait (Shouldn't need wait). These kind of problems often requires to break the interface apart and start over from scratch, they can be messy to debug but shouldn't be messy to debug, because they are simple features of the game.

The lag accumulates over time, the store is almost unusable in mid to late game, but early on it is fast, so there is something in there that adds up. Try to optimize the code or scripts, quite often, events can be executed many times for very small event types, try to rearrange the code/scripts so that they run less often.

A computer can generate up to 3 billion random numbers in a single second, and that store interface is often hanging for 7-10 seconds at a time, which is equal to an insane amount of processing power, something is being done very wrong with that. I often want to define it this way: If something hangs on modern computers, your code is ready to be trashed and if I were you, I would break down the interface and rebuild it from scratch, this time, the right way.

The last couple of years I've written many demanding programs and I've never had it hang once, because our computers are so blazingly fast that they shouldn't do and they don't do, if they do, there is something absolutely wrong with the code or script.

Just remember that your code or script, if it runs regularly, it must not spend more than 16+ milliseconds, if it goes above that and it repeats frequently, you lock up everything. Check the frequency at which your scripts run and see if there are other event types that can be used.

If, after clicking an object in the store, try to spread out the calculation, and not run it exactly when the user clicks it, try to calculate something when the user scrolls, and then calculate the other half when he clicks (precalculation) and make sure you don't allocate any unneccesary graphics objects in the store that you're unaware of, if they get allocated but not deallocated can be a problem.

What I want to say most of all, all user code/scripts need a total revival, a total rewrite and total rearrange. A full revisal and optimization of all scripts and all code. Most important of all, I cant stress it enough, is where you place and how you arrange your code. Code organization is more important than code optimization. Badly placed code is the mother of many mistakes.

When you write scripts or code, you should have one goal, the next day you reduce that code so it gets smaller and more effective, don't let it add up. Always reduce.

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. Always look for crap code, remove it, then look more, find crap code, remove it, see if code can be better arranged and put somewhere else, replace it.

The coder that fucks up a game, is the guy who makes new code, then forgets about that code and moves on the create another piece of code somewhere else. After you've made code, you should always try to go over it to see what can be taken away, what can ease off computational needs, and can it be placed somewhere else, can it be combined with other code to do many things in parallell, etc. (Try to do more things in parallell so that events can do more than one thing at the time)

A user interface is the last thing in a game that should suffer performance issues and it shows a monstrosity beneath it that is blown up and beyond any proportions.
 
Last edited:
PotMaster said:
If there is lag in the store interface, and I recall that it has tremendous lag, there is something very wrong with the code or scripts, because there is no performance issues with a simple scroller interface component. If it has, then something behind the scene is doing something amazingly wrong. It is either repeating work that absolutely does not need to be repeated, or it is waiting for an event that doesn't need wait (Shouldn't need wait).
Click to expand...
Apparently some functions get called on each frame update, which is indeed quite unnecessary. Refer to the opening post for some more details.
 
If functions are being run during each frame update you absolutely need to take it away from there because frame synchronization leaves you with only 16 millisecond run time, that needs the code to be rearranged and put somewhere else. Better yet, run your code in a separate thread. If you know how to inject dll's into the game, you can dedicate an entire thread for your scripts.

Find out what is being run on each frame update and replace those, you can do it. You can find other solutions if you try hard. It doesn't matter if they get run in between the frames, the user won't notice it.

You can solve the whole problem very easily, but it requires that you turn off frame sync, that will solve the whole problem but will produce screen tearing. It is just a matter of changing a constant from 0 to $80000000 and the whole problem vanishes, but it's not a good solution.
 
Last edited:
Pieter, a quick and dirty fix to the problem is, whatever script or code that gets called in every frame, set up a counter in your script, every time it gets run between the frames, increase the counter by 1, if the counter is not 60 or above, quit/exit the code or script. If the counter is 60 or above, run the script and reset the counter to zero again. This will yield processing time to the renderer and only run your code once per second (the code related to the store interface)

You could advance this and run one part of your code when the counter reaches 30, and execute the other half when it reaches 60 (to divide the work up) I'd recommend to divide it up, it will make the game flow better.

Like this:

SCRIPT:

Counter := Counter + 1
if Counter >= 30
Do some work here, store the results
Quit
else if Counter >= 60
Do the remaining work here, store the results
Counter := 0
Quit
end if

If you can divide it further up, is always better.
 
Last edited:
True, that could be done. For the interface there is probably a better solution, but I have been trying to think of a way to do that for the sea ai.
Depends on the area where the most can be gained by doing it.
Maybe one day. Unfortunately most changes can be quite time consuming to implement.
 
Potmaster, since you appear to be an expert on coding, perhaps you could better spend your time going back through 12 years of code rewrites and weeding out all the "crap code" entered by literally hundreds of people in the POTC script files instead of posting a wall of text telling us how we are doing things wrong and how we should do things right. After all, obviously your the expert and have plenty of free time to point out all our shortcomings!

You have been warned once already formally about the tone of your posts, consider this another formal warning. If you honestly wish to help with any of our projects, you are more than welcome to post constructive criticism, or better yet, jump in and help us. If the condescending tone or your posts continues, your next warning will come with a temp ban.
 
PotMaster said:
If there is lag in the store interface, and I recall that it has tremendous lag, there is something very wrong with the code or scripts, because there is no performance issues with a simple scroller interface component. If it has, then something behind the scene is doing something amazingly wrong. It is either repeating work that absolutely does not need to be repeated, or it is waiting for an event that doesn't need wait (Shouldn't need wait). These kind of problems often requires to break the interface apart and start over from scratch, they can be messy to debug but shouldn't be messy to debug, because they are simple features of the game.

The lag accumulates over time, the store is almost unusable in mid to late game, but early on it is fast, so there is something in there that adds up. Try to optimize the code or scripts, quite often, events can be executed many times for very small event types, try to rearrange the code/scripts so that they run less often.
Click to expand...
That's why I ask when the lag occurs so I can rearange the code which is responsible for it. Besides that we are talking about a scroller yes. But it has to poll for data each time. The game is written in a way the store doesn't has the information stored easily, it has to do some calculations before it shows it. This is done to preserve RAM usage. Remember we are talkig about a very old game which was designed for computers which less RAM so in a lot of cases the designers had to think about CPU usage vs RAM usage. In the interface cases etc they opted for more CPU usage so they could use the RAM for the more taxing parts of the game.

A computer can generate up to 3 billion random numbers in a single second, and that store interface is often hanging for 7-10 seconds at a time, which is equal to an insane amount of processing power, something is being done very wrong with that. I often want to define it this way: If something hangs on modern computers, your code is ready to be trashed and if I were you, I would break down the interface and rebuild it from scratch, this time, the right way.

The last couple of years I've written many demanding programs and I've never had it hang once, because our computers are so blazingly fast that they shouldn't do and they don't do, if they do, there is something absolutely wrong with the code or script.

Just remember that your code or script, if it runs regularly, it must not spend more than 16+ milliseconds, if it goes above that and it repeats frequently, you lock up everything. Check the frequency at which your scripts run and see if there are other event types that can be used.
Click to expand...
All fair but remember we are talking about engine code here. Not normal programming. From my experience POTC starts to hang already when it has to do more then 10000 calculations instead of the few million. This is because the engine is busy doing stuff also and determines how much CPU usage can be used to execute the scripts. We can't change this unless we are going to do a lot of reverse engenering. And that's not worth it. This is just a hobby project for a lot of people, if we find walls we can't pass because of the engine we write those things down and pitch them for HoO.

If, after clicking an object in the store, try to spread out the calculation, and not run it exactly when the user clicks it, try to calculate something when the user scrolls, and then calculate the other half when he clicks (precalculation) and make sure you don't allocate any unneccesary graphics objects in the store that you're unaware of, if they get allocated but not deallocated can be a problem.
Click to expand...
Loading textures to the engine takes a long time because they are saved compressed and the engine isn't that good at decompressing. So the programmers opted to load all textures and stuff during the loading screen so this won't be a issue. Besides that the code is already spread out. The problem mostly is that people added stuff to these code pieces and in some cases they added functions which take a long time to process while the information they give is already avaible. These are the pieces of code I need to find and change.

What I want to say most of all, all user code/scripts need a total revival, a total rewrite and total rearrange. A full revisal and optimization of all scripts and all code. Most important of all, I cant stress it enough, is where you place and how you arrange your code. Code organization is more important than code optimization. Badly placed code is the mother of many mistakes.

When you write scripts or code, you should have one goal, the next day you reduce that code so it gets smaller and more effective, don't let it add up. Always reduce.

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. Always look for crap code, remove it, then look more, find crap code, remove it, see if code can be better arranged and put somewhere else, replace it.

The coder that fucks up a game, is the guy who makes new code, then forgets about that code and moves on the create another piece of code somewhere else. After you've made code, you should always try to go over it to see what can be taken away, what can ease off computational needs, and can it be placed somewhere else, can it be combined with other code to do many things in parallell, etc. (Try to do more things in parallell so that events can do more than one thing at the time)

A user interface is the last thing in a game that should suffer performance issues and it shows a monstrosity beneath it that is blown up and beyond any proportions.
Click to expand...
All true BUT rewriting a interface script from scrap takes me about a day. Let people test it in all the edge cases (think about: during a fetch quest, after the town is captured, when the economy is low etc) will take a nother few days. Fixing the bugs still in there will take another few days. All in all it would take at least a week to pull it off. And that is if I have time to work on potc each day. This is often not the case. I and many other people believe there are more pressing things in the game we want to work on. Besides that I'm doing this for fun. And scripting an interface is not that fun. Making a new system of revising a complex system is more fun, so I prefer to do this.
Going trough the code of the interface and looking for cases where I can improve the code a bit does only take me a few hours and I could easily see if it causes any bugs. So I opt to use this methode instead.

PotMaster said:
Pieter, a quick and dirty fix to the problem is, whatever script or code that gets called in every frame, set up a counter in your script, every time it gets run between the frames, increase the counter by 1, if the counter is not 60 or above, quit/exit the code or script. If the counter is 60 or above, run the script and reset the counter to zero again. This will yield processing time to the renderer and only run your code once per second (the code related to the store interface)

You could advance this and run one part of your code when the counter reaches 30, and execute the other half when it reaches 60 (to divide the work up) I'd recommend to divide it up, it will make the game flow better.

Like this:

SCRIPT:

Counter := Counter + 1
if Counter >= 30
Do some work here, store the results
Quit
else if Counter >= 60
Do the remaining work here, store the results
Counter := 0
Quit
end if

If you can divide it further up, is always better.
Click to expand...
The store interface doesn't have anything on frame update. Some of the other scripts do. but the only thing they check on frame update is input changes. When other things are put in frame update I tend to remove them anyways.
 
Thagarr said:
"crap code"
Click to expand...

It would be better if you didn't read posts of programmers, because "crap code" is a well defined term in the programming world, it means take away excessive code, it is absolutely not a condensating tone, I urge you to not read posts made by programmers. Do you see your mistake here? You are setting up very nice traps for me. Just don't do that, don't set up traps for me, and also, make some breathing space for mistakes. You know, mistakes, tolerance and that stuff. I don't like you personally, I don't want to have anything to do with you, but you should allow people to have some technical breathing space without you twisting the terms that are not understandable for everyone, those technical terms have a place in the programming world.

The best thing you can do is to not read my posts, thats a serious suggestion. "crap code" is a well understandable term, it has absolutely nothing to do with either sarcasm, condensating tone or anything in that regard, in the coding world, it is actually a positive term in talks about code optimization and improvements.
 
Last edited:
PotMaster said:
It would be better if you didn't read posts of programmers, because "crap code" is a well defined term in the programming world, it means take away excessive code, it is absolutely not a condensating tone, I urge you to not read posts made by programmers. Do you see your mistake here? You are setting up very nice traps for me. Just don't do that, don't set up traps for me, and also, make some breathing space for mistakes. You know, mistakes, tolerance and that stuff. I don't like you personally, I don't want to have anything to do with you, but you should allow people to have some technical breathing space without you twisting the terms that are not understandable for everyone, those technical terms have a place in the programming world.

The best thing you can do is to not read my posts, thats a serious suggestion. "crap code" is a well understandable term, it has absolutely nothing to do with either sarcasm, condensating tone or anything in that regard, in the coding world, it is actually a positive term.
Click to expand...

And now stop the discussion please guys :).

Postmaster everyone is free to help. If you can optimize this interface be my guest, I still have a lot of other stuff to do.
 
PotMaster said:
The best thing you can do is to not read my posts, thats a serious suggestion.
Click to expand...
Then might I suggest that the best thing you can do is to look into the code yourself and help us out by improving it like you say.
There is plenty of stuff that needs doing and just talking about things doesn't get us that much further.
 
Levis said:
That's why I ask when the lag occurs so I can rearange the code which is responsible for it. Besides that we are talking about a scroller yes. But it has to poll for data each time. The game is written in a way the store doesn't has the information stored easily, it has to do some calculations before it shows it. This is done to preserve RAM usage. Remember we are talkig about a very old game which was designed for computers which less RAM so in a lot of cases the designers had to think about CPU usage vs RAM usage. In the interface cases etc they opted for more CPU usage so they could use the RAM for the more taxing parts of the game.


All fair but remember we are talking about engine code here. Not normal programming. From my experience POTC starts to hang already when it has to do more then 10000 calculations instead of the few million. This is because the engine is busy doing stuff also and determines how much CPU usage can be used to execute the scripts. We can't change this unless we are going to do a lot of reverse engenering. And that's not worth it. This is just a hobby project for a lot of people, if we find walls we can't pass because of the engine we write those things down and pitch them for HoO.


Loading textures to the engine takes a long time because they are saved compressed and the engine isn't that good at decompressing. So the programmers opted to load all textures and stuff during the loading screen so this won't be a issue. Besides that the code is already spread out. The problem mostly is that people added stuff to these code pieces and in some cases they added functions which take a long time to process while the information they give is already avaible. These are the pieces of code I need to find and change.


All true BUT rewriting a interface script from scrap takes me about a day. Let people test it in all the edge cases (think about: during a fetch quest, after the town is captured, when the economy is low etc) will take a nother few days. Fixing the bugs still in there will take another few days. All in all it would take at least a week to pull it off. And that is if I have time to work on potc each day. This is often not the case. I and many other people believe there are more pressing things in the game we want to work on. Besides that I'm doing this for fun. And scripting an interface is not that fun. Making a new system of revising a complex system is more fun, so I prefer to do this.
Going trough the code of the interface and looking for cases where I can improve the code a bit does only take me a few hours and I could easily see if it causes any bugs. So I opt to use this methode instead.


The store interface doesn't have anything on frame update. Some of the other scripts do. but the only thing they check on frame update is input changes. When other things are put in frame update I tend to remove them anyways.
Click to expand...

I have never looked at any of the code that ships with this mod, I didn't know it had so much code, of course that makes it a different scenario and harder to rewrite it. It's difficult to understand why compressed textures are slower to reload/decompress unless the decompressing algorithm is extremely complex or slow, it should actually be faster, especially with rle encoding (Which is used in many algorithms), if you consider the speed of the harddrive vs the speed of the processor, loading a reduced texture cuts down time spent on the drive and decompressing adds a tad on the cpu, personally, my own compression algorithms reduces the time it takes to load it and my compression algorithm some times compresses better than PNG.
 
Levis said:
And now stop the discussion please guys :).

Postmaster everyone is free to help. If you can optimize this interface be my guest, I still have a lot of other stuff to do.
Click to expand...

I were really only trying to help them locate the problem, but according to the OP there isn't really a problem. So I made a few tips when there weren't really a need for it. Its just that they are focused on other tasks, so it isn't a priority.
 
Potmaster, I certainly am no programmer and do not claim to be. However, I am an Admin on these forums and I will continue to read everyones posts, including yours. My warning was not for the term "crap code," as I stated, it was for the condescending tone of your recent posts. Since that has now turned in to a personal attack on me, I have issued another formal warning and you can have some time away from the forum to consider your posting behavior.
 
You must log in or register to reply here.
Back
Top