Need Help More direct-sail encounters

[Pieter Boelen]

Ahoy @Don Lasagnetti and welcome back in these waters!
And no worries on the real life thing. It happens to us all.

Also, THANK YOU for this update!
I hope @Grey Roger can include it in his next update.

Also, I've noticed that I kept running into the same names on pirates ships rather often and searched for the file responsible, is it possible that not all names in ships_name are actually used? I have the impression some names in there never appear at all... What's the experience of others who play a lot??

I've always been confused by the syntax of that ship names file.
Ordinary functions want a fixed number of input arguments; otherwise they cause errors.
The ship names file uses functions that seem to have a different number of names (apparently coded as arguments) per line.
It could very well be that this doesn't actually work properly.

@Grey Roger, could you confirm from experience by any chance?

 

[ChezJfrey]

Ahoy @Don Lasagnetti and welcome back in these waters!
And no worries on the real life thing. It happens to us all.

Also, THANK YOU for this update!
I hope @Grey Roger can include it in his next update.


I've always been confused by the syntax of that ship names file.
Ordinary functions want a fixed number of input arguments; otherwise they cause errors.
The ship names file uses functions that seem to have a different number of names (apparently coded as arguments) per line.
It could very well be that this doesn't actually work properly.

@Grey Roger, could you confirm from experience by any chance?

Those aren't arguments, it is a single string argument, with comma separators between names.

The reason some names show up more frequently is because of a flaw in the engine. To generate "random" numbers, an algorithm is used in rand that first needs a seed number to start with; with a given seed number, the "random" numbers generated will actually always be the same. The purpose of rand is the seed with a generally unique number and the rand function calls after that will replicate what looks to be random.

For instance, if initially seeded with a 1000, and three consecutive rand(50) calls generate 15, 21, 35, then another game session is started where the seed given is also a 1000, then three calls to rand(50) will generate the same sequence: 15, 21, 35. Normally, an application will seed with something that is unlikely to occur similarly, like app launch time from the OS. This is what the game does, but there is a problem.

So assume the game starts at time 1000 (just an arbitrary example) and seeds the random generator with that 1000. Now assume that three consecutive calls to rand(50) generates that 15,21, 35. The flaw is that within the same game session, there are multiple functions in the engine, that if called, reseed the generator again with that exact same number. As soon as those functions are called, the next three calls to rand(50) will generate the exact same sequence of 15, 21, 35...over and over, every time those functions that reseed the generator, the "random" function starts the exact same sequence because it's built upon a mathematical algorithm that for a given "starting point" will always obtain the same answers, in order.

What is supposed to happen, is that the random generator is seeded only ONE TIME, at the start of the application, with a generally unique number, then all subsequent calls to rand() will actually look random from then on.

I fixed this problem just as described, by removing all the superfluous calls to the seeding, keeping only one of them, and also went a step further by seeding with a GUID which is guaranteed to be unique per machine running the game.

 

[Pieter Boelen]

Those aren't arguments, it is a single string argument, with comma separators between names.

Oh crap, YOU'RE ABSOLUTELY RIGHT!
I must've had it wrong in my memory for years then.
Thanks very much for catching that!!

So assume the game starts at time 1000 (just an arbitrary example) and seeds the random generator with that 1000. Now assume that three consecutive calls to rand(50) generates that 15,21, 35. The flaw is that within the same game session, there are multiple functions in the engine, that if called, reseed the generator again with that exact same number.

WOW! I always find (non-)randomness really quite interesting.
But that original approach you described sounds very flawed indeed.
Sure explains a lot, though!

Thanks AGAIN.

seeding with a GUID which is guaranteed to be unique per machine running the game

I had never heard of that term, but you inspired me to look it up: Universally unique identifier - Wikipedia
I've always been uncomfortable with the "random string generation", but that actually makes a lot of sense.
Now I wonder if the software developers at work use that too. If not, I think they should.

So that's a third thanks from me.

 

[The Nameless Pirate]

Interesting @ChezJfrey, quite interesting!

Good job, on looking at this and improving it.

 

[Grey Roger]

The revised "sea.c" appears to give an equal chance of 1, 2 or 3 fleets showing up, which I'd reckon is rather excessive. 2 fleets should be signficantly rarer than 1, and 3 fleets in one encounter should be almost non-existent - why would three different groups all converge on the same spot? Also, additional CTD's are not a good thing.

There should be no problem with the extra names for pirate ships, though.

 

[Pieter Boelen]

The revised "sea.c" appears to give an equal chance of 1, 2 or 3 fleets showing up, which I'd reckon is rather excessive. 2 fleets should be signficantly rarer than 1, and 3 fleets in one encounter should be almost non-existent - why would three different groups all converge on the same spot?

Maybe you can tweak that? Sounds tweak-able, at least...

Also, additional CTD's are not a good thing.

Is that really the case, though? Also on more modern hardware?
Some testing required, right?

Also, how about including this feature with a toggle?

 

[Grey Roger]

Maybe you can tweak that? Sounds tweak-able, at least...

Perhaps instead of a 'switch(rand( 2 ) )', this:

       if (bDirectSail && CheckAttribute(rPlayer, "directsail.encounter"))
       { // if set so in DirectsailCheck() random encounter ships are created
           rPlayer.ship.pos.x = stf(Login.PlayerGroup.x); // so that DirectEncounter doesn't use OLD coords
           rPlayer.ship.pos.z = stf(Login.PlayerGroup.z);
           DirectEncounter(stf(Login.PlayerGroup.ay));
           if (DS_MULTIFLEET > 0.0)
           {
               if (frand(100) <= DS_MULTIFLEET)
               {
                   DirectEncounter(stf(Login.PlayerGroup.ay));   // Second group in encounter, chance of a third
                   if (frand(100) <= DS_MULTIFLEET) DirectEncounter(stf(Login.PlayerGroup.ay));
               }
           }
           DeleteAttribute(rPlayer,"directsail");   // clears tags from player
       }

'DS_MULTIFLEET' is set in "InternalSettings.h", default 0.0. It should be a percentage chance of a second encounter. Repeating the line in the above code means there's a percentage^2 chance of getting two additional encounters, i.e. a triple encounter. For example, if you set 'DS_MULTIFLEET' to 50.0 then there should be a 0.5 chance of an extra group and a 0.25 chance of two extra groups.

Is that really the case, though? Also on more modern hardware?
Some testing required, right?

Testing has already shown CTD's.

We've already lost players with older hardware because of the levelling "improvement". We probably don't want to lose more.

 

[Pieter Boelen]

'DS_MULTIFLEET' is set in "InternalSettings.h", default 0.0. It should be a percentage chance of a second encounter. Repeating the line in the above code means there's a percentage^2 chance of getting two additional encounters, i.e. a triple encounter. For example, if you set 'DS_MULTIFLEET' to 50.0 then there should be a 0.5 chance of an extra group and a 0.25 chance of two extra groups.

That looks perfect to me!

Testing has already shown CTD's.

Agh; that's unfortunate.
But at least now we know.

I wonder if there's a way with a PostEvent to trigger the second/third fleet a second later or so.
My suspicion is that the CTD happens because the game engine gets overloaded with having to do too much of the same thing at the same time.
If it can be delayed a bit, perhaps that is enough to make it work?

We've already lost players with older hardware because of the levelling "improvement". We probably don't want to lose more.

Cannot argue with that level of logic!

 

[The Nameless Pirate]

We've already lost players with older hardware because of the levelling "improvement". We probably don't want to lose more.

Isn't there a toggle for it?
I remember seeing it for sure.

 

[Pieter Boelen]

Isn't there a toggle for it?
I remember seeing it for sure.

There are all sorts of settings for it; but no actual toggle for what @Grey Roger refers to.
You might be thinking of the Auto Skill System setting.

 

[The Nameless Pirate]

You might be thinking of the Auto Skill System setting.

Ah yes, i must have mixed the two in my memory, thanks @Pieter Boelen!!!

 

[Grey Roger]

Agh; that's unfortunate.
But at least now we know.

We already knew. I haven't had time to test it myself. The testing to which I referred was this:

Finally though I've found some time to finalize this, and for me the attached sea.c variant is working properly, apart from occasional crashes that seem to be caused by too many ships for my workstation to handle, as far as I can tell from error and compile logs at least (~5% crashrate for me).

I'm afraid I don't agree that a 5% crash rate is "working properly".

I wonder if there's a way with a PostEvent to trigger the second/third fleet a second later or so.
My suspicion is that the CTD happens because the game engine gets overloaded with having to do too much of the same thing at the same time.
If it can be delayed a bit, perhaps that is enough to make it work?

Won't the engine still have to do too much of the same thing at the same time once all the fleets have been created and placed? It still has to move them around and display them.

In any case, having the toggle is a good idea because not everyone might want extra encounters, and having it as a float rather than a boolean means individual players who do want multi-way encounters can set the frequency to suit their play style (and possibly their hardware).

 

[ChezJfrey]

We already knew. I haven't had time to test it myself. The testing to which I referred was this:

I'm afraid I don't agree that a 5% crash rate is "working properly".


Won't the engine still have to do too much of the same thing at the same time once all the fleets have been created and placed? It still has to move them around and display them.

In any case, having the toggle is a good idea because not everyone might want extra encounters, and having it as a float rather than a boolean means individual players who do want multi-way encounters can set the frequency to suit their play style (and possibly their hardware).

I am baffled by this insistence that somehow, it's the load of simultaneous processing that is causing the crashes?

I highly doubt load is the issue; given proper coding, heavy loads just degrade speed/performance, not crashes. Rather, the problems I saw in places like AIBalls, AIGroup, ShipTracks, Particles, Lightning and Rain, where they destroy certain objects, but then in some cases try to dereference those destroyed objects is the real culprit. For an example of Rain, check out the TEHO forums and how many "rain crash" complaints there are.

ETA: I suppose I could add that there are also instances I've found where they force a cast for some objects to a type that isn't even derived from the same base class, then try to call a virtual method that doesn't exist for the type it was originally, so it crashes; that is one of the most recent causes I found for a crash during dismasting. The fact that it hasn't happened until just this last week, just illustrates sometimes how long you can get "lucky" until something in the logic finally hits a spot you hadn't arrived before, and now you aren't so lucky.

The only reason it doesn't crash more often, is pure luck that whatever it is the player is doing, doesn't cause the move into the logic that dereferences those destroyed elements. The more likely reason for more crashes in this development is that as the number of ship (or other) objects that are introduced, increases to a certain level, that luck begins to run out.

 

[Pieter Boelen]

We already knew. I haven't had time to test it myself. The testing to which I referred was this:
I'm afraid I don't agree that a 5% crash rate is "working properly".

Ah! Oh...

I had actually been curious about feedback from a more modern computer.
That could help to confirm or deny my hypothesis.

Won't the engine still have to do too much of the same thing at the same time once all the fleets have been created and placed? It still has to move them around and display them.

After the creation, yes.
But it if works for WorldMap fleet encounters, the engine should be able to handle it for DirectSail ones too.

In any case, having the toggle is a good idea because not everyone might want extra encounters, and having it as a float rather than a boolean means individual players who do want multi-way encounters can set the frequency to suit their play style (and possibly their hardware).

Definitely agreed!

I do wish one day it can be enabled by default.
It's quite the missing feature that you CAN encounter fleets in battle from the WorldMap and not in DirectSail.

I am baffled by this insistence that somehow, it's the load of simultaneous processing that is causing the crashes?

With the old version of the engine, we've definitely had that sometimes.

And you're right, with proper coding it shouldn't be necessary.
But you know better than anyone, the original game engine was not coded quite that properly...

 

[The Nameless Pirate]

I highly doubt load is the issue; given proper coding, heavy loads just degrade speed/performance, not crashes. Rather, the problems I saw in places like AIBalls, AIGroup, ShipTracks, Particles, Lightning and Rain, where they destroy certain objects, but then in some cases try to dereference those destroyed objects is the real culprit. For an example of Rain, check out the TEHO forums and how many "rain crash" complaints there are.

Ah yes the infamous rain crashes, that happened for me in the city of abandoned ships quite frequently on both COAS and TEHO at that location, but at the time i was playing COAS and TEHO the laptop i mostly used, need cleaning and most importantly a new hard drive, when i played TEHO 0n a different laptop that had better performance because it was cleaner and had a correctly working hard drive the crashes almost stopped, but rarely they would still happen.
Also after i fixed the problematic laptop, crash rate and performance was about the same as the normal one just as it should be, given the similar specs of the two.

So, the code helps but the also computer helps.

 

[Grey Roger]

I had the chance to try out the mod for extra encounters. After setting DS_MULTIFLEET to 70.0, I tried direct-sailing from Barbados to Aruba. The first encounter was two groups, but only one announcement so the second group was probably coastal traffic. The second encounter was three groups, all announced; two of them were British navy, which was bad news for group 3, a single pirate ship. I never saw what was in the third encounter because that crashed the game.

It is unlikely to be a problem with objects being created, destroyed and then dereferenced because the crash happened right away. If this were the cause, you'd expect it to happen part way through a battle.

"compile.log" indicates that there were three groups, none of them particularly large: Patrol (one Spanish brig), Pirate (sloop and brig), and Portuguese Coastal Waters Merchant Fleet (Raa frigate, Fragata Latina, and five assorted merchant ships). As the game crashed before I could see any announcements, it's possible that one of these groups was coastal traffic rather than direct-sail encounter.

"system.log" contains a message I don't recall seeing before:

Assert failed in C:\PROJECTS\DRIVE_V2\SEA_AI\AIHelper.cpp line 102, expression string x < dwRelationSize && y < dwRelationSize

There was no "error.log".

 

[The Nameless Pirate]

"system.log" contains a message I don't recall seeing before:

Assert failed in C:\PROJECTS\DRIVE_V2\SEA_AI\AIHelper.cpp line 102, expression string x < dwRelationSize && y < dwRelationSize

There was no "error.log".

I remember seeing C:\\PROJECTS\\DRIVE_V2\\ in the source code or rather the part of it, that i manged to decompile from the engine so must be an engine limitation most probably.

Have a look at the file engine 2.txt from this post:Decompiling ENGINE.exe?

You won't find AIHelper.cpp because only a part of the source code is present there but it may confirm that it has to do with the engine unfortunately.

 

[Pieter Boelen]

C:\\PROJECTS\\DRIVE_V2\\

I assume that's the folder where the developers kept the files on their own computer.

Who here know how to use PostEvent with parameters?
If nobody, I'll look it up and explain.
Might be worth a shot?

 

[ChezJfrey]

It is unlikely to be a problem with objects being created, destroyed and then dereferenced because the crash happened right away. If this were the cause, you'd expect it to happen part way through a battle.

"system.log" contains a message I don't recall seeing before:

Assert failed in C:\PROJECTS\DRIVE_V2\SEA_AI\AIHelper.cpp line 102, expression string x < dwRelationSize && y < dwRelationSize

This is fun...

We get some suppositions, based on anecdotal "evidence" that some drive cleaning "solves" crash problems, which in reality contribute nothing toward actual crashes. Suppositions added that it can't be what I state, because of some sort of expectation, based on nothing but speculation...then an actual logged error that shows in this case, evidence that fully backs what I already told everyone...engine logic problems. I did not list them all. In this case, you are at least explicitly told (this is not always the case as another problem is the engine coding does not anticipate or properly handle certain possible error conditions, so no log is thrown as it just fails CTD) that an assert failed due to an attempt to read/write an array outside the bounds of that array, which is what really caused the crash and is yet another frequent cause of the game crashing due to faulty or inadequate engine coding. For the record, this particular issue was already discovered and a fix implemented quite awhile ago.

I suppose that sounds a bit snarky, but I'm just trying to convince you all of objective reality; the original engine is flawed and for the most part, you will be unable to circumvent that. Some of you have enough experience modding/playing this game that you know this to be true and these "workarounds" like drive cleaning and such, are just wishful thinking, typically based on anecdotal coincidence, because they do not really solve the various, underlying, root problems and your drive fragmentation (or whatever) was never really the cause and had no bearing on the true issue.

For some strange reason, my old, outdated, underpowered, Win7, circa 2011, mediocre laptop, with a basic OEM video card, can somehow handle this without crashing...I assure you it has nothing to do with drive fragmentation as I never defrag (the only thing defrag might accomplish is slightly boost file read/write speed):

 

[The Nameless Pirate]

For some strange reason, my old, outdated, underpowered, Win7, circa 2011, mediocre laptop, with a basic OEM video card, can somehow handle this without crashing...I assure you it has nothing to do with drive fragmentation as I never defrag (the only thing defrag might accomplish is slightly boost file read/write speed):

I didn't mean defrag what meant is not a software problem, but hardware problem, that hard drive had to be swapped out with a new because it was going to be completely broken and not working at any point, it was nearing the end of it's life.

We get some suppositions, based on anecdotal "evidence" that some drive cleaning "solves" crash problems

Also by cleaning i didn't mean drive cleaning but opening the computer getting all the dust off and applying new thermal paste.
Both of which do make quite an impact on performance by not overheating the processor can can work an full clock, when your PC overheats the processor runs at a lower frequency in order to protect itself.
And about the hard drive, every time a sound was loading for example, the game lagged until the file was loaded on the ram, every time.

Also i didn't suggest it as a workaround but as an observation that the performance of the computer may help.

Lastly i didn't say this is not an engine problem actually i also said that it most likely is an engine problem, but that doesn't mean that a better computer doesn't help, because if you load more things performance gets worse, still as i see it that crash was caused by an engine limitation.

Some of you have enough experience modding/playing this game that you know this to be true

I also have run into them, to the biggest extent when i had to use a different model for Geralt after losing time trying to cope with the character file size limitation and in the end having to just re texture an existing PotC model.

So, we agree @ChezJfrey!!!