New Horizons on JLB Maelstrom 2.8.2 engine

[pedrwyth]

Just a thought: We intend to get another version ready for ModDB soon.
Do you think it would make sense to get that version running with 2.8.2 instead of the one currently on ModDB?

No for many reasons.

First there needs to be a stable base on the old engine to compare with when moving NH to 2.8.2. It would be good if this were a completed build 14 (or if that doesn't happen this time) at least a formalised point in coding development.

Second the people active on modding NH are generally there because they want to add/change stuff. In the time I have been active this has always caused difficulties in keeping aspirational material away from breaking/bugging the current WIP. This has been eased a bit over the past year by @Grey Roger 's robust defence of a stable base. However I don't see the active community putting their personal efforts aside to help migrate and test the code, without that effort there can be no guarantee of when the migration would be "complete" and tested. I am not looking for the new year as per new moddb aspiration (except in the sense I would hope it complete WITHIN next year).

Migrating to a new base may introduce unknown problems anywhere in the code, this is not like adding something which could (in theory) just be reverted so that bringing build 14 to a clear conclusion would be pushed into the future again. One clear area that is not currently equivalent is in particle effects. Although there is a proven approach conversion of NH particle effects is far from achieved (in fact production in XML format not even started yet).

Migrating to a new base seems much more suited to a new build version than thrown in to the end of a long process of development.

The new base would conceptually open up the whole COAS (and possibly TEHO) maps,islands,content to incorporation with NH (or vice versa) this just runs into the age old problem of too many tangents for people to run off along for a random community to manage. If it became the "official" version it has the potential to disrupt the status quo of minor tinkering and added story content that has allowed a stable mod to be maintained.

Finally there would be some difficult political issues to be addressed - what relation if any does this have to NH remastered, what about maintaining compatibility with the other Storm games? Does an engine change required elsewhere in the family of games (and that includes ERAS) get maintained in NH. What about a 64bit version and @ChezJfrey 's thoughts about moving his Maelstrom 2.8.2 on from directx 9. I suspect these aspects would be harder to address than the technical ones proved for him.

 

[Pieter Boelen]

First there needs to be a stable base on the old engine to compare with when moving NH to 2.8.2. It would be good if this were a completed build 14 (or if that doesn't happen this time) at least a formalised point in coding development.

I definitely agree with you there.

The intended January 2018 ModDB release is meant to be such a "formalised point", which is why I thought there might be an opportunity there.
There haven't been a huge amount of earth-shattering developments for the past several months, so I believe a stable release is very possible in the foreseeable future.
The main challenge is a "decision that it is ready" and then doing the actual release, which lies mostly with @Grey Roger (for the decision) and @Mad Jack Wolfe (for the release).

The new base would conceptually open up the whole COAS (and possibly TEHO) maps,islands,content to incorporation with NH (or vice versa)

Technically, that has been open for years already. Swapping out models and other resources between games has never really been an issue.

Finally there would be some difficult political issues to be addressed - what relation if any does this have to NH remastered, what about maintaining compatibility with the other Storm games? Does an engine change required elsewhere in the family of games (and that includes ERAS) get maintained in NH. What about a 64bit version and @ChezJfrey 's thoughts about moving his Maelstrom 2.8.2 on from directx 9. I suspect these aspects would be harder to address than the technical ones proved for him.

That is indeed the trickiest question by far. Simple answer: I honestly don't know.

As far as I can tell, there are many good intentions to be found here.
And I hope very, very, very much that those good intentions can lead to good things.
Nothing would make me happier.

 

[Grey Roger]

For what it's worth, I'm planning on an update release soon - it was supposed to be today, but there are a couple of other things which showed up over the weekend, so now they need to be included and tested. No further submissions will be accepted for this one, so that it can go out tomorrow.

Then I plan to release one more before I disappear for the Christmas holiday. That gives people plenty of time to test it. Barring any reports of problems, I'll probably need to compile another update early in the new year, then @Mad Jack Wolfe can make a new installer based on that.

 

[pedrwyth]

I definitely agree with you there.

The intended January 2018 ModDB release is meant to be such a "formalised point", which is why I thought there might be an opportunity there.
There haven't been a huge amount of earth-shattering developments for the past several months, so I believe a stable release is very possible in the foreseeable future.

I think I got the wrong end of your question. I certainly intend to try and marry up the next moddb version with 2.8.2. However until that is there I am sticking with the old moddb version of april 2016 rather than add in further complications in moving to a moving feast.

Technically, that has been open for years already. Swapping out models and other resources between games has never really been an issue.

Yep but I was thinking of COAS world map and new islands and mainland locations with initially potc NH quest content on the original locations but room for quest development not just swapping resources. I can't say I have looked at it or understand the issues yet (including how direct sail works in NH but is still zonal in later flavours) and maybe it is a non-starter but both being in the same engine must help I would (naively) have thought.

As far as I can tell, there are many good intentions to be found here.
And I hope very, very, very much that those good intentions can lead to good things.
Nothing would make me happier.

I'll probably need to compile another update early in the new year, then @Mad Jack Wolfe can make a new installer based on that.

I hope that goes smoothly and then I can pick up NH at that point and see how it fits onto 2.8.2 - not trivial I think was said - but since most modding these days is front end plus new resources I hope the sticking points will be mostly cleared by then by looking at the earlier moddb version.

 

[Pieter Boelen]

I think I got the wrong end of your question. I certainly intend to try and marry up the next moddb version with 2.8.2. However until that is there I am sticking with the old moddb version of april 2016 rather than add in further complications in moving to a moving feast.

Indeed maybe I was a bit vague in my wording. What I meant: Once the next version is on ModDB (in January 2018), maybe that would be a good version to update to 2.8.2 afterwards as well.
I imagine eventually then an "update to new engine" file could be uploaded to ModDB in addition, so players can test both the Storm 2.0 and Maelstrom 2.8.2 versions side-by-side.

I fully support your position to not try to adapt the new engine to a constantly changing modpack version.
That wouldn't work and you're very much right to not do that.

Yep but I was thinking of COAS world map and new islands and mainland locations with initially potc NH quest content on the original locations but room for quest development not just swapping resources. I can't say I have looked at it or understand the issues yet (including how direct sail works in NH but is still zonal in later flavours) and maybe it is a non-starter but both being in the same engine must help I would (naively) have thought.

We can certainly hope!
I do like the CoAS worldmap, that's for sure.
Though I think in theory it could already have been adapted to PotC, but that would require a lot of work and then the quests would all need to be moved to other places.
Or the towns would need to be swapped out. Or something. But I definitely believe it is possible.

DirectSail in NH uses the actual worldmap coordinates. I think it was @Screwface who recoded that, as it used to be zonal with @CouchcaptainCharles' original version.

 

[pedrwyth]

Well I'm back home and after a bit of messing around losing files, trying to reset my laptop to factory and rebuild and the like am back to looking at this.

@ChezJfrey I have a problem with the transfer goods interface. It doesn't scroll away from the first item (food). I can fix the scroll but even then it still doesn't move the choice on to the highlighted good it stays with food (and I lose the ability to swap quantities). This interface is used in ship tansfer, exchange between ships and upon ship capture too.

I enclose a save game with two ships , easiest is use f2 then ship, ship transfer to get to the goods exchange interface to see/test.

 

[ChezJfrey]

Well I'm back home and after a bit of messing around losing files, trying to reset my laptop to factory and rebuild and the like am back to looking at this.

@ChezJfrey I have a problem with the transfer goods interface. It doesn't scroll away from the first item (food). I can fix the scroll but even then it still doesn't move the choice on to the highlighted good it stays with food (and I lose the ability to swap quantities). This interface is used in ship tansfer, exchange between ships and upon ship capture too.

I enclose a save game with two ships , easiest is use f2 then ship, ship transfer to get to the goods exchange interface to see/test.

Well, if you ask me, that is a flaw in the script that needs to be reworked. That scroll works perfectly fine, without the message that is sending 'scroll change' with a -1 index, which will cause the whole list to refresh. In INTERFACE\transfer_goods.c, function ChangeScroll(), comment out the line that is causing the scroll to wholly refresh:

SendMessage(&GameInterface,"lsl",MSG_INTERFACE_SCROLL_CHANGE,"GOODSLIST",-1);

If you remove that line, then the scroll works just fine, albeit showing all the zero quantity cargo items. I would suggest not refreshing the whole list as it undoubtedly causes the selected index to then reset back to the first item.

 

[pedrwyth]

Well, if you ask me, that is a flaw in the script that needs to be reworked. That scroll works perfectly fine, without the message that is sending 'scroll change' with a -1 index, which will cause the whole list to refresh. In INTERFACE\transfer_goods.c, function ChangeScroll(), comment out the line that is causing the scroll to wholly refresh:

SendMessage(&GameInterface,"lsl",MSG_INTERFACE_SCROLL_CHANGE,"GOODSLIST",-1);

If you remove that line, then the scroll works just fine, albeit showing all the zero quantity cargo items. I would suggest not refreshing the whole list as it undoubtedly causes the selected index to then reset back to the first item.

OK thanks for the comment. I would never have dreamed of just skipping the (for me) black box send message and was working round that being in place. However given that approach I now see in another pretty much identical use in shiphold.c it doesn't seem to have that reset effect. There are also a couple of cases where it is called with current scroll number etc. So I guess with further study I will sort it out and get a revised script (with or without a revised call). In the meantime, as you say, just skipping it does workround the break in functionality so when I get bored with looking at code I can go further on with playtesting too. I still hope to be in a position to migrate to the latest code in the new year (I have another fortnight away, on my own, which looks ideal for making a start). Thanks again

 

[ChezJfrey]

I didn't really spend much time debugging, as I'm not really inclined to debug much NH script stuff if it is not related to a deficiency/problem in the engine; in this case it seems more likely a a script problem, given it works fine without that reset. I think it might also have to do with the attempt to skip over the goods that both parties have zero, which is slightly different than the shiphold version, since without the reset, those are the goods we see.

So I'd also investigate the ChangeScroll where those goods get skipped:

if(GetCargoGoods(xi_refCharacter,i)==0 && GetCargoGoods(refEnemyCharacter,i)==0) continue;

 

[ChezJfrey]

I didn't really spend much time debugging, as I'm not really inclined to debug much NH script stuff if it is not related to a deficiency/problem in the engine...

OK, I lied. This will do it:

In ChangeScroll, within the two 'for' loops for visible and invisible goods, add, just before the variable idx increment at the end of those two goods loops:

SendMessage(&GameInterface,"lsl",MSG_INTERFACE_SCROLL_CHANGE,"GOODSLIST",i);
idx++;

Make sure the other SendMessage with the -1, after those two loops, stays removed/commented out.

 

[pedrwyth]

OK, I lied. This will do it: .........

Thanks for the extra unexpected assistance, I had the call worked out but not where to place it. Without the standard transfer was OK but during the more complex (and little known) swapping ships within your fleet in mid ransack after capturing a ship the scroll got confused (as per loot examples) so it's better now. I did get a bit side tracked on getting rid of the invisible goods part of the code (the concept to add them has been there since stock game) and thus have a neater interface which I think I had working but have left them in for now rather than add in any possible unforseen knock-on complications. That will go with an upgrade to allow companion charge changes once (if) I get the latest WIP code working pretty much as is.

I am still getting a few crashes, I have a few sets of logs saved I don't know if you're interested in seeing them from those occasions?

Your re-vamp of the worldmap attribute tree had, due to a resultant mismatches, left a_map.c (the one when you press M in game) without its labels and right click info which I have now generally restored.

Some time back I restored blow and wave in the standard man animation (to cut the errors in the logs) but haven't touched the others you mentioned yet.

 

[ChezJfrey]

I am still getting a few crashes, I have a few sets of logs saved I don't know if you're interested in seeing them from those occasions?

Sure. If you can remember what you were doing during those crashes, that might help too. The most typical reason for NH crashes with the newer engine is SendMessage calls where the expected parameters have changed. Unfortunately, there is also no log error for those particular errors, but if you can remember what you were doing, and I can recreate it, I can run in debug mode and it will direct me to the specific message call that caused the problem.

 

[pedrwyth]

Sure. If you can remember what you were doing during those crashes, that might help too. The most typical reason for NH crashes with the newer engine is SendMessage calls where the expected parameters have changed. Unfortunately, there is also no log error for those particular errors, but if you can remember what you were doing, and I can recreate it, I can run in debug mode and it will direct me to the specific message call that caused the problem.

Well I can't really remember (I have been messing about a bit with different version/games) and it was none too clear from the logs what I was in the middle of doing. I will bear it in mind for next time I crash.

I have added the key for firing your pistol back into controls (init_pc.c) and it now functions (on land at least - haven't tried on a boarding yet) and so it is also now restored to options for key re-mapping. EDIT but there are other side effects - so a bit more work required I suppose ENDEDIT

 

[pedrwyth]

Ok a couple of crashes here

The first was during ransack after capturing two ships in a row and is a ctd with "R6025 pure function call" error window. Compile and system logs no error log.

The second is a crash at the point where you are about to repel the english squadron in TOSH. The save is just before the cut scenes once you descend the stairs. Danielle is supposed to take over the fort commander role and there are a number of code lines in quests_reaction.c related to trying to keep some cannon charge and type attributes one of which (gunpowder) is commented out "for crashing the game". The error here in system log is "Assert failed in AIFort.cpp line 178, expression string pACannonsType3" This is a consistent stop to the game.

 

[ChezJfrey]

The error here in system log is "Assert failed in AIFort.cpp line 178, expression string pACannonsType3"

Turns out that Type.3 is required...for whatever reason, the engine asserts such and if not there, crashes.

In SEA_AI\AIFort, all the places that set/check for Cannons.Type.1 and Cannons.Type.2, need to add Cannons.Type.3:

Fort_GetCannonsQuantity
Fort_CheckAttributes

standard\quests\quests_reaction.c

case "Story_GreenfordCapturedByBlazeAndDanielle":
case "Story_VoyageToKhaelRoaBegan":

Towns\Towntable.c
SetTownFortCommander

 

[ChezJfrey]

In fact, while we're at it...should add

ch.Fort.Cannons.Type.3.Quantity = 0;

To FortCommandants.c, so the reaction_function stuff can copy just like Type.2

Also add Type.3 to Reinit.c, too.

 

[ChezJfrey]

The first was during ransack after capturing two ships in a row and is a ctd with "R6025 pure function call" error window. Compile and system logs no error log.

Can you repeat that and get the same result each time (or nearly every time)? If so, can you set up a save where you are about to board the first, and I will try to complete the battle and board the second ship thereafter. Also, describe each action taken during the ransack so I can try to replicate.

Since I don't really play NH, I don't have any save games with enough character skill that will ensure any sort of success, without decent leveling, so it would be helpful to borrow your character progression to save time/effort. Without log errors (with pure function call errors is typical in this source...I used to get them while debugging the mast falling problems), but in debug mode, using your save as a template, if I can recreate, will help narrow down the problem/resolution.

 

[pedrwyth]

Turns out that Type.3 is required...for whatever reason, the engine asserts such and if not there, crashes.
In SEA_AI\AIFort, all the places that set/check for Cannons.Type.1 and Cannons.Type.2, need to add Cannons.Type.3:

Thanks, surprisingly enough I had already placed the two type of type.3 lines into the two cases in quests_reaction.c (copying those for type.1 and type.2) in an attempt to see if it did anything. Since I didn't know whether it was even the right direction (or if type3 was going to turn out to be a missing type.2 with one of those 1-3 instead of 0-2 changes that sometimes crop up) I didn't follow through to see where else to try. Easier just to consult the oracle.
The requisite additional placings cleared the problem. I had just stepped round it by // the main win conditions etc so I skipped the battle completely and I have now completed TOSH to beyond sinking the Black Pearl (ie stock game conclusion). As a result I intend to present an "end of term" report of my conclusions (to follow).

Can you repeat that and get the same result each time (or nearly every time)? If so, can you set up a save where you are about to board the first, and I will try to complete the battle and board the second ship thereafter. Also, describe each action taken during the ransack so I can try to replicate.

I don't know if the pure function call crash will be repeatable I had only seen it once before I think, I will have to recreate a save from a little earlier because the nearest one got overwritten - I was at a specific point in the plotline so it should be do-able. Having just finished the play through I'll go back there now. I have had other more standard ctd's on ransack when trying to open the goods swap interface -can't remember if that was the last action before the pure function case - I'll have to re-check the changes I made following your earlier help with the scrolling in there just in case I have screwed it up since - just normal paranoia. I'll get back to you.

Since I don't really play NH, I don't have any save games with enough character skill that will ensure any sort of success, without decent leveling, so it would be helpful to borrow your character progression to save time/effort. Without log errors (with pure function call errors is typical in this source...I used to get them while debugging the mast falling problems), but in debug mode, using your save as a template, if I can recreate, will help narrow down the problem/resolution.

If you ensure cheat mode is enabled in internalsettings.h then the nymeric keypad allows rapid artificial improvement to skills, abilities, god mode etc so there is no requirement to work through the levels. With god mode on sail near enough the enemy and they'll board you! - it is just tedious but inevitable to work through the decks. I guess a console code could kill off your opponents but that would be REALLY lazy. EDIT I looked it up anyway Guide - Tips & Tricks for Playing the Game ENDEDIT However the first part is getting the save and seeing if I can repeat the issue.

 

[pedrwyth]

My completion of TOSH in JLB engine led me to consider what I think I found


So the first comment must admiration for the fact that @ChezJfrey got it to work at all. The original creators had no thoughts of backwards capability when they upgraded the engine through the storm series. Add to that he fixed many of their errors and moved it forward to dx9 etc. Then consider the amount of original functionality that he has breathed into its "raw" state without any real knowledge of the complexities that have grown into NH across its many years and individuals modding. A few changes in the later engine he has used in place of removed features still need a wider view on how best to present them but they work. With his troubleshooting a few more missing bits got added/fixed along my playthrough, a nember remain.

It is no coincidence I chose to work through TOSH. It is the original plot and ignoring added side quests probably has the least modded content. So I successfully went through the base game without visiting the extra islands not in the original. There are a number of rough edges but got from one end to the other so basic stuff mostly works. Amazing achievement @ChezJfrey

I know there are some outstanding problems with the build set so expect that voyaging to the added locations will raise other issues. The biggest known problem is the 60 or so added particle effects (and including some stock game not needed for AOP) which at the present are not supported. This means that in this sphere all the careful additions of mopdders fall by the wayside. For the present if you want to see the added storylines as the writer intended you will need to do that in the original engine.

If you want to play TOSH it provides a reasonable match to the original. With the newer added storylines I suspect it currently veers too far from the mod designer intentions to be "NH". I intend to continue looking at where there are deficiencies and which can be resolved. We really need a particle effects guru to change the INI code of the base game into html equivalents.

So why would someone bother?

New Horizons territories are pretty full/ well used. You only have to look at some of the threads regarding re-use of locations in different story/time lines and the attempts to squeeze the last locations into use to understand the advantages that new pastures could bring. Add to that the possibility (even if remote) of Weaving in tales/missions from the later series if all the locations are in play.

There are reputedly many improvements that come with the new engine, look at the later games to decide yourself if having the NH content presented like that is worth the candle.

However we need the new engine to support, mimic or supplant more of the content in NH than it currently does if it has any prospect of being an accepted platform for NH content. I for one will continue to throw in my twopenny worth to help that along. Anyway I get more out of fixing things than adding content since I'm not that creative.

 

[Pieter Boelen]

Thanks a lot for testing this, @pedrwyth! And major thanks as well to @ChezJfrey for making it in the first place.
Hopefully once the Build 14 Final is released to the public, more people will gain interest in your work as well.