• New Horizons on Maelstrom
    Maelstrom New Horizons


    Visit our website www.piratehorizons.com to quickly find download links for the newest versions of our New Horizons mods Beyond New Horizons and Maelstrom New Horizons!

Solved Heap Corruption, Random Crashes

amplificar

Opcode Assemblar
Storm Modder
Heap corruption connected to the usage of DX8Render ReleaseVertexBuffer() eventually leads to the game crashing.

This current patch skips releasing the vertex buffer, which hurts game performance, but keeps the game running longer and with fewer graphical anomalies.

Once I learn more about the cause, there will be an improved patch that wont affect game performance.

Installing The Patch
A) Backup your PotC MODULES\battle_interface.dll (move it to the desktop or add ".backup" to it's filename)
B) Download and decompress EU_BI_bypassFDC0.7z into the MODULES folder

[EDIT] The previous workaround, registry modification to enhance the fault tolerant heap, was removed.
[EDIT] Improved heap corruption patch, for US/EU and Build 14.
 

Attachments

  • EU_BI_bypassFDC0.7z
    62.7 KB · Views: 593
This could very well explain many of the apparently random crashes we have experienced over the years, a lot of those have involved game saves and switching between modes. Some of the other strange crashes we have seen involve CTD's on startup that may very well be related to this memory bug, or the loading of the wrong shaders that you mentioned in the other thread. Some others are caused by audio and video codecs, and the rather strange one of keeping your hard drive defragged. :unsure

I think part of the problem lies with the developers not having an extensive amount of different testing equipment to try the game out on before it's release. The fact that Bethesda and Disney forced them to release it much earlier than originally planned didn't help either. The open beta tests that have become a trend the past few years sure would have helped tremendously with this game, it's unfortunate that wasn't an option 10 years ago.

The above registry edit is something that not many gamers would be aware of, or even consider, myself included. Personally, I think Microsoft could have done a much better job of implementing the system registry, the way it is now only confuses and confounds the normal computer user. :modding

In just the past couple of weeks mate, you have helped us narrow down some of the biggest problems that we have had with this game. Thanks once again for your time and effort, it is very much appreciated! :onya
 
Ok, I done did this. :read Now why set compatibility to XP SP2? :pirate07: Also, I have 8GB of ram. If a little bit is good a whole lot is better! :woot What is a big number I can use? This game uses almost no ram at all as near as I can tell. In fact I suspect it goes straight to the virtual memory on the hard drive, which might be the reason for the defrag issue.


For me the CTDs happen during boarding battles and after the battle when I try to sail away on the world map. So I learned to avoid battles. :boom:

EDIT: Initial testing is encouraging. Selling cargo to a merchant seems to go faster, repairing ships ditto, and no ctd in a boarding battle with me doing all the things that normally cause a ctd. Back to more testing.....:sail
 
XP SP2 spoofs DirectX 9.0c, affects memory management, emulates audio and DLL behavior no longer in Windows Vista/7, sets flags for (allowing?) the fault tolerant heap, and more.

If it helps and the game runs longer on average, then with 8GB of RAM the highest I'd recommend trying is a value of 400 in both fields. That could quadruple the length of time the game averages between crashes. The value is in hexadecimal and megabytes, so FF = 255MB, 400 = 1024MB. The field in FTH that has DelayFree in it's name will benefit from more memory than the other one. Remember to reboot after changing the registry keys.

If you have programs installed on your computer that crash more often than two or three times per hour (web browser, other games, etc) then modifying FTH could also affect them - so your web browser might start using up a lot more memory. If you notice your computer's performance slowing down then the values for FTH might be set too aggressively.
 
The reason I asked about the XP SP2 compatibility is that I tried all of the compatibility settings last year and got a drop in fps with them. I am getting less fps now than before. There might be a conflict as I have DX9c installed already. It does help POTC run better with Win7.

Does POTC crash that often for you? o_O None of my programs crashes except POTC, and I know when, so avoid that situation except for today in testing and got no CTDs so far. More testing is needed.
 
It crashes pretty often for me, but I'm doing repetitive things for debugging. So normally I'm encouraging the game to crash.

The FTH doesn't require compatibility mode. It'll make a big stability improvement all by itself. I recommended XP SP2 anyway because most people in the last thread said they have DirectX 11, and I've got DirectX 10.

The way FTH works is unusual. I used the opportunity earlier to point out what could go wrong for anybody using it. The FTH gets activated automatically, so it can affect a whole slew of programs. If a program crashes more than twice an hour the FTH starts protecting it. After seven days, it resets which programs it's affecting until they start crashing again. If you look at the registry inside FTH is a folder called State, and engine.exe will be listed in there if PotC is being protected. A few rare people might have lots of stuff listed in State, if they have bad RAM, a failing hard drive, overheating, or other serious problems. For them, modifying FTH isn't a good idea. Most people will just have PotC in there and FTH never gets used otherwise.

Thanks for testing :type1
 
I'm not a good tester for this. I hope someone with Intel graphics tries this. I have DX11 with DX9c installed also. I am getting a 10-15 fps hit when in XP SP2 compatibility mode, so do not want to do that. Others may get different results.
I looked in "state" and it is empty. POTC did crash on me last night while I was wandering around town. That is a completely random crash and was the only one.
 
I'll start posting real fixes by this weekend. Adjusting FTH was more trouble than future testing will be. Once there are real patches being used, crashing will be less common - finding events leading up to a crash will be the test. Basically, just playing the game normally.

This afternoon, I tracked down a source of corruption in battle_interface.dll (0xfdc0). I have to stare at the assembly language a while to figure out what it does and how it's supposed to work. It's very patchable, once I determine a good fix for it.

I'm sure there's more than one source of heap corruption though. I'll just track them down, one after another.
 
This sounds very promising, mate! It would be fantastic if you could improve the game's stability, and I'm sure many players will appreciate it.
Unfortunately it just goes to show how poor Akella's coding was... :rolleyes:
 
The main problem I'm looking at happens in battle_interface.dll. It's something that has to do with freeing the memory used by a vertex buffer. On my computer I've set up WinDbg to use Microsoft's symbol database. The operating system libraries show their names now, which is very nice.

The pieces of the puzzle left to identify are the three obscure battle_interface routines shown in the call stack below. Then I'll be able to determine the code in "PROGRAMS\Battle_Interface\BattleInterface.c" that triggers the problem. Ultimately, the root cause might be in the DLL, not how it's used in game... I tested the obvious thing already, commenting out "DeleteClass(&BattleInterface)" in BattleInterface.c - but that didn't prevent the library code from being triggered (nothing else I tried commenting out in BattleInterface.c has prevented triggering the code either).
Code:
(The top of the stack is the last thing being done by the program.)
(Each line down is an earlier routine, which called the next one up.)
0012f718 6efd9df2 verifier!VerifierStopMessage+0x1f8
0012f77c 6efda22a verifier!AVrfpDphReportCorruptedBlock+0x1c2
0012f7d8 6efda742 verifier!AVrfpDphCheckNormalHeapBlock+0x11a
0012f7f8 6efd90d3 verifier!AVrfpDphNormalHeapFree+0x22
0012f81c 77805674 verifier!AVrfDebugPageHeapFree+0xe3
0012f864 777c7aca ntdll!RtlDebugFreeHeap+0x2f
0012f958 77792d68 ntdll!RtlpFreeHeap+0x5d
0012f978 759174d9 ntdll!RtlFreeHeap+0x142
0012f9c0 6d7ba433 KERNELBASE!LocalFree+0x27
0012f9cc 6d7caf08 d3d8!MemFree+0x13
0012f9d8 6d80d1ac d3d8!operator delete+0x18
0012f9f4 6d80b01b d3d8!CBuffer::~CBuffer+0x3c
0012fa04 6d816e24 d3d8!CVertexBufferMT::`scalar deleting destructor'+0x2b
0012fa14 6d80b7c8 d3d8!CBaseObject::ReleaseImpl+0x34
0012fa24 045c694b d3d8!CMipVolume::Release+0x38
WARNING: Stack unwind information not available. Following frames may be wrong.
0012fa3c 020be241 DX8RENDER!DMAInterface+0x477b (this is DX8RENDER!ReleaseVertexBuffer)
0012fa4c 020bfe40 battle_interface+0xe241
0012fa68 020bfdc8 battle_interface!DMAInterface+0x940
0012fa70 004186c0 battle_interface!DMAInterface+0x8c3
0012fd64 004147db ENGINE+0x186c0

I created a temporary patch, which should work for everyone (attached to this post is "BI_bypassFDC0.7z"). The patch skips the code in "battle_interface.dll" that leads to memory corruption. I've noticed an improvement in game stability, but this patch trades memory corruption for a potential memory leak instead (kinda like what FTH does, except very constant and precise). Once I figure out what the last routines are, I'll come up with a better solution for it.

battle_interface.dll file offset 0xFDC0 was 56, changed to C3.

----------
I might've found a second bug in the game. I haven't identified it by name yet, but a missing texture gets used by the game sometimes in sailing mode (in vanilla PotC). This causes accelerated SSE2/MMX texture blitting on my computer to crash when it trys using it. It seems to depend on a random sailing event.

----------
I've started working on naming all the routines in DX8RENDER.dll and battle_interface.dll. Doing all the game modules would take months, so I'm holding off on the rest until I need them or I can find time. When I have a mostly complete list of names, I can turn them into debugging symbols. An example of the benefit to doing this is in the call stack shown above, I determined DX8RENDER!DMAInterface+0x477b is in fact a routine named DX8RENDER!ReleaseVertexBuffer. A few of the names like this one were revealed by looking for exception handling text stored in the DX8RENDER.dll. Most of the routines aren't this easy though, they have to be named by figuring out what the code does or what system libraries it calls.

----------
While digging in files with a hex editor, I found the names of the built-in functions for the game's programming language. I wrote them down as a comprehensive reference manual for myself, but I haven't determined most of the parameters and return values. This is probably already known to modders, but I thought I should share it in case it wasn't (or in case some of the functions were missed).

I was hoping to find a command like Breakpoint() that would drop into the debugger I'm using. That command is meant for a debugger built into the game though, not a separate one like WinDbg (so it does nothing but freeze the game when used). I'm thinking about trying to modify engine.exe so that Breakpoint() triggers a real application interrupt. That'd help a lot.
 

Attachments

  • BI_bypassFDC0.7z
    63.2 KB · Views: 367
  • CoreReference001.7z
    813 bytes · Views: 344
I just got done trying out the bypass. It seems to deliver a framerate hit but also seems to reduce the Graphical Anomalies. So it is a "3 steps forward, 2 steps back" kind of thing.

EDIT: Just got done playing some more and could not get it ti crash during 3 consecutive boarding battles, even when doing things like looting corpses that always caused a crash in the past. Also, the English galleon, the Revenge, has always caused heavy GA when demasted. In this battle there was almost none at all! :woot There was some when approaching port, but it went away with a quick command to raise and lower sails.
This is a huge improvement! :cheers
 
That's a huge relief. It shows that I've really found the spot heap corruption begins, and it affects the game as much with mods as without.

Was the drop in framerate gradually getting worse, or did it stay constant? I didn't notice this on my computer because I'm testing at 640x480 32bit windowed. It must be a side effect of simply disabling the release of vertex buffers. When a better fix is made, this problem might disappear.
 
The framerate drop is mainly noticed at sea and on land when many npc's are around. It dropped immediately and has stayed constant or gotten a bit better. :treasure:
 
I used your battle_interface.dll to test it but then i reverted back the the stock one to compare but i noticed the game no longers works anymore. It loads up fine but when i example buy something in the store it just crashes (XINTERFACE.DLL). So i used the modded dll file again but that did not fix it. Any help will be very welcome.
 
No it did not crash. After reverting back the the default dll file it crashed. Its weird that it only crashes at the store, going to sea otherwise works fine
 
Back
Top