I had a similar incident the other day after a spanish fleet containing queen anns revenge, a heavy line ship and some other beast of a ship turned up while I was attempting to escape a hostile spanish port with the black pearl >.<
After some frantic cannon fire and some crazy luck as I speed away from the forts range I manage to demast both QAR and the heavy line ship, I then quick save thinking I'll have a crack at turning about and loading up some buckshot now that they are somewhat disabled but the very second I click to fire my next volly I get a CTD, I reloaded several times and ever single time the very moment I tried to fire the first shot after reloading it went and CTD.. The AI ships were able to fire fine though, and I was able to sail about as much as I liked without it crashing but no matter how long I waited as soon as I hit fire - CTD.
No change to the crash wether I was in 1st or 3rd person sailing or wether I manually aimed at them, away from them, straight up into the sky so I couldn't see the shot just in case it was some rendering issue or something, all the same result.
I sadly reloaded once more and just sailed away from the fight and exited to sea. The next fight I was in didn't have the same problem at all, I was able to fire merrily away with no issue.
I've demasted multiple ships since with no ill effect, though they were much smaller ships - demasted a class 3 and a class 5 escort that were in a convoy with class 5 & 6 traders and no problem, the black pearl got double demasted by a heavy line ship - that wasn't fun, but it didn't CTD like it did in that one fight.
As a side note I sold the trashed pearl - (well, gave away seeing as it was worth 0 PoE to sell
) and kept the heavy line ship that tried to board me har har!
So perhaps there is potentially some limit to the combination of number of cannons or size/number of ships that can be in one fight or something with the added complication of demasted ships? Or perhaps the ship specific or a combination of all the above? I have no way of knowing but hopefully you guys can nail it down