1. Dismiss Notice
  2. GOG.com logo

    Thanks to YOUR votes, GOG.com now sells:
    - Sea Dogs - Sea Dogs: Caribbean Tales
    - Sea Dogs: City of Abandoned Ships

    Vote now to add Pirates of the Caribbean to the list!

    Dismiss Notice
  3. Under the Crossbones Podcast

    A Pirate Podcast with Interviews
    Music, Comedy and all things Pirate!

    - Episode Guide - About - Subscribe -
    - Twitter - Facebook - iTunes - Android -
    - Youtube - Fill the Coffers -

    Dismiss Notice
  4. New Horizons logo

    Quick links for PotC: New Horizons
    - Download latest version
    - Wiki - FAQ - Report bugs here
    - ModDB profile

  5. GOF logo

    Quick links for AoP2: Gentlemen of Fortune 2
    - Downloads and info
    - Historical Immersion Supermod
    - ModDB Profile

Dismiss Notice
New to the forum?
Please take a moment to read our Welcome Message and Forum Rules.

Engine Limitation Bad Saves

Discussion in 'Bug Archive' started by Talisman, Aug 23, 2015.

  1. Talisman

    Talisman Smuggler Storm Modder

    Joined:
    Apr 29, 2009
    Messages:
    2,825
    Gender:
    Male
    Location:
    Hiding From The Coastguard
    I have been getting several "Bad Saves" with Beta 4.

    When this happens I get this in the error.log


    :shrug
     

    Attached Files:

  2. Pieter Boelen

    Pieter Boelen (Not So) Old Seadog Staff Member Administrator Storm Modder Hearts of Oak Donator

    Joined:
    Nov 11, 2004
    Messages:
    66,586
    Gender:
    Male
    Occupation:
    Maritime Research: Project Engineer (Analysis)
    Location:
    Wageningen, The Netherlands
    I think I've seen that one occasionally very many months ago, but I believe it is just one of those random things that we cannot track down at all.
    It is unlikely that it is related to specific changes made for Beta 4.

    You can try the old tricks of removing the "options" file and emptying out your SAVE folder of everything you don't directly need.
     
  3. salonikasurf

    salonikasurf Freebooter Storm Modder

    Joined:
    Dec 26, 2008
    Messages:
    293
    Gender:
    Male
    Occupation:
    Engineer
    Location:
    Hay Point Australia
    This seems to be a problem - the further I get into the game (Beta 4) the more times I get bad saves - quicksave seems to work better.
    But still an issue.
     
  4. Pieter Boelen

    Pieter Boelen (Not So) Old Seadog Staff Member Administrator Storm Modder Hearts of Oak Donator

    Joined:
    Nov 11, 2004
    Messages:
    66,586
    Gender:
    Male
    Occupation:
    Maritime Research: Project Engineer (Analysis)
    Location:
    Wageningen, The Netherlands
    There has been a potential issue with this for years, as far as I'm aware. There is no reason why it should be either better or worse now.
    All I can suggest is to delete the file named "options" from your main game folder, don't have too many different profiles at the same time and remove all files from your SAVE folder that you don't need.
     
  5. Levis

    Levis Find(Rum) = false; Staff Member Programmer Creative Support Storm Modder

    Joined:
    Oct 6, 2013
    Messages:
    6,683
    Gender:
    Male
    Occupation:
    ICT
    Location:
    University Twente (Netherlands)
    How much RAM do you have?
     
  6. Talisman

    Talisman Smuggler Storm Modder

    Joined:
    Apr 29, 2009
    Messages:
    2,825
    Gender:
    Male
    Location:
    Hiding From The Coastguard

    I have 16 Gb of RAM
    Windows 8.1 OS

    :drunk
     
  7. Talisman

    Talisman Smuggler Storm Modder

    Joined:
    Apr 29, 2009
    Messages:
    2,825
    Gender:
    Male
    Location:
    Hiding From The Coastguard
    Beta 4 is the first time I have regularly experienced Bad Saves. I don't know if it is because this is the first time I am playing the game on Windows 8.1 ( previously I was using XP and Vista with only 4 and 2 GB of Ram respectively ) :shrug

    I am now deleting the Options File each time it happens.

    I only have 2 profiles ( one for story I am playing - the other with only 2 Saves )

    In the story I am playing I usually wait till I have about 35 Saves then I transfer all but the last 5 to an archive folder and continue playing. The Bad Saves seem to occur even when there are very few Saves in the Save folder.

    :type1
     
  8. Pieter Boelen

    Pieter Boelen (Not So) Old Seadog Staff Member Administrator Storm Modder Hearts of Oak Donator

    Joined:
    Nov 11, 2004
    Messages:
    66,586
    Gender:
    Male
    Occupation:
    Maritime Research: Project Engineer (Analysis)
    Location:
    Wageningen, The Netherlands
    :boom:

    I have marked this as "Engine Limitation" because it IS a real issue, but I doubt any of the recent code changes has caused it. It is likely that this has plagued us since the stock game.
    Even if it could be fixed, it would be a massively complicated undertaking to even try with no guarantee of success. :(
     
  9. salonikasurf

    salonikasurf Freebooter Storm Modder

    Joined:
    Dec 26, 2008
    Messages:
    293
    Gender:
    Male
    Occupation:
    Engineer
    Location:
    Hay Point Australia
    I find that I get bad saves nearly every time I try to save when time is sped up x3 etc using the R key. if I then go back to normal game speed using the G key - Game Save often then works. Also often happens if you want to save soon after a location change. I think it is a recent thing as in only the year or two. I don't remember having any issues with this in Build 13 at all.
     
  10. Pieter Boelen

    Pieter Boelen (Not So) Old Seadog Staff Member Administrator Storm Modder Hearts of Oak Donator

    Joined:
    Nov 11, 2004
    Messages:
    66,586
    Gender:
    Male
    Occupation:
    Maritime Research: Project Engineer (Analysis)
    Location:
    Wageningen, The Netherlands
    @salonikasurf: What is your current modpack version?
    Indeed @Levis' new Leveling System did have that as effect, but that was reported fixed again months back! :shock
     
  11. salonikasurf

    salonikasurf Freebooter Storm Modder

    Joined:
    Dec 26, 2008
    Messages:
    293
    Gender:
    Male
    Occupation:
    Engineer
    Location:
    Hay Point Australia
    Hi Pieter,
    14 February B14 B4.

    It may be a Windows 10 thing - although I did have issues on 8.1 as well. Sometimes it was good when an update came out and other times is was quite bad. So it is very hard to tell where the issues are.
     
  12. Ron foster

    Ron foster Landlubber

    Joined:
    Oct 11, 2016
    Messages:
    17
    Gender:
    Male
    Occupation:
    retired Coast Guard Commander, PhD scientist
    Location:
    Central US
    Over which only the coder gods have control... arrrggghhh matey:(
     
  13. ChezJfrey

    ChezJfrey Master Mariner Storm Modder

    Joined:
    Apr 24, 2015
    Messages:
    221
    I would say your first two assertions are correct: It is an engine problem, and has plagued the game since the beginning. There is also a reason why this particular 'no memory' error at line 1048 appears in the log, along with the 'Bad Save', more often in standard save, vs. quick save; that NewInterfaceTexture is being called for each slot iteration to create a texture image for the save on the interface screen, calling MSG_SCRSHOT_READ message. This then requires the engine to allocate some memory to read and store that set of image bytes for display.

    Each 32 bit process only gets a max of 2gb memory allocation. That memory is going to be consumed by the mod/program scripts, the models, textures, etc. As New Horizons has become more complex, offering more content, and more complicated, it undoubtedly initiates more memory transactions. Regardless of complexity, as one leaves locations, switches interfaces, etc., those memory allocations are deleted/restored accordingly -- they don't remain persistent forever. But here is where the 'engine limitation' comes in...the original executables are rife with memory leaks, missed/forgotten mem deletes, etc. When new memory is requested (e.g. the MSG_SCRSHOT_READ in this case), the amount of memory has to be a contiguous, unused block of the requires size, within the limitation of what Windows grants for a single 32 bit process. Memory leaks and Windows' own memory allocation algorithms, exacerbate memory fragmentation (not to be confused with disk fragmentation), which over time, severely curtails the size of contiguous, unused memory blocks and given that New Horizons now probably produces more allocate/deallocate transactions with more script activity, the memory is becoming quite fragmented.

    What this means is that the game process might actually only be actively using 500mb, if the entire 2gb memory space it can theoretically use, is greatly fragmented, and it needs 100 mb allocation, but can't actually find a contiguous, unused block of that size within its given space, the memory allocation will fail. That is the scenario you are likely seeing here, because looking at the source code, that "no error" message is clearly in the old version of the GetSaveData function of the engine.

    This leads to your 3rd statement: "Even if it could be fixed, it would be a massively complicated undertaking to even try with no guarantee of success."

    I believe it has been fixed. The newest compilation employs a different GetSaveData, and also has corrected/removed hundreds of memory leaks, and also uses a different memory allocation algorithm, much improved over the Windows standard library; it is both quicker, more efficient, and greatly reduces memory fragmentation so this type of problem is much less likely to occur. New Horizons was also modified to work with it.
     
  14. ChezJfrey

    ChezJfrey Master Mariner Storm Modder

    Joined:
    Apr 24, 2015
    Messages:
    221
    To clarify, when I state 'newest compilation' that is not 2.8 COAS, as that version still contained the many memory leaks, poorer memory allocation algorithm and other bugs; the version referred is my newest compilation. What is also interesting, that while there are a couple 'likes' for the above, to date, there is no interest in using my newest compilation and the reworked New Horizons enabling usage of this later and improved engine; so far only ERAS, a Russian mod, and Modder01 for CT have embraced it.
     
  15. LarryHookins

    LarryHookins Buccaneer Staff Member Storm Modder

    Joined:
    May 20, 2006
    Messages:
    2,291
    Location:
    Texas
    Jeffrey, if I were still playing PotC I'd definitely be using your new engine. My latest work has been modding TEHO, and I haven't seen many engine problems there, and I haven't even played TEHO in weeks. I'm currently sailing the virtual Caribbean (in scale) in Vehicle Simulator.

    Does your engine fix the need to run a frame rate limiter just to play PotC on a modern computer? That would be reason enough for anyone to give it a try.

    Hook
     
    Pieter Boelen likes this.
  16. DeathDaisy

    DeathDaisy Freebooter Storm Modder

    Joined:
    Aug 18, 2017
    Messages:
    268
    Gender:
    Female
    It was very interesting to read about why the engine does what it does! xD I kinda get the theory, but I have no idea what to do or where to even start with applying the compilation :p Is it very complicated? Or is it just a switch of exes? Or somewhere in between? if it's not too far over my head I'd definitely be interested! :beer:
     
    pedrwyth likes this.
  17. pedrwyth

    pedrwyth Privateer Storm Modder

    Joined:
    Jan 8, 2015
    Messages:
    646
    Gender:
    Male
    I did download your reworking of New Horizons to use your newest compiled engine. It is naturally enough based on NH roughly as at Moddb release (ie latest formal public release March/April 2016) if I recall correctly. I would intend to try and add in the changes to bring the game up to WIP so it has the fixes/additions introduced since then and hopefully see more interest in using it emerge. I presumed I would have to winmerge compare your files with that (MODDB) to see which you had had to alter and then try and physically do an actual winmerge with the latest. However since you released it (July?) I have been busy with my summer activities and now travelling so won't be looking in earnest any further until the end of the year. If you have documented which files you had to rework it could help me a lot when I get back to it (unless someone else steps in in the meantime).

    I also got somewhat sidetracked looking at your comments and workround for sky textures and then went looking for how it was done in later versions of storm games, one of the many fields I knew nothing about in the first place, and concluded it won't be me fixing that!
     
    Last edited: Oct 22, 2017
    Pieter Boelen likes this.
  18. ChezJfrey

    ChezJfrey Master Mariner Storm Modder

    Joined:
    Apr 24, 2015
    Messages:
    221
    Ha! So you gave it a look...that is encouraging. And yes, I picked up New Horizons awhile back and will not have the most current fixes/changes.

    My intent was to preserve as much of the New Horizons experience as possible, while getting it functional in the new. While I have ascertained it at least launches a game and I can sail, fight, save/load, etc., given my experience with Modder01 and his efforts with CT, even in the much closer 2.5 engine, there are undoubtedly a few things that still need attention. Especially since I am unfamiliar with actual gameplay in POTC, NH and CT, so I have probably missed some minor details. My hope was to at get as close as possible, then someone like yourself and others much more familiar with New Horizons could pick up and attend to some of those remaining facets.

    I will attempt to inform what I can remember.

    First, script file changes. There is a handy piece of software that might prove helpful, at: FreeFileSync

    It will help identify altered files, by selecting two directories, and changing the 'compare' filter to 'file content'. Then running the 'compare' feature, it will scan and identify changed files. The list of changed script files:

    dialog.c
    globals.c
    InternalSettings.h
    MAXIMUS_Functions.c
    messages.h
    NK.c
    quicksave.c
    sail_msg.h
    seadogs.c
    utils.c
    BATTLE_INTERFACE\BattleInterface.c
    BATTLE_INTERFACE\fast_reload_table.c
    BATTLE_INTERFACE\Flags.c
    BATTLE_INTERFACE\ispyglass.c
    BATTLE_INTERFACE\LandInterface.c
    BATTLE_INTERFACE\LogInterface.c
    BATTLE_INTERFACE\msg_control.h
    BATTLE_INTERFACE\reload_tables.c
    BATTLE_INTERFACE\utils.c
    CANNONS\Cannons_init.c
    Characters\characters_init.c
    Characters\CharacterUtilite.c
    Characters\init\CommonQuest.c
    Characters\init\FortCommandants.c
    CONTROLS\controls.c
    CONTROLS\init_pc.c
    DIALOGS\ENGLISH\eng_officiant_dialog.h
    INTERFACE\boal_map.c
    INTERFACE\info_box.c
    INTERFACE\interface.c
    INTERFACE\itemsbox.c
    INTERFACE\kam_articles.c
    INTERFACE\kam_rations.c
    INTERFACE\kam_selectmodel.c
    INTERFACE\kam_selectsails.c
    INTERFACE\kam_shipberthing.c
    INTERFACE\kam_shiptransfer.c
    INTERFACE\map.c
    INTERFACE\messages.h
    INTERFACE\NationRelation.c
    INTERFACE\ransack_main.c
    INTERFACE\save_load.c
    INTERFACE\TradeBook.c
    INTERFACE\transfer_crew.c
    INTERFACE\transfer_main.c
    INTERFACE\utilite.c
    ISLANDS\islands.c
    ISLANDS\Islands_init.c
    Locations\locations_loader.c
    Loc_ai\LAi_character.c
    Loc_ai\LAi_defines.c
    Loc_ai\LAi_events.c
    Loc_ai\LAi_fightparams.c
    Loc_ai\LAi_init.c
    Loc_ai\LAi_login.c
    Loc_ai\LAi_utils.c
    SEA_AI\AIBalls.c
    SEA_AI\AICameras.c
    SEA_AI\AICannon.c
    SEA_AI\AISea.c
    SEA_AI\AIShip.c
    SEA_AI\Script_Defines.h
    SEA_AI\sea.c
    Ships\Ships_init.c
    Storyline\Bartolomeu\SL_utils.c
    Storyline\GoldBug\SL_utils.c
    Storyline\GoldBug\quests\quests_reaction.c
    Storyline\standard\StartStoryline.c
    Storyline\WoodesRogers\SL_utils.c
    Storyline\WoodesRogers\quests\quests_reaction.c
    Towns\Towntable.c
    Weather\WhrSky.c
    WorldMap\worldmap.c
    WorldMap\worldmap_events.c
    WorldMap\worldmap_globals.c
    WorldMap\worldmap_init.c
    WorldMap\worldmap_reload.c

    Also, all the .ini files were slightly changed to disallow resizing of interface for wider resolution, most notable and significant changes were to rigging.ini for the way New Horizons handles flags. I think flags could be improved, because I have coded a feature to allow multi-row textures for flags, vs. the single row limitation, but I did not go so far as to making all the script file changes and combine the flag texture files to make use of that, and simply retained the current way they are handled in NH.

    RESOURCE animation .ani files need additional definitions for more of the 'random' actions allowed. I simply copied an existing action multiple times (with the added 'number' used for random selection). Examples:

    man_ab.ani
    blaze.ani
    Beatrice.ani
    man.ani
    beatrice_ab.ani

    SKY changes: As you mention, the old version only has a _side for sky, but newer uses a right, left, back, forward texture. To accommodate POTC/NH, I added a feature in the engine that will look for a .useOld attribute and if noted, will presume _side is right, left, back, and forward, to create the cube texture for sky. I would think a better approach would be to create the additional sky textures and make use of the expanded feature of sky wrap-around.

    Worldmap
    rename islands.gm as mein.gm
    add islands.gm locators for merchant, quest, island labels...this locator file probably needs some work, but I did the minimum to get it functional in the game. Will also need some script work to read/give the worldmap encounters the from/to locations the same way the current game works.

    I also believe some texture and/or model files were moved to some different subdirs...can't remember them all, but the newer game does have some hardcoded paths that were changed from 2.0 to 2.8 version...I just can't remember them all and did not specifically document them.
     
  19. ChezJfrey

    ChezJfrey Master Mariner Storm Modder

    Joined:
    Apr 24, 2015
    Messages:
    221
    I don't know the answer. I have never played POTC, so I do not have a decent basis for the problem in question. I also don't know if my particular PC is considered 'modern', since I run fairly archaic (relative to the bleeding edge types), LOL My most 'modern' PC is a laptop, Intel stock (no fancy video card), circa 2012 :)

    In any case, it appears to run OK in the preliminary tests I have run with no such limiter.
     
  20. Pieter Boelen

    Pieter Boelen (Not So) Old Seadog Staff Member Administrator Storm Modder Hearts of Oak Donator

    Joined:
    Nov 11, 2004
    Messages:
    66,586
    Gender:
    Male
    Occupation:
    Maritime Research: Project Engineer (Analysis)
    Location:
    Wageningen, The Netherlands
    @LarryHookins is referring to this issue: "Solved" - Windows 10: Low Frame Rates Despite (BECAUSE OF!) Good Hardware
    It is massively annoying and, as far as I'm aware, does not apply to CoAS.
    So if indeed your engine fixes that for PotC, it is a massive win for you indeed! :cheers
     

Share This Page