Updating files is too slow

Discussion in 'Game Development (Technical)' started by semichaud1, Apr 13, 2015.

  1. semichaud1

    semichaud1 New Member

    Joined:
    Apr 13, 2015
    Messages:
    1
    Likes Received:
    0
    Hi everyone. I'm new to this forum.

    I have programmed a game engine for the past two years. The engine uses data stored in files. In order to edit the data in those files I have programmed a separate procedure which I call construction tool, which allows the editing of these files. I am now testing the function of the construction tool which writes and reads data from the file in which all the value of tiles on all layers in the game is stored. My construction tool program allows me to edit a specific area of a chosen layer in a chosen region of the game world, except when the program updates the value of the area in the data file it is taking too long because the total area of 1 region is 12,800 x 7,200 pixels. I have tried making it smaller but that won't solve the issue when I will be storing data for multiple regions in the file. Also making it smaller means a smaller region size over which the player can walk.

    Am I approaching data storage in a wrong way? I have chosen to use data files because it is the only way to store really large worlds and permanent data, instead of coding the tiles in the game engine itself.

    Sorry if I am not very clear. Please ask if you don't understand my question I will try to explain the issue better.

    Thanks.
     
  2. stan

    Original Member Indie Author

    Joined:
    Jul 27, 2004
    Messages:
    188
    Likes Received:
    0
    If I understand correctly, your world is tile-based. It is also "persistent" which means that during the game the world will change and those changes must be saved back on disk?

    12800×7200=92,160,000 tiles in one layer of one region? And there are several layers? Even with only one byte per tile it’s indeed huge.

    Ideas:
    - make the regions smaller (make "sub-regions" if you have to)
    - use a different file per region or sub-region
    - don’t save the changes all the time, keep them in memory as long as you can
    - you can try compressing them with a fast algorithm before saving
     
  3. bantamcitygames

    Administrator Original Member Indie Author Greenlit

    Joined:
    Jul 27, 2004
    Messages:
    1,738
    Likes Received:
    80
    I agree with stan's suggestions. If it were me, I would make the editor load in a region/sub-region at time, allowing all of the editing to be done in memory and only write it back out to the disk when the user saves or switches to a different region/sub-region.
     
  4. Michael Flad

    Indie Author

    Joined:
    Aug 4, 2004
    Messages:
    190
    Likes Received:
    0

Share This Page

  • About Indie Gamer

    When the original Dexterity Forums closed in 2004, Indie Gamer was born and a diverse community has grown out of a passion for creating great games. Here you will find over 10 years of in-depth discussion on game design, the business of game development, and marketing/sales. Indie Gamer also provides a friendly place to meet up with other Developers, Artists, Composers and Writers.
  • Buy us a beer!

    Indie Gamer is delicately held together by a single poor bastard who thankfully gets help from various community volunteers. If you frequent this site or have found value in something you've learned here, help keep the site running by donating a few dollars (for beer of course)!

    Sure, I'll Buy You a Beer