Assets and version control

There is one tool I don't rememer the name for doing that ... I searched for it but coudn't find it, if I remember it later I'll post again.

What I do is to have a repository for Code, other for assets sources y other for binaries. The binaries one holds the assets objective files and executables.

The assets one is the biggest, after finishing a project I make a backup on DVD of the assets and delete the repository and start again with it.

 

One way we overcome this is to periodically run an svndump and then filter out the paths of the asset folders before recreating the repository.

http://svnbook.red-bean.com/en/1.0/ch05s03.html
http://allmybrain.com/2007/10/15/using-piped-svndumpfilter-commands-to-separate-an-svn-repository/

 

You can do this with Perforce (perforce.com). Perforce is free for two users or less.
You can specify for each file the number of revisions you want to keep. You can set it to one revision for binary files and unlimited revisions for source code.

I personally use Perforce for source code only. My assets are saved using a standard backup system from time to time (no source control, but I sometimes keep two versions of my backups just in case).

 

Do you have 1 game per repository? Or is it lots of games in a single repository? If it's 1 repository per game then that must be a BIG game to take up 9 GB. Otherwise you should have separate repositories per each game and backing up would be easier / you can easily archive away old projects.

 

Losing history on assets sounds horrible to me. I always want to view history or go back to any previous version.

Shrinking a repository is best done by svndump (as Indiepath suggested). This way your latest repository gets smaller (with any filter you want to apply on svndump), but you still keep *all* of your history backed up in your old repository in case you still need it. So in my opinion it offers the best of both: small latest repository and complete history (although in another repository).

The bad part is that now and then, you need to run svndump, but normally this shouldn't happen often.

 

It's possible to remove old revisions in CVS, haven't done it though as I don't mind the required space so far.

What I did wrt to most easy and fast backups ... our CVS and SVN server runs in a virtual machine - so doing a full backup to an external HD is a question of a few minutes. External 2.5" HDs (no need for external power supply) are dead cheap these day's so I use 3 of them ... one to do current daily backups exchanged with another one every few days. I have 3 because I always have one in another house, just in case the office burns down etc.

One additional advantage of the VM is, if the machine running the VM dies for whatever reason I can move our CVS/SVN to another one easily and with almost no installation/configuration work.

 

With git you can write a script that after a commit it removes the history of some files. Your script can check a special file in your repository for the list of files to keep only the latest commit of. How to do that exactly depends on your repository configuration. Git itself doesn't provide this functionality, but it provides a lot of hooks to add it yourself. If you look around you might find premade receipes for this since its something that some people need (usually those who want to keep binaries).

But start with a "git svn clone <your svn repos>" first. Git repositories are much smaller than SVN repositories and you might be able to fit it in a DVD without the need of losing binary history (which might not be a good idea since old code usually depends on old binaries/formats).

Another option would be to buy a BluRay recorder .

 

I use multiple repositories, and retire ones that I'm not using currently from my backups. I think my largest repository is only 1 GB, but yeah, SVN Dump sounds like the way to go.

Video and Audio I don't use a repository, but burn a backup ("master") copy once I'm satisfied. Songs and project files I save multiple numbered files for revisions. Not perfect, but it's better than nothing.

 

Another possibility would be using a pro account of Dropbox for asset storage. Dropbox saves a history of all deleted and previous versions of files for 30 days by default. An all-in-one simple VCS and backup solution

If you wanted to keep the full history of files, you could always enable the pack-rat add on. It costs an extra $3.99/month I think, but the file history does not count against your storage limit so you can forget about all file versions taking too much space.

 

One thing to add, when working with multiple repositories, svn:externals is very useful. With it you can create a "virtual" folder in one repository that points to a folder in another repository. You can fix the revision of the source folder or set it to just get the latest on update.

You could for example create a separate assets repository for a project and create a svn:externals folder in your main repository that points to the assets repository. This way you can backup them seperately, or if later you decide to clear old versions of assets you can do that in the assets repository without messing with the main repository.

 

We made our own software (Asset Management) to keep track of code, screenshots of artwork assets, location on the network, licence agreements and so on as our pool of assets started to get out of control. We're going to release this as a tool for game studios in the next few weeks, if you want me to keep you updated so you can have a look at the demo, drop me a PM with your email and I'll keep you updated. It's totally online through a web browser with full search functionality (requires php/mysql database) so if you have remote teams updating it then you can give them access with priviledge levels. It can also be used as a portfolio, or showcase work to clients with their own login.

Edit: forgot to mention - free for Indie developers.

 

How do you do this in Perforce? I can't find the option...

 

Is your SVN server local? If so, you could do the daily/nightly backups to an external HDD. Added benefit is you can rotate disks every day or week (allowing offsite storage of the unused backup disks).

Every so often you can then take a new full dump and burn that to DVD. If it's over the size of a dual layer dvd then use tar with the multi-volume option to span disks, you could also pipe the results through BZip which should help reduce the size.

If the repo is remote, then a full dump once every month or two + incremental dumps is probably a better option.

As already mentioned svn:external is worth looking into. We've started to use this quite a bit with our latest projects. Your final option is to buy a blu-ray burner

 

With P4V, right click a file in a changelist (marked for add and change), select "Change Filetype...", Checkbox +S server limits the number of revisions.

 

I use one repository for all the code for every game I ever worked on. Then I have a separate repository for each game's assets. (actually, squeals share the same repository as the first game) This keeps the size of the repositories manageable, allows sharing of code between projects (which is VERY common) and doesn't allow sharing of assets between projects which is almost never a problem. This means programmers need to work with two repository at once which is a slight inconvenience but not a real problem. All the contents creators (artists, level designers, sound & music guys) deal with just one repository at time (unless they are working on multiple games at once).