Tag Archives: Subversion

Torchlight 2 Toon Archiving: The Sequel

The kids left me alone last night, so I decided I’d goof around with my Torchlight  2 character archiving.  Recent reading around Git has convinced me that Git stores deltas of its binary file commits, so my last objection to using Git over Subversion for binary files has just been laid to rest.

Also, the original Perl script was a bit brain-dead in that it was good at automatically committing save files that had changed, but additions and deletions were still things I’d have to manually tell the repository about.

As these things tend to go, the lion’s share of the script was written in maybe the first 15 minutes.  The rest of the night was spent tweaking and testing the command template constants, destroying and re-creating Git archives until I had myself convinced that the script was automatically committing exact replicas of the save-game directory, regardless of additions, deletions and modifications.

The script turned out to be pretty short, and is included at the end of this blog post in all its perlesque glory.  Now that I have the trick of it though, the pattern is pretty much applicable to any save-game directory I might want to subject to version control.  The human-readable formula is basically:

  1. Delete all contents of the directory within your version control repository that contains the copied image of your save-game directory.
  2. Fully (recursively) copy the contents of that save-game directory into  the recently emptied archive directory.
  3. Ask the repository to identify all files that have been deleted from the  copied file set that it is currently archiving.  If there are any deleted files, stage these files to also be deleted in the repository on the next commit.
  4. Ask the repository to identify all new files that it is currently not archiving. If there are any new files, stage these  files to be added into the repository on the next commit.
  5. Have the repository apply in a single commit, all deletions, additions and modifications identified.  
#!/usr/bin/perl

# An archive utility for Torchlight 2 characters. Works by commiting the current
# save-game contents to a GIT repository housed in the parent directory to the script.
#
# (c) 2013, Lindsay Bradford, released under the Creative Commons Attribution licence.
# http://creativecommons.org/licenses/by/3.0/
#
# The parent directory has two subdirectories, being "bin", and "toons".
#  The "bin" directory contains this script.
#  The "toons" directory contains the save-game content of the game.
#
# Usage:
#    archiveTLToons.pl <"optional commit message string"">
#
# Modify the constants below to suit your own environment

package ArchiveTL2Toons;

use strict;

###### constants for directory locations and command templates below #####

use constant GAME_SAVE_DIR =>
	"/home/linds/.wine/drive_c/users/linds/My Documents/My Games/Runic Games/Torchlight 2/save/76561198044661040/.";

use constant ARCHIVE_TOON_DIR =>
	"../toons/";

use constant CLEAR_ARCHIVE_COMMAND =>
	sprintf "rm -rf '%s'", ARCHIVE_TOON_DIR;

use constant COPY_COMMAND =>
	sprintf "cp -r '%s' '%s'", GAME_SAVE_DIR, ARCHIVE_TOON_DIR;

use constant REMOVE_MISSING_ARCHIVE_COMMAND =>
	sprintf "git ls-files --deleted \"%s\" | xargs -r git rm --quiet", ARCHIVE_TOON_DIR;

use constant ADD_UNTRACKED_ARCHIVE_COMMAND =>
	sprintf "git add --all \"%s\"", ARCHIVE_TOON_DIR;

use constant COMMIT_ARCHIVE_COMMAND => "git commit --quiet --m \"%s\"";

##### Methods below #####

# Bootstrap method.

&archiveTL2toons($ARGV[0]);

# Archives the current set of Torchlight 2 toons by
# deleting the archive contents, taking a recursive
# copy of the save directory back into the directory
# and commiting a snapshot of the copied content.

sub archiveTL2toons() {
  my $commandLineComment = $_[0];

  if ($commandLineComment eq "") {
  	$commandLineComment = "Commit of current save state.";
  }

  &runCommand(
    "Clearing archive content...",
    CLEAR_ARCHIVE_COMMAND
  );

  &runCommand(
    "Copying Torchlight 2 save game content to archive...",
    COPY_COMMAND
  );

  &snapshotArchive($commandLineComment);
}

# Commits a snapshot of the current content
# of the archive, assuming that the current content
# is exactly what the commit should contain.  Specifically:
#   * Any files missing  from the archive are deleted in the commit
#   * Any new files found are automatically added with the commit
#   * All modified files are commited as-is.

sub snapshotArchive() {
 my $commandLineComment = $_[0];

  &runCommand(
    "Staging removal of missing files from archive...",
    REMOVE_MISSING_ARCHIVE_COMMAND
  );

  &runCommand(
   "Staging addition of untracked new files to archive...",
    ADD_UNTRACKED_ARCHIVE_COMMAND
  );

  my $message = &getNowTimestamp . " | " . $commandLineComment;

  my $commitCommand = sprintf COMMIT_ARCHIVE_COMMAND, $message;

  &runCommand(
    "Commiting staged snapshot of save-directory to archive...",
    $commitCommand
  );
}

# Generates a timestamp of the current time.

sub getNowTimestamp() {
 my ($sec, $min, $hr, $day, $mon, $year) = localtime;
 return sprintf("%04d-%02d-%02d %02d:%02d",
       1900 + $year, $mon + 1, $day, $hr, $min);
}

# Simple method that prints the $message supplied,
# runs the $command specified, and prints any results
# the command generates.

sub runCommand() {
  my ($message, $command) = @_;

  print "$message\n";

  my $result = `$command`;
  print $result;
}

On a final note, I installed EPIC for Eclipse to modify the script, so despite my intense dislike for the lack of automated refactoring, I’m begrudgingly having to admit that it’s working better for me than doing it in a text editor.

Advertisements

Engineering Archivable Torchlight 2 Characters

[Update: It’s been less than 24 hours since this post, and Runic have release patch 1.16. The patch allows me to play VonMalefic again (joy). Looks like it was more the game choking on a legitimate game file, rather than the file corruption I was afraid of.]

I’m in a painful place since the weekend, where patch 1.14 of Torchlight 2 caused a night of game crashes with my main character. Patch 1.15 came down the wire the next morning and bam, I can no longer even load the character without the game crashing.

Seem that this is a known issue on the forums (and here too). The toon has a full set of Mondon armour and Twitch (a legendary greatsword) that I spent quite a bit of time trading bits for to eventually complete.

Here is the toon VonMalefic, in a Steam screenshot I took jJust after the last piece of Mondon kit was added:

VonMalefic in better days

VonMalefic in better days

Now, I’m not pleased about game save-file corruption at any time, but at the peak of my obsession with a new computer game, it’s doubly painful.  It’s a software issue, and what does any self-respecting programmer do when he runs into a software issue?  They get all Bob-the-Builder on the issue!

Now, Steam is a great environment for ensuring off-site backups, but if a file gets corrupted, now I have an off-site backup of a corrupt file.  What I need is an archive of toon history that I can roll back to a “known-good point” with.  What’s the difference between a backup and an archive?  Well you see…. ahh screw it, here’s something that does a better job of nailing the difference between backups and archives better than I care to in this post.

My threadbare archiving solution?  I set up a Subversion repository on the machine, and write a very basic Perl script to automatically copy save-file games into the repository and take a snapshot.  My heart belongs to Git nowadays for file repository management, but last I checked it was still crap at binary files, so it’s Subversion all the way.

And finally, that threadbare script (I’m emulating the game using Wine under Ubuntu, which is why the directories look a little odd to Windows users):

#!/usr/bin/local/perl

package ArchiveTL2Toons;

use strict;

$ArchiveTLToons::SaveFileDir = "/home/linds/.wine/drive_c/users/linds/My Documents/My Games/Runic Games/Torchlight 2/save";
$ArchiveTLToons::ArchiveDir = "/home/linds/TL2ToonArchive/WorkingCopy";

sub copySaveFilesToArchive() {
  `cp -r \"$ArchiveTLToons::SaveFileDir\" $ArchiveTLToons::ArchiveDir`;
}

sub getNowTimestamp() {
 my ($sec, $min, $hr, $day, $mon, $year) = localtime;
 return sprintf("%04d-%02d-%02d %02d:%02d\n", 
       1900 + $year, $mon + 1, $day, $hr, $min);
}

sub snapshotArchive() {
 my $timestamp = &getNowTimestamp();
 my $cmd = "svn ci $ArchiveTLToons::ArchiveDir -m \"Commit of changes @ $timestamp\""; 
  #print $cmd;
  `$cmd`;
}

sub archiveTL2toons() {
  &copySaveFilesToArchive();
  &snapshotArchive();
}

&archiveTL2toons();

The script currently only handles character file changes easily. Any time I add or delete characters, I currently need to manually tell Subversion about the additions and deletions. Still, right now it gets me a very basic archiving solution, allowing me to recover from save-file corruption in the future.

On a final note, I’m conflicted on what to do now. I’ve lodged a support ticket with Runic in the (remote) hope that the save-file is somehow salvageable. I’ve resigned myself to that path being unlikely, so I’ve re-rolled Von. The replacement toon (Dawnshammer) will be more thoroughly Min-Maxed than Von, who bares the scars of my n00bishness with the game.

Still, Von wasn’t too bad all things considered, and entirely fit for the end-game grind. I’d rather have Von back over re-investing all that development/trading time with Dawnshammer. Fingers crossed Runic deliver some magic on this one.

To subversion, or not to subversion, that is the question

My current job (which I’m loving to bits) involves looking after a piece of software that started life in VB6 with no source management to be seen. The primary author, and my boss, is not hip-to-Subversion. This presents a problem for me as Subversion is my poison of choice for supplying the metaphorical “safety net” when I tightrope walk out into unexplored software change territory.

Right now, we’re trying to find a sweet-spot between version control, and the free-form copy model that he’s been using to date. We’re not there yet, but today’s blog entry captures yesterday’s win on using TortiseSVN to give me a quick rundown on the differences between two branches in a repository. One with revision history, one without.

First off, I built a sandpit repository, and dropped both versions of the codebase in as separate branches. The trunk branch was taken from the “blessed” repository, with complete revision history. The other branch is my bosses copy, with no revision history available. Once the branches were loaded, I pulled the repository browser for my sandpit repository:

I then marked the trunk branch for comparison, as below:

Next, I compared the trunk branch URL against the branch I want to merge against:

Finally, TortiseSVN spits out a complete list of differences between both branches. At this point I can step into each file with TortiseMerge and get a feel for what I’m facing before I do the merge proper.

As a closing note, the actual automated merge failed miserably via TortiseSVN. I have a vague memory of walking a similar path to this before. Back then I gave up on the GUI, and went back to the SVN command-line for merges to give me that extra control over what I needed to do. One of the complicating factors this time around is that this strikes me as being similar to dealing with vendor branches, except that the entire codebase could be considered a vendor branch.