18 Feb 2007

Nebula3's Zip file system

Game applications often use file archives to reduce file system clutter and improve performance when many small files must be opened and read by the application. While Nebula2 used a proprietary archive format (NPK), Nebula3 uses standard Zip files. This has a number of advantages:
  • no self-written tools required to create the archives, just use the zipper of your choice
  • simple file encryption supported
  • smaller disc footprint
  • usually higher read performance because disc bandwidth is often the bottleneck, not decompression speed
The current implementation has a few disadvantages though:
  • no write support (not a big deal, NPK's didn't support writing either, and game resources are usually read-only anyway)
  • No random access (no seeks), this is a bit more critical, and could be solved with a more advanced implementation. Currently this is circumvented by decompressing the entire contents of a in-zip-file into memory and allow seeking on this in-memory-copy. This approach basically disables all types of file-streaming scenarios (especially streaming audio).
Accessing the content of Zip archives is completely transparent to the application once a Zip archive is mounted through the IO::Server::MountZipArchive() method. The IO::Server::CreateStream() method will check whether an URI actually is a file in one of the mounted Zip archives, and return a ZipFileStream object instead of a FileStream object if needed. The application just uses the returned Stream object as usual and doesn't need to care whether it's working with a "real" file, or a compressed file in a zip archive.


Eric said...

But isn't seeking in the archive a bad idea anyhow? I mean the project should know where the data is, always. Or am I overseeing something?

simple zip has another disadvantage: for those who know its a snap to unveil the package contents no matter how, but its much easier if its just a zip! But who cares.. borrowing stuff is no crime ;]

Floh said...

Seeking in this context means you have a virtual cursor into a file and move this cursor back and forth to define where data is written/read, this is necessary for some usage scenarios (streaming sound data from disk for instance).

The ZIP file can be encrypted by a password key (Winzip supports this for instance). This is a relatively weak encryption, but should be good and fast enough for general use.

Alex said...

At operation with zip files recommend to use modern tools,on example this application-repairing zip file with cyclic redundancy error,I finded it in google when I had been problems with zip file,utility is free as far as I know,it can extract needed document from corrupted archive and repair it,will successfully recover all documents,works faster and recovers more documents, than other services,program works under all supported versions of Microsoft Windows operating system,will mark all files for zip crc failed repair and zip archive crc repair,repair zip files crc error and you'd like to save recovered data,also possible to select your Zip file from the list of previously recovered archives.

Alex said...

My sister told me about her problem,and I advised-recover broken zip files.This tool helped me and friends too not once,also program is free as far as I know.It can recover corrupted files in Zip format and compatible with all supported versions of Windows operating system.

Alexis said...

I often join my zip files and once I did it and both files were deleted!I didn't know what to do.But I used next tool-best recovery zip software,which I found in one forum and tool solved my problem.Moreover it is free and can too extract data from corrupted files with *.zip extension.