Loading Zipped ROMs

[MottZilla]

I searched and there was no topic about this. What are some ways to go about loading Zipped ROM files? I've know about ZLIB however it doesn't actually handle ZIP archives. So can anyone recommend a relatively easy way to pass some library a file path and get it to decompress the ROM inside the zip to some free memory area for me?

[tepples]

I searched and there was no topic about this. What are some ways to go about loading Zipped ROM files? I've know about ZLIB however it doesn't actually handle ZIP archives.

I seem to remember that Info-ZIP UnZip can be built as a library.

So can anyone recommend a relatively easy way to pass some library a file path and get it to decompress the ROM inside the zip to some free memory area for me?

Even if you can't get Info-ZIP UnZip to work as a library, you could still run Info-ZIP fUnZip through a pipe, using popen() or your platform's equivalent.

[Near]

If you're okay with the zlib license, Nach made some modifications to zlib and let me use it for my emu ... it extracts ZIP files. You're more than welcome to use it. Extractor looks like this:

Code: Select all

uint8_t* ZipReader::read(unsigned length) {
  uint8_t *data = 0;

  if(!filesize) return 0;

  if(length <= filesize) {
    //read the entire file into RAM
    data = new(zeromemory) uint8_t[filesize];
    unzReadCurrentFile(zipfile, data, filesize);
  } else if(length > filesize) {
    //read the entire file into RAM, pad the rest with 0x00s
    data = new(zeromemory) uint8_t[length];
    unzReadCurrentFile(zipfile, data, filesize);
  }

  return data;
}

ZipReader::ZipReader(const char *fn) : filesize(0), zipready(false) {
  unz_file_info cFileInfo; //Create variable to hold info for a compressed file
  char cFileName[sizeof(cname)];

  if(zipfile = unzOpen(fn)) { //Open zip file
    for(int cFile = unzGoToFirstFile(zipfile); cFile == UNZ_OK; cFile = unzGoToNextFile(zipfile)) {
      //Gets info on current file, and places it in cFileInfo
      unzGetCurrentFileInfo(zipfile, &cFileInfo, cFileName, sizeof(cname), 0, 0, 0, 0);

      if((cFileInfo.uncompressed_size <= MAXROM+512) && (cFileInfo.uncompressed_size > filesize)) {
        strcpy(cname, cFileName);
        filesize = cFileInfo.uncompressed_size;
      }
    }

    if(filesize) {
      unzLocateFile(zipfile, cname, 1);
      unzOpenCurrentFile(zipfile);
      zipready = true;
    }
  }
}

ZipReader::~ZipReader() {
  if(zipfile) {
    unzCloseCurrentFile(zipfile);
    unzClose(zipfile);
  }
}

The zlib library itself is all in pure C89, and you could thus write your interface to it the same way, but it's pretty bulky at ~513kb.

[blargg]

Take a look at my not-officially-released File_Extractor. It supports ZIP, GZ, 7Z, and RAR. If you want just ZIP support, the rest can be removed completely. One thing, it only supports "deflate" and "none" as compression methods for ZIP, which are the most commonly used compression types.

Code: Select all

int main()
{
    fex_err_t err;
    File_Extractor* fex = fex_open( "test.zip", &err );
    if ( !fex )
        error( err );
    
    while ( !fex_done( fex ) )
    {
        const char* name = fex_name( fex );
        long size = fex_size( fex );
        if ( is_name_we_want( name ) )
        {
            /* Either have FEX read data into memory and give us a
            pointer to it... */
            const unsigned char* data = fex_data( fex, &err );
            if ( !data )
                error( err );
            
            /* ...or read it ourselves, in however many chunks we like. */
            error( fex_read( fex, my_buffer1, first_size ) );
            error( fex_read( fex, my_buffer2, second_size ) );
            /* ... */
        }
        error( fex_next( fex ) );
    }
    fex_close( fex );
    return 0;
}

[Near]

Wow, that's really cool. The 7-zip support is really tempting. I love the simple API to it, too.

Does your library support automatic UTF-8 -> UTF-16 conversion for Windows? If not, there's no way to open Unicode file names; unless of course you also have a wchar_t interface (yuck.)

Too bad JMA is GPL, and RAR is ... anti-competitive? Getting those both in there as LGPL would make this one killer all-purpose library for emulators.

I wonder if it's possible to clean-room unrar and implement that as pure LGPL ... doubt he has a patent on the algorithm.

[Dwedit]

Oh noes! It doesn't support those really old "Exploding..." ZIP files!

[MottZilla]

I'll try out your File Extractor blargg. I see no reason why I'd need to remove GZ,7Z, or RAR support, even if I'm just going to use Zip.

[Near]

Well, using RAR means you can't release your code as GPL (or let others do so), or make a WinRAR competitor. If you're not planning on doing either, yeah, may as well keep it in there.

[tepples]

The GNU GPL's copyleft restriction generally stops at the boundary between one process and another. If you keep anything that touches a RAR file in a separate process that communicates with the main process through a pipe (like fUnZip does), and you can keep the rest of the program pure GPL.

[blargg]

Does your library support automatic UTF-8 -> UTF-16 conversion for Windows? If not, there's no way to open Unicode file names; unless of course you also have a wchar_t interface (yuck.)

It supports reading archives from anything, using an abstract base class, so one could provide a custom Windows file reader.

[MottZilla]

Thanks alot Blargg. My emulator now has zip loading support and it only took a few minutes. It was extremely easy to add using your File_Extractor package. =) I disabled 7zip (and never enabled RAR) because I was getting a linking error. But anyway it loads zipped ROMs no problem now so I'm happy.

[Near]

The GNU GPL's copyleft restriction generally stops at the boundary between one process and another. If you keep anything that touches a RAR file in a separate process that communicates with the main process through a pipe (like fUnZip does), and you can keep the rest of the program pure GPL.

Yeah, the file-roller approach would work. Just tacky to have a bunch of executables like that.

It supports reading archives from anything, using an abstract base class, so one could provide a custom Windows file reader.

Awesome. It figures you'd have thought of everything ;)

[polaco]

Cool, I was looking for this

Thanks blargg

[tepples]

Yeah, the file-roller approach would work. Just tacky to have a bunch of executables like that.

It's not tacky to have a lot of EXEs. Look at how many EXEs come with CC65. It's no worse than the dozen DLLs that some apps ship with, except a defect in an EXE is less likely to make an unrelated part of a program crash.