nesdev.org

blargg wrote:I've finally got back to working on the NSF timing and NSFE generation tools. Sorry for the sudden absence from when it was being discussed over a month ago.

I've been meaning to PM you to ask you about it. Nice to know that you were still working on it.

blargg wrote:They are almost done. I'd like to get things fine-tuned before I release the source code. In particular, the M3U functionality needs to be evaluated. I used Duck Tales as an example and made an archive of the files at each step in the process, going from original NSF to timed and tagged NSFE to M3U:

What about the loop number option I've talked about earlier? Remember that there are songs with intros and then looped part, so simply doubling the detected time won't work.

Also the m3u file doesn't work with XMPlay+NSFPlug. Changing the "<?>" to "?" in track info entries fixes that, not sure if it's NSFPlug quirk or does the extended M3U format doesn't support < and > chars.

blargg wrote:As for Win32 builds, we'll need someone who can compile the C++ source code.

Define C++ source code. I have a MinGW and VC2005Express installed here (and might try adding Cygwin if those two wouldn't work for your code).

What's an example of where finer time resolution would make a difference? I could allow fractional seconds, as in 1:20.5 for a song that's 80.5 seconds. I've never understood the need for this, but would like to. NSFE times are in milliseconds, for reference.

What about the loop number option I've talked about earlier? Remember that there are songs with intros and then looped part, so simply doubling the detected time won't work.

Currently the only timing information for a track is a single duration. The idea is to make the track act like a normal music track that has a fixed length. I can't see how having data on the intro and loop length helps much, since you can always just tell the player to play the track longer if you want to listen to it more. Most players need built-in end-of-track silence detection anyway, so if you extend a non-endless track then the player will detect the silence and stop at the normal end anyway.

I'd like to stick to standard NSFE fields, and the only supported timing data I see is track duration and fade duration. This also causes anyone timing a track to produce an NSFE, which is a preferable format to an NSF+m3u (in my opinion). Supporting other data would be possible by adding a new block type to the NSFE, but I really doubt that people doing timing and tagging will bother to determine these times, due to their limited use. Again, I'm open to examples where these would really help.

Also the m3u file doesn't work with XMPlay+NSFPlug. Changing the "<?>" to "?" in track info entries fixes that, not sure if it's NSFPlug quirk or does the extended M3U format doesn't support < and > chars.

I don't have a Windows machine so I didn't have any way to test the M3U file; I just made it match the format you gave. I really hate those fields being filled in with "?" or any other crap. In my opinion, if the field is unknown, it should be BLANK! I have deal with this in my standalone game music player by replacing things like "?", "<?>", "< ? >" with "" (empty string). Err, back to your topic, I'll make the m3u string filter function easy to extend to cover any other illegal characters. Any suggestions on what to replace them with? Right now we've got these characters: ,<>

Define C++ source code. I have a MinGW and VC2005Express installed here (and might try adding Cygwin if those two wouldn't work for your code).

It's pretty plain code making fairly light use of C++. Most of the code (the NSF playback engine) has been compiled on many compilers and platforms already. As long as you know how to compile and build a program, I can fix any compilation errors that arise.

And just to recap, the idea of timensf is not to automatically time every track of every NSF properly, but to get enough of them right that the time spent filling in the gaps and fixing any bad times is much less than timing them all manually.

blargg wrote:I can't see how having data on the intro and loop length helps much, since you can always just tell the player to play the track longer if you want to listen to it more.

Well, timing for two loops seems to be a de facto VGM standard (see snesmusic.org or www.caitsith2.com/gsf. Most of PSF/USF files are also timed this way).
Consider this:

You start with making NSF+M3U, where the time fields are:

ending tracks - time (if no fade -> track is ending)
endless tracks - first_loop_time,second_loop_time,fade,number_of_loops

where fade and number_of_loops would be changable by command line arguments (with defaults being, say, fade=7 and number_of_loops=2).

Then you run a NSFE maker, parsing the M3U file, and set the track duration field to first_loop_time + second_loop_time*(number_of_loops-1) - of course if number_of_loops=0 or undefined, then don't substract and multiply.

No loss of timing information (as the m3u stays, and could be included in with NSFE), and you can easily retime both NSF and NSFE to your needs (changing the number_of_loops and fade would be quite easy given that the M3U is just a normal text file).

Thing is - if you can detect the first loop point automatically, the second one should be doable too. And detecting them manually is about as much work as timing the file from scratch.

blargg wrote:Err, back to your topic, I'll make the m3u string filter function easy to extend to cover any other illegal characters. Any suggestions on what to replace them with? Right now we've got these characters: ,<>

'*' seems to be a no-no too.
You might consider changing (C) string to '©' (ASCII $A9). Is handled fine here. I've been changing the ',' in author field with '&'.
You can change the <?> to blank field, or 'Unknown Author', no real preferences here...

blargg wrote:As long as you know how to compile and build a program, I can fix any compilation errors that arise.

Okay, there shouldn't (hopefully) be much problems given that it's a CLI program.

blargg wrote:What's an example of where finer time resolution would make a difference?

Whenever the actual length of a song does not coincide with the boundary of a second, so pretty much always.

blargg wrote: I could allow fractional seconds, as in 1:20.5 for a song that's 80.5 seconds. I've never understood the need for this, but would like to. NSFE times are in milliseconds, for reference.

The actual song length would be in frames (1/60 second), but if NSFe requires milliseconds, then convert the frames to milliseconds (factor 1000/60).

What I'm still lacking are the practical problems caused by the limitations. In other words, describe a user scenario where these limitations manifest as a problem:

1) Limiting timing to second accuracy
2) Having a single track duration without any loop points

As for using M3U as the intermediate text file, its format is not near as suitable for easy editing of tags:
It also needs extension to support a field for the copyright, ripper, and optional playlist. Why does an M3U repeat the game name, author, and copyright for every track? If we can improve the appearance of the M3U, I'd be more open to using it as the intermediate format. Say something like this:
Other tracks commented out so they won't play normally, but so tool can still add their timing information to NSFE.

blargg wrote:What I'm still lacking are the practical problems caused by the limitations. In other words, describe a user scenario where these limitations manifest as a problem:
2) Having a single track duration without any loop points

Well, as long as I *can* specify the amount of loops the timing tool will detect, I don't really care whether it's outputing single time or not (I don't even care for tags much, I will have my player in the background, so tags are just a bonus. It's times that matter).

I must have misunderstood you, because I thought you were saying the timing tool only detects time of the single loop. And now I see that the times for Duck Hunt are indeed two loops. Blah, lateness induced stupidity.

blargg wrote:What I'm still lacking are the practical problems caused by the limitations. In other words, describe a user scenario where these limitations manifest as a problem:
Limiting timing to second accuracy

What I'm still lacking is the practical problem caused by implementing the millisecond field as well. Oh well.
If you round down, you'll lose the end of the song. If you round up, there will be a blip of the first few frames of looping heard before moving on to the next song in the case of looped songs, which sounds stupid, or a variable amount of silence between songs in the case of non-looped songs, which also sounds stupid.

Knurek wrote:Well, as long as I *can* specify the amount of loops the timing tool will detect, I don't really care whether it's outputing single time or not (I don't even care for tags much, I will have my player in the background, so tags are just a bonus. It's times that matter).

I think I'm better understanding the big picture. My idea was to write some tools to help with timing and tagging NSF files to produce NSFE files suitable for submission to Disch's NSFE archive, so that everyone benefits from the work. Your idea seems to be a quick-and-dirty tool to privately time NSF files for personal use only, so no NSFE would ever come of it. It only seems worthwhile to write the tools if they help expand the public NSFE archive, and make that as easy as possible, so that many people can run the timing tool, do fixups, and add to the NSFE library.

I'm not opposed to using an M3U as the output of the timing tool, as long as it is easy to edit and can represent the basic NSFE information, including the optional playlist.

NewRisingSun wrote:What I'm still lacking is the practical problem caused by implementing the millisecond field as well.

Are you proposing to have all time fields in milliseconds (or even 1/60 seconds), or in both mm:ss and milliseconds, or perhaps mm:ss.msec? Only the latter seems practical, but I don't think the M3U file allows for that accuracy.

Propose a way to incorporate it into the toolchain. Maybe we should recruit some people who are willing to time and tag many NSF files using these tools, then ask them what format they're willing to enter times in. They'll be taking the NSF, running timensf on it, then fixing up any bad/unspecified times using a normal NSF player. Most NSF players I know only show seconds, so I don't know where they'd get this accuracy from. I really doubt any taggers will care to write the NSF to a wave file then use a sound editor to get timing points to 1/60 second accuracy.

I still can't imagine any significant audible effects of having only second accuracy. There are two basic track types: those that end, and those that are endless. For those that end, there should be some silence afterwards, so no problem there. For endless tracks, a player should fade them out, so a +/-0.5 second variance shouldn't make an audible difference either.

Timensf does work at 1/60 second accuracy and could preserve this in its output. It also knows the intro and loop times, so could output these if we decide to use an M3U variant as the intermediate format. Ideally it'd be nice to have timing at the native resolution, i.e. 1/60 seconds, but I just can't picture anyone bothering with this when they have to manually time a track.

If you round up, there will be a blip of the first few frames of looping heard before moving on to the next song in the case of looped songs, which sounds stupid,

It'd be just finishing fading out by then. On my player I use a 6-second exponential fade, so an extra 0.5 seconds of the loop won't be noticeable. If the player is not doing any fading out, then it already sounds stupid regardless.

or a variable amount of silence between songs in the case of non-looped songs, which also sounds stupid.

Wouldn't a player detect the silence at the end of the track and adjust the end-point on its own? It's got to do this for untagged tracks, so it might as well do it for tagged tracks too (only turning on silence detection a few seconds before the end, of course).

blargg wrote: I think I'm better understanding the big picture. My idea was to write some tools to help with timing and tagging NSF files to produce NSFE files suitable for submission to Disch's NSFE archive, so that everyone benefits from the work. Your idea seems to be a quick-and-dirty tool to privately time NSF files for personal use only, so no NSFE would ever come of it. It only seems worthwhile to write the tools if they help expand the public NSFE archive, and make that as easy as possible, so that many people can run the timing tool, do fixups, and add to the NSFE library.

I did write earlier (whether here or on the new rips thread), that I can't really use NSFE. Not that I'm opposed to the idea or something, but the current implementation leaves a bit to be desired (short story - until NSFPlug supports them, no NSFE for me, and that's not bloody likely).

Also, look at the facts - NSFE aren't all that popular. Disch site has what, 300 packs, half of which are Tsu Ryu's works. Compared to 1200+ game NSFs and 1200+ 2A03.org collections that's close to nothing.
Having your tool might speed up things a bit (though there's still the whole lotta work other than timing - I don't really see any givers for tagging an untranslated RPG/Mahjong game), but let's face it - it will be years (if ever) before it catches up.

I don't want to wait years to listen to Lagrange Point or Metal Slader Glory (I mean, listen with times and all). You can call me impatient, you can call me selfish, but that's just the way it is.

Personally I find the way HVSC does things a good one. Standard, untouched SID files, separate textfile with time information (automatically generated, but they're taking correction submissions), another textfile with tag information. I'd change the identification from path based to MD5 hash based, but there are sid plugins that do that already. Too bad nobody's gonna do a similar NSF collection anytime soon.

Also, look at the facts - NSFE aren't all that popular. Disch site has what, 300 packs, half of which are Tsu Ryu's works. Compared to 1200+ game NSFs and 1200+ 2A03.org collections that's close to nothing.

NSFE files aren't that numerous, but I think this is due to 1) NSFE being a much more recent file format than NSF, and 2) the only NSFE tagging tool (that I know of) requiring that one run Windows and use Winamp. One of my goals with these tools was to allow anyone to make NSFE files.

If this argument supports the position of avoiding the NSFE file format, then it applies equally to the M3U+NSF format (unless I'm missing something here).

Having your tool might speed up things a bit (though there's still the whole lotta work other than timing - I don't really see any givers for tagging an untranslated RPG/Mahjong game), but let's face it - it will be years (if ever) before it catches up.

Before my tools catch up with what? I don't follow. My understanding is that currently we have a large number of NSF files without any track names or timing information. We would like to have this information. This requires two things: people to add it, and a file format that captures it. The tools are to make timing and tagging take less time. Then we have two candidate file formats: NSFE or NSF+M3U. NSFE is well-documented, avoids having multiple files, and is supported on most platforms. M3U+NSF can be derived from an NSFE if necessary, though it has some fields not captured in the standard NSFE blocks.

I don't want to wait years to listen to Lagrange Point or Metal Slader Glory (I mean, listen with times and all). You can call me impatient, you can call me selfish, but that's just the way it is.

What's keeping you from it? Is it only available in NSFE format or something? That was one point of having the NSFE-to-NSF+M3U tool.

Personally I find the way HVSC does things a good one. Standard, untouched SID files, separate textfile with time information (automatically generated, but they're taking correction submissions), another textfile with tag information.

Why not use an updated SID file format that can hold all this information in one file? Multiple files creates more of a mess for the end-user, having to keep them together.

I'd change the identification from path based to MD5 hash based, but there are sid plugins that do that already. Too bad nobody's gonna do a similar NSF collection anytime soon.

Hmmm, this is a different approach. I take it this has a central database server that people submit timing information to, and then it makes use of the original untouched SID rips that people already have, obtained from elsewhere?

It looks like everyone has their own idea of how NSF timing and tagging should be done. Apparently this discussion needs to be taken to another site of more general interest, so users, NSF player authors, and taggers can all discuss different approaches, and the benefits and drawbacks of each.

blargg wrote:Are you proposing to have all time fields in milliseconds (or even 1/60 seconds), or in both mm:ss and milliseconds, or perhaps mm:ss.msec? Only the latter seems practical, but I don't think the M3U file allows for that accuracy.

In the text file, just add the milliseconds, so instead of 01:33, you write 01:33.667. The milliseconds will be truncated for the M3U file, but since the .NSFe file is created from the .txt file and not the .m3u file (if I understand the process correctly), this doesn't matter.

It is possible to play tracks that normally loop directly one after another without fading IF the song lengths are set precisely, then it will not sound stupid. DeliPlayer does this well for many Amiga sound sets, and I think it sounds quite nice.
Also, it's good to have the actual exact song length for sheer properness, just as it is proper to emulate the NES' waveform at 42 MHz, even though directly creating the filtered signal at 14 MHz produces the same result, as far as I have observed.

You are correct that most people won't open a sound editor to get that accuracy, but with timensf, it should be possible. Then again, most people wouldn't spend the time to rip an .NSF either, and yet some do

blargg wrote:NSFE files aren't that numerous, but I think this is due to 1) NSFE being a much more recent file format than NSF, and 2) the only NSFE tagging tool (that I know of) requiring that one run Windows and use Winamp. One of my goals with these tools was to allow anyone to make NSFE files.

Another reason is that there's just one site (to my knowledge) that offers NSFe files. If you can get Zophar's and PlanetEmu to host NSFe files, describing them as the preferred format, I'm sure the demand would increase as well.

Sorry for the necrobump, but since p1xl released FlashNSF, I would be very much interested in this tool, but it is unfortunately not online anymore. blargg, would you happen to still have it?

Thank you!