June 01, 2005

Curious George: Too many files, can't burn a DVD. I'm doing data archival and there are 15,000 files in my directory. All of them must be put on the DVD. Nero Express's "Select Files" menu grinds to a halt because it can't populate the list with so many files. Is there a different DVD data authoring program for Windows that might be slick enough to not get bogged down by something silly like this? Does anyone make a command-line burner? Thanks, fellow monkeys.
posted by rolypolyman 18 years ago
  • Would it work to make some zip files and burn those to DVD?
  • Thought of that, but I frequently have to suck files from the disc (unzipping would take too much time), and one bad block on the disc would affect the whole ZIP archive (and require more work) rather than affecting just one little file that might never be used.
  • Download Cygwin and create an archive with tar. You get the benefit of zip (lots of files in one) but tar is smart enough not to go insane when files in the archive are borked. --Pat
  • How about using something like EasyISO to create a .iso file, then burning it to CD?
  • ...or DVD.
  • Just to give you all some idea of how much time it takes to add 15,000 files to a project, I tried out these programs. CDBurnerXP Pro 3 Freeware List the 15,000 files: 1.4 minutes Add the 15,000 files to project: Locked up (No feedback, I halted it at 10 minutes) DeepBurner Free List the 15,000 files: 2.0 minutes Add the 15,000 files to project: 10.1 minutes Time to prepare before burn: 6.0 minutes (Currently burning; we'll see if it works ok) zippy -- Thanks for the tidbit, though archives really won't work as any unzipper is a big time bottleneck.
  • and i thought I was obsessive about my porn collection...
  • Yeah, I'm hoping someone out there appreciates the slutz with nutz I'm packing in with their geophysical data.
  • #2 says: one of the problems may (at a guess) be that many programs don't scale when they have large numbers of files, due to inefficiencies in adding the next file on the list. The standard Windows tree view has this problem. You may be able to create it more easily from the command line. Also you may just not be able to fit 15,000 files on a DVD due to potential large cluster size.
  • 'Archival' is the adjective, btw - you want 'archiving'.
  • You can use mkisofs for Windows to make the ISO image to burn. I use GNU/Linux but it should work the same. Do this: mkisofs -r -J -udf -o IMAGE_NAME.ISO /PATH/TO/FILE/SET/ This will make an ISO image (change the name IMAGE_NAME.ISO to what ever you want) that will use "rational" Rockridge, Microsoft Joilet, and also a UDF file system that you should be able to burn to DVD using Nero. The UDF support in mkisofs is currently an overlay to the Joilet file system. The Rockridge support will allow most non-windows systems to read the disc, too. You can omit this (-r switch) if you only plan to use it with Windows machines. You might be able to add Macintosh HFS support if you need to, but I've never done that so I can't help you there. You can take a look at the manual page for mkisofs to get an idea of what is required if you need to change anything. Hopefully that will work for you.
  • MonkeyFilter: the slutz with nutz I'm packing in with their geophysical data.
  • Sorry if you've already tried this, but are you dumping all these files into the root directory of the DVD? If you are, try setting up one or more subdirectories in the root directory, and placing your files in that. I've used Nero to place substantially more than 15,000 files on a disk before; it should work.
  • Organize your files you crazy monkey! Any group of anything that's 100 objects or more can be efficiently organized. Though smaller chunkes of objects can be easily organized, it gets a bit too easy to get so restrictive you end up with as many organizational rules as you have files. Time, Date, Creator, Editor, Owner, Content, Type etc etc are all viable criteria. Create an organizational plan for your data and *stick to it*. Then you won't have 15,000 files that have to be individually dealt with, you'll have 1500 directories which is easier for humans and machines alike to parse. Also, you never gave us the specs of your machine. It could simply be that you're using an underpowered system or a filesystem that isn't appropriate for this type of use. Please share!