Hello all,

As I have mentioned before, I have my entire (rather large) CD collection as mp3 on a linux server. I am trying to create a list of the contents in the order in which they were uploaded to the server from my ripping machine. As an aside, I'm doing this because I know that files in a certain date range have only version 1 id3 tags, while others have the date screwed up in the tag, etc.

ls -lag * gives me the correct output, which I redirect into a text file. Here's the problem:

How to actually get that list into some kind of delimited file with real date fields? Here's what the data looks like:

Allman Brothers Band:
total 240
drwxrwxr-x 25 501 501 4096 Feb 18 18:45 .
drwxrwxr-x 1010 501 501 24576 Feb 18 19:03 ..
drwxrwxr-x 2 501 501 4096 Jul 3 2001 A Decade of Hits 1969-1979
.
. (etc) more entries...
.
drwxrwxr-x 2 502 501 4096 Jan 20 2002 Eat A Peach
drwxrwxr-x 2 502 501 4096 Feb 18 18:44 Enlightened Rogues
drwxrwxr-x 2 501 501 4096 Aug 15 2001 Fillmore Concerts Disc 1
drwxrwxr-x 2 501 501 4096 Aug 15 2001 Fillmore Concerts Disc 2
drwxrwxr-x 2 502 501 4096 Feb 18 18:44 Hell & High Water
drwxrwxr-x 2 501 501 4096 Aug 15 2001 Live At The Ludlow Garage 1970 Disc 2
.
.
.
drwxrwxr-x 2 502 501 4096 Feb 18 18:45 Wipe the Windows, Check the Oil

Altan:
total 96
drwxrwxr-x 7 501 501 4096 Jan 17 2002 .
drwxrwxr-x 1010 501 501 24576 Feb 18 19:03 ..

You see, the date is actually 3 fixed-width fields, but if the date is this year, it puts the time instead of the date. The artist (parent directory) appears above with a ":" character at the end. There is a blank line before each new artist. The output I want is:

Artist, Album, Date

Where the Artist comes from the line above the contents the Album is the subdirectory name, and the Date is a usable reformatting of the 3 fixed width date fields. All of the other stuff, the blank lines and the . and .. lines need to be stripped.

By changing the system date on the machine, to 2004 for instance, I can eliminate the time vs. year problem, but I'm still having problems parsing this file.

Times like these I wish I knew perl. Can anyone rip this out off the top of their head?

Oh yeah, one last thing that helps is that the number right after the permisssions in the "." line contains the number of entries in that Artist subdirectory + 2 (+2 because . and .. both count as entries).

Thanks in advance,

Jim

Well one thing you can do is pass the --full-time switch to your ls command, that will make the dates easier to deal with.

The better way would be to write a perl script that would deal with the files' metadata directly instead of trying to parse ls's output. If you're stuck with that, though, you can use ls's --full-time option, if you're using GNU ls.

Then you can use cut and/or awk and date -d to get it rectified, if needed.

Assuming your directory structure is consistently Artist/Album, then this ought to work:

#!/usr/bin/ksh

for artist in *; do

cd $artist

for album in *; do

echo -e "$artist $album\c"

ls --full-time -d $album | cut -c 44-67

done

cd ..

done

Well, the -T option in linux gives the full time, so that's a help. What I need now is to get the parent directory name to appear for each entry, or figure out some way to do that. This is closer, though. I didn't even think that there would be a "full time" option on ls...

Jim

Whoa. Thanks, Bitt! I'll give that a shot. I appreciate the help! I'll let you know how it works...

Thanks again,

Jim

I'm not sure if it's what you're looking for, but

ls --full-time */* | sed 's#/# #'

will also get you pretty close.

OK, no ksh on the system. Grrrr. I understand what you're doing, though, and I'll rewrite it in bash or csh. Or, I'll give your alternate version a try.

Thanks again,

Jim

Awright, the problem with the script is that the directory names have spaces in them so it tries to list each word separately...

$ find . -type f -print0 | xargs -0 ls --full-time

And to do the whole thing...

$ find . -type f -print0 | xargs -0 ls --full-time | cut -c44- | sort

Won't that sort command sort them by day of the week instead of the date?

$ find . -type f -print0 | xargs -0 ls --full-time

[puts on stripy shirt, blows whistle, gesticulates]

Unix foul. Using two commands where one would do. Five-yard penalty. First down.

$ find . -type f -a -printf "%T@ %Td-%Tb-%TY %p\n" | sort -g | cut -f2- -d' '

Peter

Edit: should be -printf, not printf

OK, I am willing to admit it. You guys have a bigger UNIX than me. :-)

Seriously, thanks for the help. Only problem is, none of these things work.

First, the -f on find finds the files, not the "Album" subdirectories. Changing it to -d finds all the directories, not just the Album ones (subdirectories), and the nice single command option doesn't work at all.

I'd really like to get this working. I'm not good at the regular expression stuff and I appreciate all of your help.

I have two systems that I can access the volume from. One is a RedHat linux machine (older, 6.2, I think, with a new kernel) and the other is an OpenBSD 3.0 machine. So, those are the commands I can use, though I can (and would) add whatever command(s) I need to get this to work.

ls on these machines has a -T option that gives full time.

The nice printf solution doesn't work, as far as I can tell, for two reasons: 1. I can't get find to output to printf without using a pipe (or at least, I don't know how on my system -- there is no -a option), and 2. The formatting doesn't work:

find . -type f | printf "%T@ %Td-%Tb-%TY %p\n" | more
-csh: illegal format character

The original:
find . -type f -a printf "%T@ %Td-%Tb-%Ty &p\n" | more
find: printf: unknown option

Comments?

Thanks again,

Jim

find . -type f -a printf "%T@ %Td-%Tb-%Ty &p\n" | more
find: printf: unknown option

Oops. That should be -printf, with a dash. I'll edit the post. And surely you're interested in fixing the files that were created at a certain time, not the directories?

And in fact you probably should change the %T's to %C's in order to get ctime instead of mtime.

Peter

He meant for there to be a `-' before the printf:

find . -type f -a -printf "%T@ %Td-%Tb-%Ty &p\n"

Well, yes, but I ripped them as entire CDs, so I want a list of CDs that I need to take another look at. Then I'll fix up all the files, or potentailly re-rip them, depending upon which category they are in.

JC

You know, this many flavors of unix thing sucks. -printf is not an option on my find. There is -print and -print0 but no -printf

"No problem", says me, "that's what pipes are for":

find . -type f -print | printf "%T@ %Td-%Tb-%Ty &p\n"
-csh: illegal format character

Standards are good things. Does this suck horribly, or is it just me? Am I back to redoing Bitt's script in csh? Am I a stupid ass for not being able to figure this out? (sure feels like it)...

Damnation. I think my printf sucks, too. man printf gives these arguments:

Format:
A character which indicates the type of format to use (one of
diouxXfEgGbcs).

A field width or precision may be `*' instead of a digit string. In this
case an argument supplies the field width or precision.

The format characters and their meanings are:

diouXx The argument is printed as a signed decimal (d or i), un-
signed octal, unsigned decimal, or unsigned hexadecimal (x or
X), respectively.

f The argument is printed in the style [-]ddd.ddd where the
number of d's after the decimal point is equal to the preci-
sion specification for the argument. If the precision is
missing, 6 digits are given; if the precision is explicitly
0, no digits and no decimal point are printed.

eE The argument is printed in the style [-]d.ddde+-dd where
there is one digit before the decimal point and the number
after is equal to the precision specification for the argu-
ment; when the precision is missing, 6 digits are produced.
An upper-case `E' is used for an E format.

gG The argument is printed in style f or in style e (E) whichev-
er gives full precision in minimum space.

b Characters from the string argument are printed with back-
slash-escape sequences expanded.

c The first character of argument is printed.

s Characters from the string argument are printed until the end
is reached or until the number of characters indicated by the
precision specification is reached; however if the precision
is 0 or missing, all characters in the string are printed.

% Print a `%'; no argument is used.

Standards are good things.

And the best thing about standards is that there are so many of them to choose from.

You know, this many flavors of unix thing sucks. -printf is not an option on my find. There is -print and -print0 but no -printf

Your find sucks. That option is in GNU find 4.1 of November 1994. Try it on the Redhat box? What does "find --version" say?

The standalone printf command doesn't stand a chance of doing the right thing as it doesn't have all the information that find does.

Peter

Screw it.

#!/usr/bin/perl



opendir(TOP, ".");



foreach $dir (readdir(TOP)) {

        next if $dir eq ".";

        next if $dir eq "..";

        if ( -d $dir ) {

                opendir(SUB, $dir);

                foreach $subdir (readdir(SUB)) {

                        next if $subdir eq ".";

                        next if $subdir eq "..";

                        if ( -d "$dir/$subdir" ) {

                                $mtime = (stat("$dir/$subdir"))[9];

                                print "$dir $subdir " . localtime($mtime) . "\n";

                        }

                }

                closedir(SUB);

        }

}



closedir(TOP);

Edit: cleaned up getting the mtime

Thank you again! I really appreciate the help. I do realize that you all have other stuff going on and I really appreciate the time you all have spent. Thank you.

The perl script just about does it. For some reason, the date is goofed up. All the directories show between Mar 2 and today and some time that I can't figure out. I don't really know what date it could be, actually, not the last accessed date... Not the creation time of the file, at any rate.

That bit should be possible for me to figure out, however. I'm also going to change it to tab-delimited or fixed-width so I can use it more easily.

This is a huge help, Bitt.

Thanks again!

Jim

As to formatting, read the perlform man page.

As to times on directories, this is one of the minor pieces of Unix arcana.

There are three times associated with every Unix file (or directory, since directories are actually files, too). There's the ctime, mtime, and atime. atime is access time. It's the last time that someone looked at the data in the file. mtime is the modification time. It's the last time someone changed the data in the file. ctime is change time, not! creation time. It's the time that the file's metadata was changed. This is stuff like file creation, obviously, chown, chmod, etc. That last one trips people up. There's no dedicated creation timestamp in Unix.

But then you have to understand how directories are modified. When a new file is placed in a directory or is removed, all of the directory's times change. If any of a file's times change, the directory's atime changes. I think that's it.

I don't know exactly which time you're looking for.

Looked up "stat" in my perl book (really, I'm going to learn perl, honest!), and we just needed another undef in the argument list for stat. We were getting atime instead of mtime.

Lovely! Awesome!

I need to fix the formatting of the date because I still have that fixed-width issue with the day field. Or, I need to sort the output on date. Those things should be simple enough for me to figure out. I'm off and running.

Thanks again everyone!

Jim

Take a look at the updated script above. It's a little easier to read with respect to the stat output (array reference vs. all those undefs).

Also, if you're going to sort, I'd put all the info in an array (of arrays, probably), and then sort on the time. But make sure you don't localtime() it until you print out. It'll be much easier to sort on the raw timestamp (which is just the number of seconds past Unix epoch, as usual) than to try to do it on the munged output-style data.

Eh, what the hell:

#!/usr/bin/perl



opendir(TOP, ".");



$i = 0;

foreach $dir (readdir(TOP)) {

        next if $dir eq ".";

        next if $dir eq "..";

        if ( -d $dir ) {

                opendir(SUB, $dir);

                foreach $subdir (readdir(SUB)) {

                        next if $subdir eq ".";

                        next if $subdir eq "..";

                        if ( -d "$dir/$subdir" ) {

                                $mtime = (stat("$dir/$subdir"))[9];

                                $list{"$dir/$subdir"} = $mtime;

                        }

                }

                closedir(SUB);

        }

}



closedir(TOP);



format =

@<<<<<<<<<<<<<<<<<<<<<<<<< @<<<<<<<<<<<<<<<<<<<<<<<<<<< @<<<<<<<<<<<<<<<<<<<<<<<

$artist,                   $album,                      $time

.



foreach $sdir (sort { $list{$a} <=> $list{$b} } keys %list) {

        ($artist, $album) = split(/\//, $sdir);

        $time = localtime($list{$sdir});

        write;

}

It's attached, too.

It worked! I have what I need.

You're the greatest. Thanks a ton, Bitt. Thanks also to all of you who tried to help me get this done from the UNIX prompt. I was able to make some changes to the perl script and get the output format that I wanted.

Anyhow, thanks again,

Jim

I think that is some of the easiest reading perl that I've ever seen. You're obviously not a real(tm) perl coder!

Even

foreach $sdir (sort { $list{$a} <=> $list{$b} } keys %list)

?

And you've gotta wonder what that ``$i = 0'' is in there for. (Oops.)

Because every program needs an "i" loop counter, whether it's used or not.

I never said that I actually had parsed every line and performed a code review, but simply inferred that it would not be a difficult job

It sure made it easier for me to modify to do what I needed!!!

JC