Time Zone Setting

Posted by: tonyc

Time Zone Setting - 19/02/2004 20:10

As documented in this thread, many people have had difficulties getting apps other than the player to display local time properly. Regardless of the TZ setting, "date" always returns UTC. I tried copying a bunch of files from /usr/share/zoneinfo on another system, but this didn't help the situation (although the files I copied into there became available in the player app's Timezone menu, strangely.) I also tried creating a symlink /etc/localtime which links to my timezone, but still, 'date' returns UTC. It seems that there's some piece of configuration that nobody's found yet that needs to be in place for localtime() to perform time zone correction. Anyone have any ideas?

Also, from what I understand, the player uses UTC internally, but for display purposes, offers the "Timezone" setting. I don't know where that setting is saved, but my initial peeking in the flash savearea tells me it doesn't live there, so I'm guessing it's somewhere on hda3. Can any of the empeg gods enlighten us on exactly where it lives, or at least on what sector? If I can go find that, I won't need to worry about the TZ variable or anything else.
Posted by: mlord

Re: Time Zone Setting - 19/02/2004 21:12

Mmm.. It appears to be stored in bytes 0x51 - 0x53 of the flash savearea. Not sure what the value there actually means yet..
Posted by: wfaulk

Re: Time Zone Setting - 19/02/2004 21:31

I'm not exactly sure what you mean by ``the TZ setting''. I don't know if you mean within the OS or the player option. Under most, if not all, Unices, the OS's timezone setting is an environment variable set by init before it spawns any other processes, which inherit the variable. The empeg's init does not do this. You should see a difference if you set the environment variable within a shell and then run date. (Remember to export it!) I assume that the variable is TZ, but I could be wrong. There's also the possibility that the empeg's libc doesn't support timezone distinctions, but I doubt it. It's also possible that while you've obviously found the correct directory in which to place the config files for the player that the correct directory for the OS might be elsewhere.
Posted by: mlord

Re: Time Zone Setting - 19/02/2004 22:00

Okay, there are four bytes for timezone info, in the flash savearea at offsets 0x51 0x52 0x53 0x54. These hold subdirectory indexes (alphabetic order assumed) for the timezone info under /usr/share/zoneinfo/

I have mine set to Canada -> Eastern, so the flash savearea shows 1C-04-00-00, which means "the 28th entry" (Canada), and then "the 4th entry" (Eastern). No more subentries for me, but in other parts of the world there may be.

Cheers
Posted by: mlord

Re: Time Zone Setting - 19/02/2004 22:10

Futher to this, /usr/share/localtime is a symlink which normally should point at the appropriate zone file from one of the subdirectories. On the empeg, it points at /etc/localtime.

In this case, /etc/localtime *should* be a symlink pointing back to a file under /usr/share/zoneinfo/..., but instead it simply does not exist. All of the "TZ" operations depend upon this existing, but since it does not exist, no timezone processing is done by the C-library.

I think that perhaps I should have Hijack detect accesses to /etc/localtime, and automatically redirect those back to the correct zoneinfo file, based on the path traversal info from the 4 bytes of the flash savearea. Once this is working, timezone info should be automatic for programs that use the standard time functions, outside of the kernel.

Mmm.. I wonder if I can also get the UTC offset somehow from within the kernel, for the Hijack "clock" function? Dunno how to parse the (largish) zoneinfo files, so I'd rather not do it that way.

???



Posted by: tfabris

Re: Time Zone Setting - 19/02/2004 22:12

You da man, Mark. Nice sleuthing.
Posted by: tonyc

Re: Time Zone Setting - 19/02/2004 22:21

I think that perhaps I should have Hijack detect accesses to /etc/localtime, and automatically redirect those back to the correct zoneinfo file, based on the path traversal info from the 4 bytes of the flash savearea. Once this is working, timezone info should be automatic for programs that use the standard time functions, outside of the kernel.
That would be a *really* sweet hack.

BTW, I figured out why I wasn't seeing the flash savearea change when I looked at it earlier... I didn't have a /usr/share/zoneinfo directory, because I use my own /usr partition and don't let the player overwrite it. So if you don't have anything there, the player has a default list of time zones, but when you select one, there are no subdirectory entries to point to, hence no change in the flash.
Posted by: mlord

Re: Time Zone Setting - 19/02/2004 22:34

Okay, the libc on the player does indeed look for /etc/localtime, or at least that string exists in the libc file. But I don't seem to have much luck getting it used.

One can do this in bash: export PS1="\@> "
which sets the command prompt to be the 24-hour timeofday, but on my player it just gives me UTC, even when I create /etc/localtime appropriately.
Ditto for the "ls -l" command.

Mmm.. I wonder why it's not working?

-ml
Posted by: tonyc

Re: Time Zone Setting - 19/02/2004 22:39

Mmm.. I wonder why it's not working?
Geez, if you're stumped.... What you're doing is pretty much what I did before I posted about this... Including the /etc/localtime symlink. Very strange.
Posted by: mlord

Re: Time Zone Setting - 19/02/2004 22:42

Okay, I just uploaded strace to watch what the code is actually doing when getting the date/time.

Instead of /etc/localtime, the "ls -l" command is trying to access /home/empeg/arm-empeg-linux-new/etc/localtime

So, I created that directory, and symlink'd the file to /etc/localtime (which I've already created previously), and now.. voila! Correct local time is now displayed!

So, I could have Hijack just translate accesses for /home/empeg/arm-empeg-linux-new/etc/localtime into the correct path under /usr/share/zoneinfo, I suppose.

Tricky that, because Hijack would have to sort the directories (the flash savearea values appear to depend upon alphabetic directory order, which is not the REAL directory order..). That gets a little bit ugly to figure out, but perhaps not too bad.

Anyone want to write some code, which given the four indexes (from savearea), traverses the directory tree to find the correct zone file?

Cheers
Posted by: tonyc

Re: Time Zone Setting - 19/02/2004 22:57

Hm, from userland it's just: 

#include <dirent.h>



int main()

{

	struct dirent **namelist;

	int i, n;



	n = scandir(".", &namelist, 0, alphasort);

	if ( n < 0 )

		perror("scandir");

	else

		for (i=0; i < n; i++ )	printf("%s\n", namelist[i]->d_name);

}

.
Not sure if that helps you, though.
Posted by: mlord

Re: Time Zone Setting - 19/02/2004 23:13

No help there, thanks.

I already do directory parsing in khttpd, but it has ocurred to me that there's another way to do this.. I'll just observe which zonefile the player software accesses, and remap to that!

Should be ready shortly.

-ml
Posted by: tonyc

Re: Time Zone Setting - 19/02/2004 23:19

Should be ready shortly.
Awesome.
Posted by: brendanhoar

Re: Time Zone Setting - 19/02/2004 23:34

Hijack get more truly evil every day. I like it.

-brendan
Posted by: mlord

Hijack v375 - 20/02/2004 00:25

Okay, it's building now, and should be uploaded within 10 minutes or so.

Hijack v375:

-- automatically set up userspace timezone stuff. If you change the timezone in the player, Hijack will not "see" the change until after the next player restart.

-- the timezone information is NOT available until after the first startup of the player after boot.

Cheers
Posted by: tfabris

Re: Hijack v375 - 20/02/2004 00:45

the timezone information is NOT available until after the first startup of the player after boot.
So no help for the TTSclock people (oh well ), but scads of help for what Tony is doing.
Posted by: tonyc

Re: Hijack v375 - 20/02/2004 05:58

Three things..

1. Looks to me like the index.html for Hijack's web page is missing a couple of the relative patches. The patches still exist but I think you probably just clobbered the HTML or something.
2. Just gave it a shot and it works flawlessly.
3. I think it's absolutely sick that you can do something like this in 20 lines. I sincerely hope you don't get paid by the SLOC for your kernel work!
Posted by: tms13

Re: Hijack v375 - 20/02/2004 06:26

Mark, could we also initialize zoneinfo from config.ini if there's an appropriate entry there? This would allow us to get the right timezone for TTSClock. Or are you saying that we have it by that stage (I'm not sure exactly what stage you mean by "not until after the first startup of the player after boot").

I ought to get around to installing strace myself - I set TZ for TTSclock, but that seems only to affect the DST selection and not the offset; I don't understand why.
Posted by: peter

Re: Time Zone Setting - 20/02/2004 07:22

/home/empeg/arm-empeg-linux-new/etc/localtime
*tut* Spot the mis-crosscompiled glibc

There's something else affected by this problem as well: locale maybe?

Peter
Posted by: tms13

Re: Hijack v375 - 20/02/2004 08:06

Mark - the Hijack page is missing diffs for v372-374. I want to try 375, but I'm scared of what might be in it
Posted by: mlord

Re: Hijack v375 - 20/02/2004 08:50

I've just re-uploaded the index.html for the Hijack site -- last weekend I had to rebuild my local server from backups, and seem to have somehow reverted to an earlier copy of that file.

The relative diffs are all intact, so no worries there.

We could do the timezone fix much earlier -- at config.ini parsing or even before. But this will require adding kernel code to parse the /usr/share/zoneinfo/ subdirectories as described above. A better fix, for sure, but I just don't have the time to do it right now. Patches accepted, though.

Longer term, I thing we should just replace the Linux distron on the Empeg with a newer one, with correct libc (or binary-patched to fix that bug!), and a BusyBox plethora of utilities, including a telnetd (native to BusyBox-1.00pre).

We could fix the zone link then, too, using a custom init that parses /dev/empeg_state to extract the zone path, and then set up appropriate ram-based symlinks and a TZ variable.

Cheers
Posted by: mlord

Re: Hijack v375 - 20/02/2004 09:02

Mark, could we also initialize zoneinfo from config.ini if there's an appropriate entry there?
Ahh.. I should really pay more attention. So you're asking for a way to set the zoneinfo from config.ini, so that it can be made available earlier. Okay, I can do that, but this setting would then either:

(1) completely override the player's zoneinfo setting, so the player might end up using a diiferent timezone than the add-on software. OR...

(2) the player's timezone could replace the config.ini timezone once the player continues startup (after config.ini, unfortunately).. But this could also be a bit confusing if config.ini and the player's view disagree.

I don't like either solution, myself. Really, Hijack should just do the directory lookup itself earlier on, but.. see my previous posting for that. Patches accepted.

Cheers
Posted by: mlord

Re: Hijack v375 - 20/02/2004 09:09

Hey, I'm wrong about the alpha-sorting!

The indexes from the savearea are REAL directory entry indexes, counting from zero. Canada is 28 (the 29th entry), and Eastern is 4 (the 5th entry). This makes parsing in kernel space much much simpler.

Maybe I'll have a go at it.

Cheers
Posted by: mlord

Hijack v376: zoneinfo from config.ini time - 20/02/2004 10:47

Okay, Hijack v376 now sets the timezone link from within the config.ini parsing, before launching apps etc..

Should be available for download in 10-15 minutes from now.

Cheers
Posted by: mlord

Re: Time Zone Setting - 20/02/2004 16:01

Speaking of embarassing paths.. here's one from an strace I did a while back:
/home/empeg/toolchain_binutils-2.10.1_gcc-2.95.3-arm010218_glibc-2.1.3/arm-empeg-linux/share/zoneinfo/Universal
Dunno if it's still there now. Probably.

Cheers
Posted by: mlord

Re: Time Zone Setting - 20/02/2004 16:05

Yup, still there in v2-final:
/home/empeg/toolchain_binutils-2.10.1_gcc-2.95.3-arm010218_glibc-2.1.3/arm-empeg-linux/share/zoneinfo/Universal
Posted by: peter

Re: Time Zone Setting - 21/02/2004 14:51

/home/empeg/toolchain_binutils-2.10.1_gcc-2.95.3-arm010218_glibc-2.1.3/arm-empeg-linux/share/zoneinfo/Universal
That is indeed quite embarrassing. Cross-compiling entire Linux userlands is not the black art it once was, but the glibc in car-player images hasn't been updated since before I joined Empeg, if ever. It's still somewhere waay down on my v3 wishlist to cross-compile an entire brand-new userland for car-players, with more modern bash, coreutils, glibc. and so on -- at least, if someone fixes my GCC3 PR from like 2001 or if I can get the Apple no-clone-structors patch working for 3.3.[23] for ARM. (The "real-world example" in the PR is, of course, the car-player software.)

Peter
Posted by: tonyc

Re: Time Zone Setting - 21/02/2004 15:12

somewhere waay down on my v3 wishlist
Would donations (in the form of libations at The Wrestlers, of course) have any chance of raising this higher on your list? Or do the problems you mentioned preclude you from being able to do it even if you were so motivated?
Posted by: peter

Re: Time Zone Setting - 21/02/2004 15:46

Or do the problems you mentioned preclude you from being able to do it even if you were so motivated?
I guess it'd be possible to stick with GCC 2 and just update the rest of userland: you wouldn't get glibc 2.3 but you'd get 2.2, and everything else probably still compiles with GCC 2. With a bit of care this could even be done without breaking all third-party userland software: it doesn't bother me whether the backwards compatibility stuff in glibc is bloaty or not, because the player is statically linked with (potentially) entirely its own build of glibc.

What exactly are you hoping for from a new userland?

Peter
Posted by: mlord

Re: Time Zone Setting - 21/02/2004 17:19

For a new userland, I'd simply use the latest BusyBox (www.busybox.net), with nearly *everything* configured for busybox. This would give us nearly 200 very useful userland commands, all in one executable (with lots of symbolic links to itself) that might weigh in at around 500-700KB total. Or slightly more if statically linked with uclibc (instead of glibc).

Built-in webserver, telnet daemon, networking utilities, shell utilities, SHELL (ash), init, etc..

Practically nothing else required except the stuff under /empeg/ and a few files in /etc/ to have a VERY functional Linux system. In less space than the current bash executable alone!

Cheers
Posted by: tonyc

Re: Time Zone Setting - 21/02/2004 20:43

What exactly are you hoping for from a new userland?
Recently I wanted to try out a memory debugger/profiling type application but it required a more recent glibc than the empeg has. I also figure there would be performance and stability benefits from using more recent versions, but I don't keep up well enough to know if it would be a significant improvement or not.
Posted by: alex25

Re: Time Zone Setting - 22/02/2004 04:08

I'm not sure if it's worth mentioning. Maybe you talk about something completly different, but my second harddrive on my spare empeg player still contains and runs the full blown debian installation in a chrooted environment. Including apache, php, sshd, ... (http://empeg.homelinux.com)
I'm always using the latest testing debian distribution which includes at the moment gcc-3.3 and libc-2.3.2. The latest apt-update I've done was about 1 month ago. Does this help in any way?
Posted by: peter

Re: Time Zone Setting - 22/02/2004 04:28

For a new userland, I'd simply use the latest BusyBox
Now there's a good idea. I'll look into it. Another thing I'd get round to investigating if v3 were a proper project is linking the player itself against uclibc -- there must be a load of stuff in glibc, even a static link, that we don't need. Locales, for instance. Anyone know whether uclibc gets timezones right?

Peter
Posted by: mlord

Re: Time Zone Setting - 22/02/2004 09:33

I have not looked in depth at the uClibc timezone code, but tere are some notes taken from the "glibc vs. uClibc" readme file:
time functions
--------------
1) Leap seconds are not supported.
2) /etc/timezone and the whole zoneinfo directory tree are not supported.
To set the timezone, set the TZ environment variable as specified in
http://www.opengroup.org/onlinepubs/007904975/basedefs/xbd_chap08.html
or you may also create an /etc/TZ file of a single line, ending with a
newline, containing the TZ setting. For example
echo CST6CDT > /etc/TZ
3) Currently, locale specific eras and alternate digits are not supported.
They are on my TODO list.

wide char support
-----------------
1) The only multibyte encoding currently supported is UTF-8. The various
ISO-8859-* encodings are (optionally) supported. The internal
representation of wchar's is assumed to be 31 bit unicode values in
native endian representation. Also, the underlying char encoding is
assumed to match ASCII in the range 0-0x7f.
2) In the next iteration of locale support, I plan to add support for
(at least some) other multibyte encodings.

locale support
--------------
1) The target for support is SUSv3 locale functionality. While nl_langinfo
has been extended, similar to glibc, it only returns values for related
locale entries.
2) Currently, all SUSv3 libc locale functionality should be implemented
except for wcsftime and collating item support in regex.
Posted by: peter

Re: Time Zone Setting - 22/02/2004 17:53

/etc/timezone and the whole zoneinfo directory tree are not supported.
I've a nasty feeling that this is another way of saying "it doesn't work out for itself when DST starts and ends", which would be a real chore to implement ourselves.

Peter
Posted by: Mataglap

Re: Time Zone Setting - 22/02/2004 18:28

When DST ended a while ago and I didn't have to change the clock in the car, that was nice. Really appreciate that. It'd be a bad feature to loose.

--Nathan
Posted by: mlord

Re: Time Zone Setting - 22/02/2004 19:03

From looking it over very quickly, it appears that so long as we supply it with a Posix style TZ variable (which can specify DST), then it might do the Right Thing. Gotta try it on something first to make sure.

Failing that, I could implement an /etc/localtime (zone file) parser to do the Right Thing rather quickly -- if we ignore leap seconds. Basically the same 20-30 lines I just put into Hijack could be dumped into a custom add-on library to handle this for the player.

Cheers
Posted by: tonyc

Re: Time Zone Setting - 22/02/2004 20:44

Yeah, that's not a bad idea for development and experimentation purposes, but I like where this conversation is going in terms of thinking about getting the standard empeg distribution updated. I can always run another environment built from scratch, but it sounds like a little elbow grease could net a much leaner and more powerful userland environment that everyone would benefit from.
Posted by: tms13

Re: Time Zone Setting - 24/02/2004 11:40

In reply to:

It appears to be stored in bytes 0x51 - 0x53 of the flash savearea.

Would somebody update the relevant part of RioCar.Org with this, please? The interpretation can be found elsewhere in this thread; in brief, it's a series of directory indices.
Posted by: tonyc

Re: Time Zone Setting - 24/02/2004 11:57

Would somebody update the relevant part of RioCar.Org with this, please?
Will do. I've found other bits in there too for things that aren't listed, and have updated my own copy of that Excel spreadsheet that has the flash map... Just haven't uploaded it yet. I've also answered some of the things I wasn't sure about... I'll get that page updated ASAP.
Posted by: Daria

Re: Time Zone Setting - 04/07/2011 01:10

Originally Posted By: peter
[color:"#FFA500"]<blockquote>/home/empeg/toolchain_binutils-2.10.1_gcc-2.95.3-arm010218_glibc-2.1.3/arm-empeg-linux/share/zoneinfo/Universal</blockquote></font>That is indeed quite embarrassing. Cross-compiling entire Linux userlands is not the black art it once was, but the glibc in car-player images hasn't been updated since before I joined Empeg, if ever. It's still somewhere waay down on my v3 wishlist to cross-compile an entire brand-new userland for car-players, with more modern bash, coreutils, glibc. and so on -- at least, if someone fixes my GCC3 PR from like 2001 or if I can get the Apple no-clone-structors patch working for 3.3.[23] for ARM. (The "real-world example" in the PR is, of course, the car-player software.)
<br>
<br>Peter
<br>


Totally fixed... for gcc 4.4 :\