Starting programs from the kernel...

I have built an experimental patch to execute a user program from within the kernel by adding a flag to config.ini... I haven't worked out passing parameters to it but it seems to work with the viewer program... upload the viewer binary to somewhere on the empeg, chmod 755 it and then add a line to config.ini

[hijack]
; full path to execute... (no parameters available yet)
exec=/drive0/bin/viewer

I am trying to get the code to work for the kftpd SITE EXEC command but it doesn't seem to work yet... will keep working on it... For the moment this might be an easier way to run user programs rather than having to put a custom /sbin/init or /sbin/hijack on the root partition (which gets wiped with an upgrade)... I haven't tested it with many other executables but would be interested on hearing whether it works with user binaries other people have written... I don't think it will execute scripts at the moment... will work on that when I work out how to pass parameters...

Cheers

Kim

PS patch is against hijack v236

Note there are issues with the patch...

because of the way hijack reads config.ini and processes the [hijack] section it tries to execute the programs every time config.ini is read... which would be every time the player is restarted... this confuses things a bit it seems...

Maybe the [hijack] section should be processed only once and marked with a flag...?

Cheers

Kim

Maybe an @Boot 'special' token is needed for config.ini options like yours?

This could be used to tell the Kernel [aka Hijack] to parse (and in your case exec the program indicated) this config.ini entry only once during boot up of the kernel and not anytime after that (i.e. whenever it feels the urge to reparse config.ini?)

I'm not sure if this is a easy mod to the Kernel or not, I would think some kind of global static variable which counts how many times config.ini has been read to date since rebooting and starts at 0 would be useful here - may already be such a beast lying around for you to use.

If its 0 then we read config.ini we know to honour the @boot entries otherwise we ignore them.

Eitherway I like it... very simple to have apps start at boot...

Although your suggestion would be a preferred way, a cheap hack could be to check the uptime and only exec @boot items if it's less than say 20 or 30 seconds.

Yeah... what I have at the moment in my work tree is an exec_once= and an exec=...

I have left the exec= in for the moment just in case I can think of some interesting reason for wanting something run every time the config is read (nothing has come to mind so far)

As far as making it an @boot special... could be a good idea... can't think of any other flag that might use it at the moment though. I do want to check and see whether @DC etc work with the exec= lines as I can see that being more useful... but mostly I want to try and work out what is missing from the code to make it work for SITE EXEC etc...

Cheers

Kim

As far as SITE EXEC goes... what I am currently seeing is that SITE EXEC does run the requested program... I have a couple of simple programs that just blitecho to the screen and I can see them working... and further if I make a program that doesn't exit but instead continuously blits to the display the ftp client sits there waiting for it to return as I would expect... and the player continues happily in the background and I can still start other concurrent ftp sessions.

The strange one is starting a process which forks... it seems the forked process gets killed off when the original process dies and the ftp server finishes processing the client's request. I think this may be because the new processes are spawned as child processes of the ftpd thread, which when it has returned from processing the parent process dies, and takes all it's children with it...

This seems to be what is stopping me from calling programs like viewer from site EXEC.

Any one have any clues on how to make a child process ignore its parent dying in a kernel context...?

Cheers

Kim

The parent might be sending a HUP signal to the child. Try masking that signal (signal 1, BTW) and see if that solves the problem. (I'm guessing this because it's a common issue on Unix machines when a long-term process's shell dies. There's a program called ``nohup'' that that you can use to prevent it from getting this signal.)

Here is the latest version of the patch I am playing with...

this one understands exec= and exec_once= options in the [hijack] section, it now also understands parameters, although there is a hardwired limit of 10 parameters at the moment... if this is a problem I will work on fixing it.

SITE EXEC from ftp works with parameters as well, though there is still the problem of programs that are supposed to fork and keep running being killed off when the parent process returns... and I haven't looked at trying to do anything useful with the stdout/stderr of the executed programs yet... and I don't think it reports execution failures correctly... I might work on that next, will be a nice way of running things like ps remotely...

Cheers

Kim

PS the patch is vs v236 of Hijack

Its a bit more subtle than that [at least in user space, kernel launched processes are probably at least as complex if not more so].

In order not to be killed by your parent dying, you need to do several things, other than just ignore SIGHUP.

In theory [from my working in this area 10 years ago now] you are supposed to become a process group leader - this is normally done in the child side of the fork() process.

You can become a process group leader by various arcane means, one common way is I recall using is by closing all open file descriptors you have open (including stdin,stdout,stderr), then ignoring SIGHUP, and then you call setpgrp() I think, then you (re)open a new controlling tty, then do the execve or whatever you are going to do.

The closing and opening of the stdin/err/out etc breaks the connection between the parent process'es process group leader (probably the kernel or init or something) and the forked child process.

The child process then becomes its own process group leader and will not die if the parent dies [all of this is what the nohup command does].


so, I'm no linux expert on this, but this may jog a few memories amoungst some others and get on the right track.

@boot could be used to launch a userland app starter (as per my comments in a related thread).

Or user could configure their systems so that apps are run immediately that the kernel starts (or shortly afterwards) by using it.

I can't think of any right now, but it could make the empeg 'pre-init' program less of a issue, in that if you can launch apps you want at boot up, then why bother with any other mechanism just yet?

Of course, we have to be able to make kernel space launched apps run without issues like any normal userland app would.

I am not sure if starting every app from the kernel is a good idea, starting most apps from userspace would help protect to some extent the system from badly behaved apps I suspect.

But right now, being able to launch anything reliably on demand is a real holy grail and I don't much care how we get there as long as its simple to deploy and configure.

The @boot flag wouldn't get read until config.ini is read... which isn't until the player tries to read it and is hence quite a bit after /sbin/init (or the hijack pre-init) gets run... This is why I have to have a different patch in my kernel to detect the presence of a tuner before the player starts rather than just running my previous userland program with this patch... (a work around discussed in a different thread)

With my comment about what would make use of the @boot flag I meant if it is only going to be placed in front of exec= options then I can't see when it isn't going to be wanted in front of exec=... ie it is the same issue as with my exec= vs exec_once=... unless you can think of a different config.ini option @boot would be useful in front of...?

So far running things with exec_once= seems pretty happy with my patch... the only issues I am having at the moment is from kftpd's SITE EXEC... and you could happily use it currently to start a userland launcher...

My current use is:

[hijack]
exec_once=/sbin/swapon /dev/hda6
exec_once=/drive0/bin/viewer

Cheers

Kim

Hmm...

I am not sure whether this is something I should be doing to things I run from SITE EXEC then... I would need to keep the stdout, stderr open if I want to get the results of the command back to the ftp client... and we want to know when the parent process has finished to be able to return a response to the client... so maybe this is something I should ignore from SITE EXEC and if people want things like viewer to run from SITE EXEC they have to add it into their code when they fork...

That would make things much easier... then all I need to do is work out how to get stdout/stderr from a process back to the ftp client, the site exec already seems to wait for the parent process properly...

I think I will ignore forking processes not behaving properly from SITE EXEC for a while...

Cheers

Kim

Yes, I see your point.

the only thing I could think where exec would be useful would be a small utility that popped up to tell you that config.ini has just been reread!
[kind of useless].


re: config.ini getting read late, Well in that case I guess you could make a call to read config.ini first before the player got a chance.

BTW: Does the kernel not start /bin/init itself? If so, how does it do it reliably is that a good place to look?
[if not, then who launches init if not the kernel?]

My comments above about becoming a process group leader might help you get started in the right direction.

The only thing you might have to watch out for is making sure that you start your process(es) after the kernel has started init - a lot of Unix things depend on init having a process id of 1 and any process that gets PID 1 that isn't init may get given more grief that it can handle later on.

If you find the source code to init you should be able to find the magic commands you need to do to get a userland app launched safely.
[and looking in the kernel code for what it does to run init and what it does BEFORE and after init is launches would make useful study too I think].

What I suggest you do here is consider connecting stderr (and stdout) in your fork code (child) to a TCP Stream Socket, then the kftpd thread can monitor the socket for 'data' (via a normal read of a file descriptor) and feed any data back that arrives on the socket back to the ftp client.
You could use a pipe - it would be easy to do in the fork() code and is what the 'tee' command does now, it would only need to be a one way pipe (i.e. from child to kftpd thread). Stdin is assumed to come from elsewhere or is not available (not opened) before execing the userland app.

the kftpd threa can know when the child died by the socket/pipe being closed (giving EOF to the readingend of the descriptor) and/or by the process generating a SIGCHLD.
[thus exiting the 'wait for child process to exit' loop].

Of course if the child launches a app that never exits and thus never closes stderr then you get a stuck kftpd client session.

But thats probably less likely than anything else going wrong and would be a good indication of a badly behavedTM application.

Yeah I had a look at how init was started when I made my first attempt at this patch... it is too much of a special case to take much from... All my code is based on the kmod calling of modprobe... if I don't mind site exec calls not allowing processes to fork and exist after the parent process has finished then I think for all other types of processes I have things working as they should (or will do once I have the stdout/stderr tied back properly) so I shall leave that case for the moment and try and work on the stdout/stderr stuff first...

The only place I can think of to put a call to read config.ini before the player starts is in the syscall that execs the player... ie check to see if the path being executed is the player executable and try and read config.ini first before letting the call continue... but I am not sure what else that would interfere with... I might give it a go later and if it works move my tuner-detect code back out of the kernel to a userspace program again...

Cheers

Kim

Okay, in user space, unless there's some requirement that the process actually have a controlling terminal, ignoring SIGHUP is enough. If it does need a controlling terminal, then it's unlikely to be running as a daemon anyway.

The proper way to write a program to do this sort of thing itself is to have the child after a fork call setsid(). setpgrp() does the same thing, more or less, but has been deprecated in favor of setsid(). You can close FDs 0, 1, and 2 if you want, but it's hardly required.

But, as far as I know, there only way to prevent a child from dying when his parent exits that wants to do so without rewriting the application is to use nohup. And it definitely doesn't close the controlling terminal, as it write's the process's output to nohup.out.

Hmm...

I might see if I can get nohup itself running on the empeg... and then try and use it to site exec the viewer to see if it fits the theory of what is going wrong...

Cheers

Kim

Userspace apps don't need a controlling tty, but its very hard for a userspace app not to have one due to the way most programs are started (from a tty or pseudo-tty).
Normally most apps have one or more of stdin,stdout and stderr connected to a real life tty somewhere (even if just for error logging).

You are right in that if you call setsid() it creates a new process group which breaks the connection from the process group it used to be part of.

But heres the kicker - if the new process group has ANY tty file descriptors open then that process is still listed as having a controlling tty.

If a process piror to a setsid call closed all open tty file descriptors, then subsequent to the setsid() call opens a (new) tty, then that tty will become the controlling tty for that app. You can prevent this but its requires some special IOCTLs to do so.

If the app never opens another tty in its life it will not have a controlling tty - ps normally shows this by putting a '-' as the tty name - thats why some processes like init show up this way.

Now, before the fork() is called a signal(SIGHUP,SIG_IGNORE) should be done to tell the kernel not to bother the app (or its children) with SIGHUPs unless they want to enable them.

Then if the user app is well behaved (or doesn't know about signals), it will only enable SIGHUP again in its code if its has a handler for the signal and wants to be told when its process group leader (parent or grand parent etc) has died.

Without a controlling tty then the kernel will not normally kill the child processes when the process who started them exits, when a process ignoring sighup becomes orphaned by its process group leader dying, then the kernel changes its parent pid to 1 (meaning init), which is why lots of background processes have 1 as the Parent PID in ps.
This is what nohup does as nohup launches the program then exits, forcing the process started to be inherited by init.

All this means is that in general userspace apps don't need to bother with signals esp. SIGHUP provided its handled for them by the launching program (e.g. kftpd / kim's code).

Hmmm. On my Unix machines, nohup actually calls sigaction() before exec()ing the process to be nohupped to ignore SIGHUP. (I don't have a Linux machine handy, but that's how it works under Solaris and OpenBSD; Solaris actually ignores SIGQUIT as well.) And that signal mask gets inherited by the exec()'d process.

Also, in both of those cases, nohup never exits; it just exec()s the new process, which is how it's able to keep its parent process be the shell (or whever called nohup) and not init, until the shell (or, again, whatever other process) exits.

It sounds, though, like you might be talking specifically about userland apps initiated by the kernel, so I'll have to default to you there.