Compiling userland apps into kernel?

Hi,
I just got my red hat 7.3 box compiling kernels for the empeg. My next question is, is it possible to compile a userland amp into the kernel and have it automatticly run on init? this was getting rid of the preinit program?

Yes, but it would require a significant amount of effort, and is not really that great an idea anyway. There's a reason for the separation of kernel and user spaces.

So basiclly i have to install the preinit if i want a userland app running all the time binding to the hijack menu?

I'm biased. Someone else field this one.

Well, you need some kind of way to start up your app so it can bind to the menu.

OR.. you could tackle a project that has been needed for about 6months or so: I need somebody to figure out (code & test) a kernel function to launch a userland commandline, and wait (non-blocking poll) for it to exit, and retrieve the exit status.

Given that functionality, we could turn the whole thing around, and have Hijack launch the apps on-demand (menu item selected), using config.ini to bind menu entries to userland commandlines.

If it were all userland, it would be as simple as invoking execve() from libc. In kernelspace, there is also an "execve()", but the semantics differ ever so subtlely. I hacked at it briefly last winter, and got nowhere on the first couple of attempts, and dropped it for more pressing features that needed doing (so I could keep posting to get my count/title up higher!).

Cheers

Hi mark, thanks for the reply and the info. I don't think with my knowledge of c, and the trouble i'm having with the ftp command in linux i don't think i could ever make something like that, but at least i got a kernel to compile. I just need to figure out something i can add to my empeg, and i can't think of anything

So basiclly i have to install the preinit if i want a userland app running all the time binding to the hijack menu?

Preinit or a replacement init file. Picker is an old "select what to run at boot time" project and it included a script that replaced the binary init file. In the script you can start up various applications. If using the Picker you can select what to start (4 variations by default, more ifyou nest Pickers). Unfortunately I can't find it right now, Tony Fabris used to have a "Picker & VolAdjust for dummies" package with the Picker stuff and instructions on how to use it for different VolAdjust settings, but the location where it used to reside is empty.

I'm attaching an example of the script version of init though.

/Michael

For whatever reason, I was never able to get a script to work properly as init. That was the reason I wrote empeg-preinit. I figured it had to with the kernel not wanting to deal with a script and needing a binary. Maybe I was just a moron.

Keep in mind, though, that if you're using a Hijack kernel, it will look for /sbin/hijack (or is it /bin/hijack?) before it looks for /sbin/init, so you could put it there in order to keep from overwriting or renaming the empeg's original init.

The picker package was supplanted by Hijack, which is much more graceful about things. That whole package was designed around 1.03 and would fail on 2.0 if its instructions were followed to the letter (if I recall correctly), so that's why I'm not publishing it any more.

I really REALLY like Mark's idea for launching userland apps from a Hijack menu driven by config.ini. Anyone want to help him out with that idea?

Actually, the Picker itself works just fine with 2.0b11 & Hijack (though I suppose you mean your entire packaging for using it with VolAdjust) and I'm currently using it to start/not start various things at boot time, giving me different "profiles".

But yeah, to have Hijack menu entries to start/stop apps would be very sweet. Unfortunately I'm not at all familiar with programming that sort of stuff...

/Michael

Correct, the picker itself was fine. I forget which component of my packaging and instructions was wrong for 2.0, but I just didn't feel a need for re-doing it after Hijack came out. Figured if someone else wanted to do a nice, more generic distribution/readme for Picker, then they could.

How about just compiling an initial ramdisk in to the kernel and
have pre-init run from there? I know Mark's (very sensible) design
philosophy is to make installation of hijack as painless as possible for
the end-user, but so far this has meant some things going in the kernel
which might have been better in userland. Maybe having a way to ship
a userland program in the kernel (using initrd) would get around this
more elegantly than doing yet more stuff in the kernel.

Hi.

The empeg kernel already uses an initrd (that is loaded from flash). Don't know how good the kernel would handle a stack of initrds.

The other way would be to patch the kernel to boot from /dev/hda2, and install a new root filesystem there that contains the preinit and mounts the original root filesystem to run the player/original init. The tools to do this are readily available. You can even create a custom .upgrade file to initialize the hda2 partition, install the patched kernel and that could even do a multi-staged install IIRC, installing a "small" hda2 image first, which installs a userland app on the music partition and a "full" hda2 image next which install the preinit and a multitude of tools on the hda2 partition.

cu,
sven

Picker can be found at http://www.ioctlsoft.com/empeg/ though, as Tony points out, it is showing it's age.

Cheers,

Rex.

Please see my reply to this in the main programming thread.

John

Mark,

I don't understand your post about the execve function. Why can't you use it again?

You CAN use execve(), and probably HAVE to use it. But just keep in mind that the kernel version of this function (the one we're discussing) does not work 100% the same as the userland libc version. Very very similar, except it does not return anything (eg. the PID), but rather just does an exec() from the current process/thread to run the specified command (never returns, as far as I can tell).

So you have to fork the process first, and then somehow figure out the new PID that execve() transmogrifies itself into, and then figure out how to poll for (1) startup, and (2) completion of that thread.

The source code is there. Have a look.

Cheers

Umm, I don't want to try and teach Mark or anyone else how to suck eggs or anything.

But I think we have a slight mis-understanding somewhere.

My understanding is that the exec family of functions do not return any return code - unless they fail (in which case the code after the exec* call is then executed, so any return code test is really unnecssary - in the normal case the code after the exec* call never gets run, in the exeption case, it does).

However, the normal method of starting a process is to call the fork() function first, which makes two identical copies of the running process.

The fork() call should return the PID of the new process to the caller (i.e. the kernel), and the child process get a 0 return value from the same fork call.
Thats how the two identical processes running the same identical code can work out which is which after the fork() call.
(fork() may need to replaced with kfork() when the parent process is the kernel, I don't know).

If the fork() call does not return the PID to the parent then I guess the parent has a problem in that it cannot identify the process ID of the new process just created (which then procedes to call execve(...) ), so it doesn't then know the PID to wait for until it exits.

The only way around that would be by using a shared resource (e.g. shared file descriptor) or something similar to allow the two process to conduct a private IPC conversation to allow the child process to indicate to the parent process, its Process ID (the parent can open open a regular file before the fork and ensure it stays open), then the child can (after the fork) move the shared file pointer via lseek to a integer offset that represents the PID of the child - which the parent can also request via a lseek call is one such not so elegant but effective way of doing this that I have used in the past).

The other question I have is how does the current kernel lauch the init process it does after boot?
And can we not use that code, wrapped up in a nice routine?

init() is launched as a kernel_thread, similar to kftpd and khttpd. But the kernel doesn't need to wait for init() to exit (it never does), so there is no PID management required.

As for inter-thread communication, that's easy: variables within the kernel are GLOBALs, assuming one chooses the right parameters when creating the kernel_thread.

(under Linux, everything is a thread. Some threads share address spaces ("true threads"), and some don't ("processes")).

Cheers

Okay, has anyone done anything on this yet?
Christmas break is coming soon, and last year at that time I implemented a khttpd for Hijack while trapped at the in-laws over the holidays. With a similar scenario shaping up again this year, maybe I'll look at something simpler, like getting execve() to work for config.ini menu items..

Anyone else done anything on this??

Cheers

-ml

In reply to:

---

Well, you need some kind of way to start up your app so it can bind to the menu.

OR.. you could tackle a project that has been needed for about 6months or so: I need somebody to figure out (code & test) a kernel function to launch a userland commandline, and wait (non-blocking poll) for it to exit, and retrieve the exit status.

Given that functionality, we could turn the whole thing around, and have Hijack launch the apps on-demand (menu item selected), using config.ini to bind menu entries to userland commandlines.

If it were all userland, it would be as simple as invoking execve() from libc. In kernelspace, there is also an "execve()", but the semantics differ ever so subtlely. I hacked at it briefly last winter, and got nowhere on the first couple of attempts, and dropped it for more pressing features that needed doing..

---

Good to see you back, Mark. Check the bottom of this thread. Kim Covil wrote a patch a while back that I'm pretty sure does what you want. Basically you add an "exec=" line to your config.ini and it fires the apps up when Hijack loads config.ini. The next step is, I guess, to add menu item(s) to Hijack for each of the user's installed apps which, when selected, would call the hijack_exec() function that Kim's patch provides.

I tried to do this myself, but I get a kernel panic when selecting apps. You will undoubtedly have more success, I'm sure.

Anyway, the patch itself is here.

And if you're looking for other things to do in Hijack, the thread linked above has some of the big ones people have been asking for in your absence.

Also, the audio overlay stuff has improved to the point where I think we should think about making it part of Hijack. I have it delaying allocation of the buffers until the first overlay audio write. You may wish to double-check my work since I'm an amaterur when it comes to the kernel. Earlier issues with volume spikes have been fixed, too. Anyway, here's the patch.