Fixing vfdlib for v3.00

Posted by: pim

Fixing vfdlib for v3.00 - 18/07/2003 06:10

Are there any volunteers for fixing vfdlib on v3.00?
It does not seem the original author, Richard Kirkby
is on this board.

The rendering of text using this library does not seem to work
anymore with the fonts supplied with v3.00-alpha3.

As a workaround one could probably copy the old fonts
to a separate location and use those, but this is tedious.

Pim
Posted by: tonyc

Re: Fixing vfdlib for v3.00 - 18/07/2003 06:48

Richard hasn't posted since February (his BBS name is kirkis) but he's always been good about responding to emails. I spoke with Tony F. yesterday about this font issue and said that if the Empeg guys could provide me with the specs for the new font format, I could give it a go.
Posted by: tfabris

Re: Fixing vfdlib for v3.00 - 18/07/2003 08:53

Yeah, I suspect that what happened is that 3.0 has revamped the font files to use unicode, so if you re-use the player fonts in VFDlib, the 3.0 fonts won't work. The solution would be, as you said, to upgrade VFDlib to support both possible font sets and autodetect which is present. We need only to know the spec for the new font files, and I've already placed a request for this with The Guys. They're pretty busy with some Real Life deadlines right at the moment, so I wouldn't expect a reply immediately.

There's no chance that someone could simply look at the new font files and reverse-engineer the spec, is there? Maybe it's as simple as "the file format is the same, you just have a larger unicode index instead of an 8-bit index"?
Posted by: tonyc

Re: Fixing vfdlib for v3.00 - 18/07/2003 09:02

There's no chance that someone could simply look at the new font files and reverse-engineer the spec, is there? Maybe it's as simple as "the file format is the same, you just have a larger unicode index instead of an 8-bit index"?
Possible, but more difficult, and if the team isn't in a hurry to release 3.0, I'm not in a hurry to fix vfdlib for 3.0. I'd rather have a spec in front of me than count on Doing the Right Thing with reverse engineering. Or at least someone with a blue nickname to say exactly what you just said about a larger unicode index.
Posted by: pim

Re: Fixing vfdlib for v3.00 - 18/07/2003 09:16

If I recall correctly, the new fonts are smaller. So they might
be encoded more efficiently, while supporting larger character
sets at the same time.

According to this thread, the character sets haven't grown, though.

Pim
Posted by: tonyc

Re: Fixing vfdlib for v3.00 - 18/07/2003 09:56

Your analysis would seem to be correct about new fonts being more efficiently encoded:

17588 large.bf
15626 large.bf.new
9272 medium.bf
8502 medium.bf.new
6500 small.bf
6102 small.bf.new

My guess is that efficiency comes at the expense of ease in decoding, but I'm sure it's not rocket science.
Posted by: Daria

Re: Fixing vfdlib for v3.00 - 18/07/2003 13:14

There's no chance that someone could simply look at the new font files and reverse-engineer the spec, is there? Maybe it's as simple as "the file format is the same, you just have a larger unicode index instead of an 8-bit index"?


I'm sure there's a chance it could be done, but I bet most of the people who could do it are either busy or insufficiently special.
Posted by: tonyc

Re: Fixing vfdlib for v3.00 - 20/07/2003 08:58

Well, I'm not the best reverse-engineer on the planet, but I can find no obvious file format for the new files. If anyone else wants to try, be my guest, but unless someone can get me a format spec (or the old spec and tell me what the differences are) I won't be the one patching vfdlib.
Posted by: tfabris

Re: Fixing vfdlib for v3.00 - 20/07/2003 10:28

Cool, thanks for trying.

And by the way, I sent Tony *just* the font files, not a full copy of 3.0.
Posted by: tonyc

Re: Fixing vfdlib for v3.00 - 20/07/2003 11:28

And by the way, I sent Tony *just* the font files, not a full copy of 3.0.
Rub it in why don't you.
Posted by: rob

Re: Fixing vfdlib for v3.00 - 20/07/2003 14:16

This is a bad week to ask for, well, pretty much anything - but hopefully next week I can dig out the font spec. I also don't mind if someone sends 3.0 to Tony for app fixing purposes.

Rob
Posted by: tfabris

Re: Fixing vfdlib for v3.00 - 20/07/2003 14:21

Thanks, Rob.
Posted by: kirkis

Re: Fixing vfdlib for v3.00 - 21/07/2003 18:01

Hi Guys,

I'm still around but have zero time to devote to empeg development. I couldn't even begin to try fixing the problem without v3.00 anyways.

If someone is kind enough to take on the task of doing the fix I'd be happy to put it into a new vfdlib release.

Richard
--
Posted by: Daria

Fixed vfdlib for v3.00 - 18/09/2003 21:18

Apply the attached diff. Enjoy, people lucky enough to have the rest of the alpha
Posted by: Daria

Re: Fixed vfdlib for v3.00 - 18/09/2003 21:21

And actually, in fairness, it's not done. I'm still trying to figure out what the deal with the last 32 characters is.
Posted by: Daria

Re: Fixed vfdlib for v3.00 - 18/09/2003 21:51

In order for this to make sense, see previous patch.

unk2 is an offset. At that offset appears to be a table, which appears to be a pair of char for each entry. I'm not sure what the purpose is, since it appears the first char in the pair counts up from 0, and the other is always 0.

At least, looking at 13pixel.bf and large.bf both seem to indicate this.

Hm. Looking at 13pixelnumeric.bf, I guess the table is actually one which tells us at what offset to fill in the character.

Ok, a little more coding...
Posted by: Daria

Re: Fixed vfdlib for v3.00 - 18/09/2003 21:55

Hm. No, I understand now. Finished fix shortly.
Posted by: Daria

Fixed better vfdlib for v3.00 - 18/09/2003 22:31

Ok, this one should deal correctly with the packing. It appears to work with the v2 and the v3 fonts I have. It's not clear I have all the v3 fonts.
Posted by: pim

Re: Fixed vfdlib for v3.00 - 19/09/2003 05:39

Hey that's great! Unfortunately I won't be able to try it out, as I'm just packing for a short holiday trip to the US (Chicago & New Orleans). I will certainly make a new build of my vfdlib-enabled rsync port when I return.

Thanks,
Pim
Posted by: Daria

Re: Fixed vfdlib for v3.00 - 19/09/2003 08:37

Actually, I was lying in bed thinking about it, and there are still 2 bugs. I'll try to get a corrected patch out before I leave for Ohio.
Posted by: tonyc

Re: Fixed vfdlib for v3.00 - 19/09/2003 08:40

Actually, I was lying in bed thinking about it, and there are still 2 bugs. I'll try to get a corrected patch out before I leave for Ohio.
Great stuff! I had planned to restart emphatic development soon so I'll give this a shot once you've got the bugs worked out.
Posted by: Daria

Re: Fixed vfdlib for v3.00 - 19/09/2003 08:42

One is easy. free(cOffset); before the font register routine returns.

The other should also be easy, I'm testing it now.
Posted by: Daria

Fixed vfdlib for v3.00 ter - 19/09/2003 08:44

This one won't write into random memory it shouldn't when it finds a character that shouldn't be mapped.
Posted by: julf

Re: Fixed vfdlib for v3.00 - 19/09/2003 12:29

In reply to:

I'm just packing for a short holiday trip to the US (Chicago & New Orleans)



New Orleans OK, but holiday in chicago?
Posted by: wfaulk

Re: Fixed vfdlib for v3.00 - 19/09/2003 12:44

From what I hear, it's a toddlin' town.
Posted by: mwest

Re: Fixed vfdlib for v3.00 - 19/09/2003 12:56

I saw a man, he danced with his wife.
Posted by: tonyc

Re: Fixed vfdlib for v3.00 ter - 23/09/2003 18:41

Okay, vfdlib (as written) doesn't seem to be working for me with 3.0 fonts. Here's where vfdlib_registerFont bombs out:


if (bfHeader.version == 2) {
fseek(fp, bfHeader2.offset, SEEK_SET);
if (fread(cOffset, sizeof(CharOff), totalChars, fp) < totalChars) {
fclose(fp);
return -1;
}
}

.

I threw in some debug and the fread() call seems to be reading 115 chars, and totalChars seems to always be 200.

I tried having it continue on regardless of what fread() returns, and after everything loads up, this is what I get in emphatic:


The characters all seem to be formed properly (individually) but the character map seems out of place... Watching lyrics scroll by looks pretty funny.

Any idea what's up? I'm using small.bf, medium.bf, and large.bf (the 3.0 versions.)
Posted by: Daria

Re: Fixed vfdlib for v3.00 ter - 23/09/2003 20:20

I'll look later. Maybe when I cleaned it up I broke something.
Posted by: Daria

Re: Fixed vfdlib for v3.00 ter - 23/09/2003 20:42

Oh, sh*t.

It works when I run gpsapp_host on a PC, and fails in the way you describe on an empeg I dropped the fonts on.
Posted by: Daria

Re: Fixed vfdlib for v3.00 ter - 23/09/2003 20:50

Ok, I know why, and given what I was thinking earlier today it makes sense. It's not a table of 2 chars, it's a list of shorts, so presumably it is some encoding supporting more than 256 characters.
Posted by: Daria

Fixed vfdlib for v3.00 final try? - 23/09/2003 20:53

Ok, this time for sure?
Posted by: tonyc

Re: Fixed vfdlib for v3.00 final try? - 23/09/2003 22:26

Splendid. That did the trick!

Awesome work. Your sudden inspiration has even inspired me to play with emphatic a little bit.
Posted by: Daria

Re: Fixed vfdlib for v3.00 final try? - 23/09/2003 22:30

I need to finisdh some work stuff, and then I hope to do something more. I don't know what yet.
Posted by: tonyc

Re: Fixed vfdlib for v3.00 final try? - 24/09/2003 09:59

I need to finisdh some work stuff, and then I hope to do something more. I don't know what yet.
Hmm. In the userland arena, I'm not sure there's anything I'm particularly itching for. Most of the empeg things I'd want would be API changes in Hijack to allow for multiple apps to use the display, allow "context switching" between them, etc, a la some of the stuff that was discussed in the Hijack Wishes for 2K3 thread. I don't know if you're much into kernel hacking, but this is #1 on my empeg wish list. There was a lot of good discussion on ways to create a more flexible graphics/input API, but nobody really agreed on one particular way to do it.
Posted by: Daria

Re: Fixed vfdlib for v3.00 final try? - 24/09/2003 10:14

Well, gpsapp can always use some love, but it's not clear if atttempting to do anything to gpsapp+roadmap is the right way, or trying to rewrite the display stuff would be better. And then, how to do routing... I was never good with graph theory, unfortuntely.
Posted by: tonyc

Re: Fixed vfdlib for v3.00 final try? - 24/09/2003 10:20

I was never good with graph theory, unfortuntely
Jan is pretty good in that arena though, isn't he? Does he still have plans to remain active in gpsapp development? I haven't seen him around the BBS much lately.
Posted by: Daria

Re: Fixed vfdlib for v3.00 final try? - 24/09/2003 10:27

I don't know if you're much into kernel hacking


I used to a lot of it. I still do it least as much as is necessary to debug OpenAFS, which over the last few days has been a lot due to how lookups works and how we support rewriting part of the path based on the OS you're running (basically).

But part of the reason I've taken up the Mac gauntlet is massive disappointment at what's happened with Linux 2.6. I'm not interested in doing weird stuff so I can have the capability of having groups of processes running with the same unix uid having disjoint, possibly conflicting, groups of authentication credentials. What we did before was really a hack, and it won't work anymore without an even more heinous hack to allow it (Linux has no loadable syscall interface, and changes happened to make it hard to load a syscall anyway). That doesn't bother me. What bothers me is that basically people seem uninterested in either taking any interface offered which will solve the problem, or providing one, or suggestions to what they want, if they don't like what they've seen.

jaharkes did a patch for the issue which consisted of something like 47 lines of changes (add/change/removed lines); Really, having a small blob in the task structure to store presistent data which gets copied on fork/clone doesn't seem like a huge burden. Yet, here we are.

So, I'm basically ignoring the other issues which need to be dealt with before OpenAFS will run on Linux 2.6 (without the missing functionality, but it would still be able to read and write files); Either someone else will deal, or one of my jobs will force me to deal, or I'll blissfully not care.

Hijack, at least, is probably interesting enough to overcome the "I'm bitter at Linux", if I had something to do which might be interesting or useful. But I suspect I'd do poorly at implementing a graphics API. I might try anyhow.
Posted by: Daria

Re: Fixed vfdlib for v3.00 final try? - 24/09/2003 10:50

Don't know. Should probably ask, but I suspect he's been busy, and I know I've been busy.
Posted by: tonyc

Re: Fixed vfdlib for v3.00 final try? - 25/09/2003 18:04

I don't know what yet.
I just thought of something for you to do with your empeg hacking time... vfdlib has bitmap support, but only if you load the bitmaps up in its own format (BITMAP_4BPP, etc.) I asked Richard awhile back to think about adding in libjpg or libpng support such that you can create a vfdlib bitmap from JPG or PNG files. This would be real sweet for a feature I'd like to put into emphatic (scrolling album covers.) How hard do you think that'd be?
Posted by: Daria

Re: Fixed vfdlib for v3.00 final try? - 25/09/2003 21:13

Funny you asked. I wonder what I did with the colormap reducer I did when I thought it might be interesting to crop USGS maps to display as background for gpsapp... before proving to myself that 128x32 was simply too small for it to be worthwhile.

So, yeah, that's the thing. How do you allocate a colormap that makes sense? I don't know that I'm clever enough to figure out how to dither without doing something that will be a pig.
Posted by: tman

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 07:18

Somebody did get the GD library working I believe. Can't quite remember what application it was actually in however.
Posted by: tonyc

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 07:34

So, yeah, that's the thing. How do you allocate a colormap that makes sense?
*stares blankly*

I guess I naively thought that the JPG/PNG libs themselves would have options available for color-depth reduction and dithering. But upon further inspection I guess that's not the case. I think the GD graphics library I use in empan takes care of things automagically.

I'm not sure that dithering will even make much of a difference on a 128x32 screen. Maybe just calculating the intensity of each pixel and converting that to the closest "empeg color" would be a good start? So if pixel A is (255, 0, 0) and pixel B is (0, 0, 255) they'd look the same on the empeg display, but probably both be dimmer than a pure white pixel (255, 255, 255). Not sure how beautiful the results would be, but for a first crack at things, I'd try that, and then maybe look into how tough it'd be to add a dithering algorithm.

A few prayer sessions to the Google Gods found me this

http://thorkildsen.no/faqsys/docs/dither.c

Any help?
Posted by: tonyc

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 07:39

Somebody did get the GD library working I believe. Can't quite remember what application it was actually in however.
I can't take credit for making it work, but I did use it in 3 of my apps. I was trying to avoid using GD in emphatic because it's a real dog performance-wise.
Posted by: Daria

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 09:33

I'll have to find whatever provides the px_ functions because I think px_wgrey and px_rgrey at least are things I'd want. Also, have you displayed the whole palette on the empeg? You can display more than 4 colors but iirc not many more than 4 were useful.

Is ~4 shades of "grey" going to be useful for displaying an album cover?
Posted by: tonyc

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 10:22

Also, have you displayed the whole palette on the empeg? You can display more than 4 colors but iirc not many more than 4 were useful.
From what I understand, while the display itself supports more than 4 shades, Hijack's sreenbuf is just 2048 bytes, so > 4 shades can't be represented. I think the stuff Toby does in the visuals involves straight assembly to the display which allows more "shades." And from what I've seen, other shades wouldn't be very useful due to too much flickering.
Is ~4 shades of "grey" going to be useful for displaying an album cover?
I think if it scrolled through the album cover vertically, yes. I did a few with empan and they were recognizable, though obviously not photorealistic. To give you an idea of what to expect, if you have emphatic installed, bring up its menu and hold down the knob while the "On" menu item is selected for a couple seconds, and let go. You'll see the emphatic "easter egg" which will show some credits and then scroll an image from the Simpsons. That's about the level of detail I'd expect for album covers.
Posted by: tfabris

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 10:36

Okay, that easter egg is TOO COOL.
Posted by: genixia

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 10:47

From what I understand, while the display itself supports more than 4 shades, Hijack's sreenbuf is just 2048 bytes, so > 4 shades can't be represented. I think the stuff Toby does in the visuals involves straight assembly to the display which allows more "shades."

I don't remember seeing any kernel mechanism that would allow him to do so - 3 shades + black appears to be it.
Posted by: tman

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 11:09

Umm... 128 (width) * 32 (height) * 4bpp = 2048 bytes * 8...
Posted by: tman

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 11:11

The EMPEG_DISPLAY_WRITE_PALETTE ioctl can select the 4th palette which is a 1 to 1 mapping. Most of the values don't work very well though.
Posted by: wfaulk

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 11:12

It's 4bpp, not 4Bpp.

That is: 128 * 32 * 0.5B = 2048B

Edit: On second thought, I think I don't understand what you're saying. Where does the *8 come from?
Posted by: tman

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 11:16

The display buffer is 2048 bytes and 8 bits per byte so you get 16384 bits in total.
height x width x bits_per_pixel = 128 x 32 x 4 = 16384 bits as well
Posted by: tonyc

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 11:16

Umm... 128 (width) * 32 (height) * 4bpp = 2048 bytes * 8...
I echo Bitt's sentiments. What are you trying to say? Your math confirms what I was saying, which is that it only supports 4-bit color depth.

Edit: I am officially an idiot. 4bpp = 16 shades. I need more coffee.

Posted by: tman

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 11:22

It's two pixels packed into each byte in the screen buffer. Which means each pixel is 4 bits. This can represent 16 different values not just 4.
Normally the kernel driver will remap it so that it's only 4 distinct values but if you use the EMPEG_DISPLAY_WRITE_PALETTE ioctl then you can get all 16 "colours" on the display.
Posted by: tonyc

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 11:26

then you can get all 16 "colours" on the display.
Yep. But as I said before (trying to appear intelligent after screwing up the math so poorly) the other shades are pretty much worthless everywhere except in the visuals (where flicker can be considered "part of the effect.")
Posted by: tman

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 11:29

Toby did time dithering as well to increase the number of effective intensities. [edit]Hugo said that John tried out the shades again and found one more was okay. It's not used however.[/edit]
Posted by: tonyc

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 11:31

And I think somebody did retry the entire range of possible intensities and found that 1 or 2 of them weren't that bad actually.
Hmm. I'd be interested in seeing that.
Posted by: Daria

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 12:53

I had a program somewhere which just did a 'test pattern' of 16 bands of color. It would be easy to write again, which is good, since I may have tossed it.
Posted by: tonyc

Re: Fixed vfdlib for v3.00 final try? - 26/09/2003 13:24

I had a program somewhere which just did a 'test pattern' of 16 bands of color. It would be easy to write again, which is good, since I may have tossed it.
Well, regardless of whether it's 2bpp or 4bpp, I'd love to see JPG/PNG format in vfdlib.
Posted by: mlord

Re: Fixed vfdlib for v3.00 final try? - 27/09/2003 06:08

Do you have a Central?

I'm thinking of porting parts of Hijack to the Rio Central's kernel in hope of opening up that platform somewhat. Initially, just kftpd.c from the Hijack sources.

Problem is, I don't have a Rio Central. So if somebody else wanted to take over this mini-project..

Cheers
Posted by: Daria

Re: Fixed vfdlib for v3.00 final try? - 27/09/2003 09:46

No Central. The thought of getting one has crossed my mind, but I suspect I shouldn't flush any money until the oil spill cleanup is done, since I don't know how much of that the insurance will cover (they did cover 100% minus the deductable of the tank, oil, and oil dry removal the night of the incident).
Posted by: siberia37

Re: Fixed vfdlib for v3.00 ter *DELETED* - 15/10/2003 14:19

Post deleted by siberia37
Posted by: Daria

Re: Fixed vfdlib for v3.00 ter - 15/10/2003 14:20

The last patch I sent, weeks and weeks ago, worked fine. Can you debump this?